GPM VLM Scanner (Internal)

This is the node that does the work: point it at a folder and a local vision-language model walks it, reads every image, and writes a JSON sidecar beside each one. No API, no key, no Ollama server - the same reason people run an abliterated 8B locally for prompt enhancement instead of paying per call. Offline and uncensored is the entire pitch.

It's also slow, and it's the node in this pack most likely to fight you.

What it produces

One .json per image, same basename, beside the picture - the sidecar convention training tools already use, except structured rather than a caption blob:

{
  "sdxl_person": "subject description",
  "sdxl_scene": "environment description",
  "pony_person": "...", "pony_scene": "...",
  "natural_person": "...", "natural_scene": "..."
}

The preset you pick decides which pair gets written: SDXL → sdxl_person/sdxl_scene, Pony → pony_*, Natural Language → natural_*. Keys you didn't ask it to touch are preserved, so GPM can share a sidecar with whatever else you keep there.

overwrite_mode is the collision policy and the difference matters. SKIP_EXISTING skips an image only if the selected family already has data - a file captioned for pony still gets scanned when you scan for sdxl. OVERWRITE_FAMILY rewrites that family's two fields and nothing else. So the sane first run is SKIP_EXISTING, and re-running it later only handles new and failed images.

How it works

It runs in a subprocess. The worker is a separate process that always exits when the scan ends. That's not architecture for its own sake - it's the VRAM release path. The model stays loaded for the duration of the scan so you're not reloading a 7B per image, then the process dies and your diffusion model gets its card back.

A watchdog watches progress, not time. There's no "give up after N seconds for the whole folder" knob. The parent watches sidecar files appear; if none appears for roughly three normal output intervals (minimum ~12 seconds), the current image is treated as stalled, deferred for that run, and left eligible for a later SKIP_EXISTING rescan. Far better than a folder timeout, since nobody knows in advance how long 3,000 images will take.

Only Qwen2.5-VL is validated. The family gate reads the filename and blocks anything the author hasn't confirmed for image-input correctness. Qwen3-VL and Gliese/Qwen3.5-VL are recognised but refused for scanning. That's this pack's real ceiling: among people captioning datasets, Qwen3-VL is what the conversation moved to, and GPM won't use it yet. Conservative and defensible - and annoying if the Qwen3-VL GGUF is already on your disk.

Inputs and outputs

• root_folder - walked recursively. Supported: .jpg, .jpeg, .png, .webp, .bmp.
• prompt_preset - which family's two fields get written: built-in SDXL, Pony, Natural Language, or any user preset you made with the Saver.
• overwrite_mode - SKIP_EXISTING (default) or OVERWRITE_FAMILY.
• scan_limit - 0 means everything. Set it to 5 first. Do this: you want to see one sidecar before committing an evening.
• write_scan_report - ON drops gpm_scan_report.txt in the root folder with counts and failures.
• model_name / mmproj_name - your GGUF and its matching projector. Leave mmproj_name on (auto) unless you have several.
• debug_mode - ON writes runtime trace data into the sidecar's gpm_meta. Use it when captions are bizarre, then turn it off.

Outputs are summary_json (total found, processed, skipped, failed, plus a failure list) and status_text - the same thing in one line, and the one you'll actually read.

Runtime settings are fixed here: 4096 context, all layers offloaded, 512 batch, sampling values from the preset. Nothing to tune. When you want knobs, that's the Advanced node.

Install

Registry name is gallery-prompt-manager; in ComfyUI Manager search Gallery Prompt Manager. Manual:

cd ComfyUI/custom_nodes
git clone https://github.com/Mystic419/GPM-for-ComfyUI GPM
cd GPM
python -m pip install -r requirements.txt
python install.py

Then the models: a Qwen2.5-VL GGUF and its matching mmproj into ComfyUI/models/llm/, ComfyUI/models/llm/GGUF/ or ComfyUI/models/GGUF/, then refresh. Fresh scanner nodes prioritise detected Qwen2.5-VL files in the dropdown. The pack's own workflow uses the Qwen2.5-VL Abliterated Caption GGUF family as the reference pair; abliterated matters here for the same reason it does for any local captioner - the instruct builds carry refusals.

install.py handles llama-cpp-python, and it's the step people skip. Plain pip install llama-cpp-python gives you a CPU build; the installer leaves a working install alone, otherwise tries CUDA wheels before CPU. Its archived wheels install with --force-reinstall --no-deps specifically so they don't stomp shared ComfyUI dependencies - if an older build did, python -m pip install "numpy<2.3.0,>=2" is the documented repair.

Where people get burned

• CPU-only llama-cpp-python. Scanning "works" and takes all night. GPM VLM Internal Diagnostics says so in status_text.
• Model without its projector. The worker reports mmproj_name is required for internal GGUF mode.
• Model and mmproj from different releases. They may load and write plausible captions about things that aren't in the picture. Same release, always.
• Expecting one run to refresh another family. SKIP_EXISTING won't. Use OVERWRITE_FAMILY deliberately.
• Old workflows after an upgrade. The pack's own notes say recreating scanner nodes is safer than trusting saved widget state - the lifecycle widgets changed between builds.