Vision LLM Judge

Running 40 variations and eyeballing which one wins is a ritual, and it doesn't scale. Vision LLM Judge is the node that does the eyeballing for you: it scores a batch or list of images with a vision GGUF model against a rubric you write, and hands back a structured verdict per image - score, tags, comment, and the index of the winner. It's the "automated judge" pattern that Ouroboros-style self-correcting samplers use, packaged as a standalone node you can drop on any comparison workflow.

How it works

Under the hood it's a local LLM node with eyes. It reuses the pack's Local LLM machinery, so the model loads once and stays warm across every image - only the first image pays the load cost. The clever part is the output contract: a GBNF grammar forces the model to emit clean {score, tags, comment} JSON regardless of what it wants to say, so the verdict always parses. In multi-criteria mode the grammar is generated on the fly to force {scores: {…}, tags, comment} with one sub-score per criterion.

The wiring that matters

• config - this is the part beginners trip on. It takes a Local LLM Settings (GGUF) node that has a Vision Settings (GGUF) (mmproj) attached. No mmproj, no vision, hard error. So the real setup cost is: a GGUF LLM, a matching vision projector file, and llama-cpp-python.
• images - a batch or an image list (mixed sizes fine).
• rubric - what to judge on, in plain English. "Rate anatomy, prompt adherence, sharpness. Penalize extra fingers." Be concrete - the quality of the scores is only as good as how specific your rubric is.
• criteria (optional) - multi-criteria mode: one criterion per line (name or name: description). Each gets scored, the overall score is the average. It ships pre-filled with a sensible example (overall_quality / anatomy / prompt_compliance / camera / text). Clear it to get a single overall score instead.
• prompts (optional) - per-image generation prompts as ----separated blocks, so the judge can assess prompt adherence. This is where the pack's Get Accumulator (prompts) feeds in.
• score_min / score_max - the scale. Default 1–5; the best_index output is 0-based, ties go to the first.

There's also system_prompt (the judge persona, editable, falls back to a strict default) and comment_style (default "one concise sentence") - both can only change quality, never break parsing, since the JSON shape is grammar-managed. unload_after_run lets one node free VRAM without touching the shared config - handy when you judge a batch, then want the VRAM back for generation.

Outputs

summary is the human-readable report per image (feed it to a Show Text node). results_json is the structured verdict list - the pack's Image Compare renders it as a read-only judge section per image (stars/tags/comment). best_index is the 0-based index of the top image, which is what you'd wire into a selector for "auto-pick the winner." help prints a cheat-sheet.

Install

Via ComfyUI Manager (search "Kinburg-Nodes") or git clone https://github.com/Kinburg/Kinburg-Nodes into custom_nodes. This is the one node in this family that drags in a real dependency: the Local LLM nodes need llama-cpp-python (CUDA build), which install.py installs automatically to match your torch's CUDA version - Manager runs it for you, or run <ComfyUI>/.venv/Scripts/python.exe <ComfyUI>/custom_nodes/Kinburg-Nodes/install.py by hand. You also need a vision GGUF and its mmproj file in ComfyUI/models/llm.

Honest expectations

The grammar guarantees the JSON, not the wisdom. Scores are only as reliable as the vision model and how concrete your rubric is - the LLM-in-ComfyUI doc's honest ceiling applies: even the best VLMs mix up attributions and will happily reward a clean render of the wrong idea. Keep temperature on the Settings node low (0.0–0.2) for stable, repeatable scores, and treat the verdict as a strong filter, not a verdict from god. This is arbitrary Python that loads models from the network - install from the official repo and read what a fresh node does before trusting it.