FL PenguinVL Image Query

This is the workhorse of the pack. FL PenguinVL Image Query takes one image, one text prompt, and returns the model's answer as text - no API key, no cloud, no per-request billing. You use it for the boring-but-valuable jobs: captioning images for dataset work, reading text out of a screenshot (it handles OCR fine), asking what style or character a reference image actually is before you try to match it, or checking a generation for issues before you commit to upscaling it.

How it works

The flow is short. Your ComfyUI IMAGE tensor gets converted to a PIL image, optionally resized, then wrapped into a chat message - system prompt first, then the image plus your text prompt - and handed to the processor. The processor tokenizes it, the model generates under no_grad(), and the output is decoded and returned as a single STRING. Straightforward stuff, and it behaves.

Two details are worth knowing because they affect your results. First, decoding: when do_sample is on (the default), temperature and top_p shape the output; flip it to False and the node goes greedy - same image, same prompt, same answer every run. That determinism is gold for dataset captioning, where you want repeatable labels. Second, the model runs in the dtype your loader picked (bf16 by default), and everything gets moved to that on the GPU.

Inputs that matter

Only three are required: model (from the Load Model node), image (any single IMAGE - if you feed a batch it just uses the first frame, so use the Batch node for lists), and prompt - multiline, defaults to "Describe this image in detail."

Then the ones you'll actually touch:

• max_long_side - resizes the image's long side to this many pixels (default 512) with LANCZOS, 0 = no resize. This is the accuracy lever. The author's own tip: bigger isn't better. A 4K render downsampled to 512 is plenty for the model to see and costs a fraction of the tokens.
• max_new_tokens - cap on the reply (default 512, up to 4096). Raise it if answers get cut off mid-sentence.
• system_prompt - optional context. This is where you enforce format, and the README's suggestion is a good one: "Respond with only a comma-separated list of tags" turns a captioner into a tagger.
• do_sample, temperature, top_p - the sampling knobs; see above for determinism.

The output is text, a plain STRING. Wire it to a text viewer or Preview Text node to read it, or straight into a prompt field if you're doing prompt-reversal.

Setting it up

Same pack install as the Load Model node: ComfyUI Manager → search "FL PenguinVL", or git clone https://github.com/filliptm/comfyui-fl-penguinvl.git into custom_nodes, pip install -r requirements.txt, restart. The first run downloads the Penguin-VL checkpoint automatically (2B is ~6 GB VRAM in bf16; 8B ~18 GB - pick by card, not by ambition).

Common gotchas

• Oversized inputs. Feeding a native 1024x1024 SDXL image at max_long_side 512 is the intended pattern - you don't need full resolution for the model to understand an image. If you're hitting high VRAM or slow runs, this is the first dial to lower.
• "It gave a different caption last time." That's sampling, not a bug. Set do_sample to False for repeatable output.
• Answers getting chopped. Bump max_new_tokens; 512 is a safe default but a detailed caption can run past it.

One honest caveat: if you're captioning thousands of images, a single image-per-run node is not the throughput king - that's what the Batch node in this pack is for, and even then it's sequential. For the occasional image, a test render, or a dataset where you care about quality over speed, this node is the one you want.