FL PenguinVL Video Query

FL PenguinVL Video Query is the pack's party trick: it takes a sequence of frames and answers natural-language questions about them - "does the subject's motion match the prompt," "what's the scene order," "describe what happens." If you're generating video in ComfyUI, this is the node that closes the loop. Instead of eyeballing 80 frames to check whether your Wan or LTX output actually did what you asked, you hand it to a vision-language model and read the verdict.

How it works

Here's the thing that catches people: this node never decodes a video file. "Video" in this pack means an IMAGE batch treated as frames - you extract the frames yourself with whatever node you normally use to turn a clip into a batch (the usual suspects are the video-loading helpers like Video Helper Suite), and feed that in. Once it has your frames, the node uniformly samples up to max_frames of them - evenly spaced picks from the whole batch, so a 120-frame clip at max_frames 16 still spans the entire timeline. Those frames get converted to PIL, resized if you set a max_long_side, and sent to the processor as a proper video message (type: "video" with a frame count), which is how Penguin-VL sees a moving scene rather than a stack of unrelated images.

Inputs that matter

Required: model, images (the frame batch), and prompt - default "Describe what happens in this video."

The one knob unique to this node:

• max_frames - how many frames to sample (default 16, max 128). The tooltip is blunt: more frames = more VRAM. 16 is the author's recommended default and it genuinely is the sweet spot for most clips. Bump it toward 128 only when you need to catch a fast action or a detail that might fall between sampled frames - and only if your GPU can take it.

The rest are the familiar query knobs: max_new_tokens (512 default), temperature, top_p, do_sample (False = deterministic), system_prompt for output format, and max_long_side for resizing each frame before it's encoded. Same semantics as the image nodes - long-side 512 keeps VRAM and token cost down without hurting understanding.

Output is text: one STRING describing what the model saw, ready for a text viewer or straight into your prompt pipeline.

Install

Pack-level, same as the rest: ComfyUI Manager → search "FL PenguinVL", or git clone https://github.com/filliptm/comfyui-fl-penguinvl into custom_nodes, pip install -r requirements.txt, restart. Models auto-download on first run to ComfyUI/models/penguin_vl/.

Gotchas worth knowing

• Frame extraction is on you. No mp4 path, no auto-decode. If you plug a video directly in, nothing happens - convert it to frames first.
• Uniform sampling skips. That's the point (coverage of a long clip), but a quick gesture that happens between two sampled frames can be invisible to the model. If you're verifying specific motion and the answer feels vague, raise max_frames and re-run.
• VRAM is the real ceiling. 8B model + 128 frames is asking for an OOM on a mid-range card. 2B at 16 frames is the budget config and it handles most "did the motion hold up" checks fine.
• Determinism for QA. If you're comparing two renders and want apples-to-apples descriptions, set do_sample to False; sampling noise can otherwise make identical clips read differently.

For a new-ish model this one's niche is clear and useful: local video understanding without an API call, wired right into the graph next to the renderer that made the clip. Feed it frames, get a read on what actually happened, iterate.