Lance Video Understanding

Video understanding is where unified models earn their keep: one 3B model that can watch a clip and answer questions about it, locally, is still rare. Lance Video Understanding takes your frame batch and a question - "What does the person do after the door opens?" "Describe this video in detail." - and returns a text answer. No denoising, no sampling, just the vision encoder plus the LLM doing its thing.

How it gets there is worth knowing because it explains the node's one non-obvious input. ComfyUI hands you frames as an IMAGE batch, but Lance's official video path wants an actual video file, so the node writes your frames to a temp MP4 (imageio, libx264, 12fps) and builds the official x2t_video interleave - video, then instruction + question, then the answer slot. Same validate_on_fixed_batch call as everything else in the pack, understanding_* flags set. Output is text, a STRING you can feed to Show Text or pipe anywhere text goes.

The inputs that matter

• frames - from ComfyUI's native Load Video or VideoHelperSuite. The node subsamples long clips down to max_frames.
• question - defaults to "Describe this video in detail."
• system_instruction - the official x2t_video instruction prefix, "Watch the video carefully and answer the question," exposed as an editable string like its image sibling.
• max_frames - default 121, range 5–121 in steps of 4. This is the practical one: the Wan temporal VAE only eats 4n+1 frame counts, so the node uniformly subsamples longer batches to whatever you set. The 121 cap is the model's hard limit; dropping to 41 or 61 makes the ViT encode much faster with a real quality tradeoff on long clips.
• max_tokens - 512 default, 64–1024.
• temperature (1.0) and do_sample - greedy decode by default, which is the right call for question answering; sampling buys variety you usually don't want.
• model - the loader output, and for this task it should be the video variant (lance_3b_video), which is what carries the temporal understanding path.

Wiring it up

[Lance Model Loader (lance_3b_video)] → [Load Video] → [Lance Video Understanding] → [Show Text]

Gotchas

Two things trip people up. First, the temp-MP4 round trip makes this node slower than its image sibling - every run encodes a video file before the model even looks at it, and the ViT encode of a 121-frame clip is the real bottleneck. Keep max_frames as low as your question tolerates. Second, the 40GB VRAM floor of the whole pack still applies; the model's loaded in full whether you're generating or just asking questions. And the 4n+1 frame rule is baked into the subsampler, so don't be confused if the model "sees" fewer frames than your source clip has.