KIE • Sample Video Frames

Want an LLM to describe what happens in your video? You can't hand it a video - you hand it frames and hope. KIE • Sample Video Frames does the sampling part locally, for free, and then hands you a prompt that tells the model, in no uncertain terms, not to make things up.

It's the honest version of this trick. The node's own description says its output "can miss action between frames or hallucinate," which is more candour than most of this category manages.

What it produces

Four outputs, each with a job:

• ordered_frames - a batch of 2 to 12 frames at 512×288, evenly spaced across the analysed span. Ordered, so index N is later than index N−1.
• labelled_contact_sheet - the same frames burned into a two-column sheet with FRAME 3 · 8.41s labels, which is the thing you actually look at to check nothing's missing.
• frame_times_json - the timing record: total duration plus the exact seconds each frame was sampled at.
• vision_review_prompt - a ready-made instruction string that embeds frame_times_json, tells the model to describe only visible events, forbids inventing speech, sound, or action between frames, and demands JSON back in an events array.

Wire ordered_frames and vision_review_prompt into a vision-capable LLM node - a KIE Gemini or Claude node, or a local VLM - and you've got a described timeline. That's a paid call, and you choose it deliberately. Everything upstream of it is local.

The sampling detail worth knowing

Times aren't random; they're linspace across the span, from zero to limit − 1/fps. That backoff from the exact end is deliberate: some decoders return no frame when you seek to precisely the last timestamp, so the node steps one frame short rather than failing on the final sample. Small thing, but it's why you don't get a spurious "could not decode a sample frame" on an otherwise fine file.

The pre-sampling size is 512×288 per frame. That's fine for "what's in this shot" questions and useless for reading text on a sign, which is a reasonable trade for keeping the batch small.

Why the review gate exists

The chain the pack wants you to run is:

Sample Video Frames → vision model → (you watch the video, fix the JSON)
  → KIE • Review Video Events (editor_reviewed = true)
  → KIE • Sound Cue Sheet / KIE • Music Brief from Video

KIE • Review Video Events refuses to pass anything through until you flip editor_reviewed on, and it stamps each event as editor_verified_event with a note that it's still just editor-reviewed still-frame notes, not a verified audiovisual transcript. That's the whole design philosophy in one node: models suggest, humans confirm, then the machine spends money.

The pack ships a template for this - review-video-events.json - which displays time-labelled frames without invoking a paid model. Import it, swap in your own video, and you can see the shape of the pipeline before you connect anything expensive.

Inputs and outputs, plainly

• video - ComfyUI VIDEO.
• frame_count - 2 to 12, default 8. More frames is more tokens on the vision call, and more chances to catch the action you care about. Twelve is the ceiling.
• max_seconds - 1 to 3600, default 300. Sampling is spread across min(duration, max_seconds).

Installing it

cd ComfyUI/custom_nodes
git clone https://github.com/felipederosilva/ComfyUI-KIE-Nodes-Next
cd ComfyUI-KIE-Nodes-Next
pip install -r requirements.txt

Restart; it's under KIE Next / Studio / Video Analysis. Same dependency note as its sibling: OpenCV is imported at runtime and is not in the pack's requirements.txt, because ComfyUI already ships it. On a stripped install, pip install opencv-python fixes the Video frame sampling needs OpenCV error.

Gotchas

Frames wrong for the action. Sparse sampling means a two-second event between samples vanishes. Raise frame_count, or narrow max_seconds around the section you care about.

The model invents dialogue. It will, sometimes, no matter how good the prompt is. This is why the review node exists - read the JSON against the actual video before you let it drive SFX generation.

Batch-size surprises downstream. ordered_frames is a genuine batch. If you feed it somewhere expecting a single image, index one frame out first.

A batch of null/black frames. Unsupported codecs. Remux to H.264 MP4.