Nodes/ComfyUI-iterator-nodes/迭代视频并按间隔提取多帧 (For Chat)
ComfyUI Node

迭代视频并按间隔提取多帧 (For Chat)

Turn a folder of videos into keyframes your vision LLM can actually read

By GHOSTLXH·Created about a year ago·Updated about a year ago· 7
迭代视频并按间隔提取多帧 (For Chat)
    • content_item_1
    • content_item_2
    • content_item_3
    • content_item_4
    • content_item_5
    • content_item_6
    • content_item_7
    • content_item_8
    • content_item_9
    • content_item_10
    • video_filename
    folder_pathC:\path\to\your\videos_folder
    frame_interval30
    max_frames_to_extract5
    image_formatjpeg
    quality85

    This is the pack's showcase node - the one the whole README demo workflow is built around. The pitch: point it at a folder of videos and it walks them one at a time, pulls spaced-out keyframes from each, and hands those frames to a vision LLM in a format it can ingest. That's the "video frame extraction and tagging" pipeline - you get a folder of clips in one end and captions or descriptions for each one out the other, no per-video fiddling. It's also the fiddliest node in the pack, and the author is refreshingly honest about that.

    How it works

    Familiar iterator engine: cached sorted file list, IS_CHANGED returning NaN so it fires every queue run, exception when the folder's done. The interesting part is the frame extraction. For the current video it opens the file with OpenCV, seeks to frame 0, reads a frame, then jumps forward by frame_interval frames and repeats until the video ends or it hits the frame cap. Each selected frame is converted BGR→RGB, compressed with Pillow, and base64-encoded into a data URL wrapped in a dict: {"input_image": {"image_url": "data:...;base64,...", "detail": "high"}}. That dict is an OAI_CONTENT_ITEM - a content block in the shape the OpenAI-style chat APIs expect.

    frame_interval is in frames, not seconds. The README's example: a 16fps video with frame_interval=16 gives you one frame per second. The 30 default is "one frame per second" only at 30fps - adjust it to your clip's actual frame rate or you'll oversample or undersample without realizing why.

    max_frames_to_extract caps the count. Default 5, max 10. Even if the interval math would yield 20 frames, it stops at the cap. This is your token/byte budget control, and it matters more than the quality slider.

    The other two inputs: image_format (jpeg / png / webp, default jpeg) and quality (10–100, default 85, applied to jpeg/webp). Keep it on jpeg unless you need lossless - a png keyframe is several times bigger as a data URL, and the whole frame payload rides inside your API request.

    The outputs

    Up to ten content_item_1content_item_10 outputs (OAI_CONTENT_ITEM), padded with None when a video yields fewer frames, plus video_filename (STRING, base name without extension). The intended wiring, straight from the README: feed the content items into the OpenAIContentConnector node from spawner1145/comfyui-aichat (the author names and credits this companion pack explicitly), which sorts and merges them into one message for an OpenAI-format chat node. That's where your API key lives - this node itself only formats frames, it calls nothing.

    Installing it

    Ships in ComfyUI-iterator-nodes:

    cd ComfyUI/custom_nodes
    git clone https://github.com/GHOSTLXH/ComfyUI-iterator-nodes
    

    then restart ComfyUI, or use Manager and search the pack name. No model downloads. It imports OpenCV and the pack has no requirements.txt, so if it errors on import, pip install opencv-python. You'll also need to install comfyui-aichat for the recommended chat-side wiring.

    Where people get burned

    • The README's big warning is real. Every "OpenAI-compatible" API - OpenAI, but also the resellers and model providers that bolt on a compatible endpoint - differs on file upload rules, response format, and size limits. The author's advice, in so many words: expect problems, read your provider's API docs, and be ready to edit the node's Python to match. It's not plug-and-play across providers, and no amount of tweaking sliders fixes a provider that rejects your content shape.
    • Data URLs get big fast. A handful of high-quality 1080p jpegs as base64 can dwarf your text prompt and blow token/byte budgets or get rejected outright. Short clips, lower quality, or max_frames_to_extract down - in that order.
    • "detail": "high" is hardcoded in the content items; you can't turn it off from the UI.
    • Family rules apply: "Run (on change)" queue mode for full automation, cached file list, end-of-folder exception is the designed stop.

    It's the most opinionated node in the pack and the one with the highest ceiling - a genuine hands-free "folder of videos → LLM captions" machine. Just budget time for provider roulette before it sings.

    Categoryutilities/loaders/OpenAI

    Inputs (5)

    NameTypeDefaultDescription
    folder_pathSTRINGC:\path\to\your\videos_folder
    frame_intervalINT30
    max_frames_to_extractINT51–10
    image_formatCOMBOjpeg3 options: jpeg, png, webp
    qualityINT8510–100

    Outputs (11)

    NameTypeDescription
    content_item_1OAI_CONTENT_ITEM
    content_item_2OAI_CONTENT_ITEM
    content_item_3OAI_CONTENT_ITEM
    content_item_4OAI_CONTENT_ITEM
    content_item_5OAI_CONTENT_ITEM
    content_item_6OAI_CONTENT_ITEM
    content_item_7OAI_CONTENT_ITEM
    content_item_8OAI_CONTENT_ITEM
    content_item_9OAI_CONTENT_ITEM
    content_item_10OAI_CONTENT_ITEM
    video_filenameSTRING