Nodes/ComfyUI-SwissArmyKnife/Multi-Caption Combiner
ComfyUI Node

Multi-Caption Combiner

Merge frame-by-frame captions into one coherent description with Gemini

By sammykumar·Created 12 months ago·Updated 8 months ago· 6
Multi-Caption Combiner
    • combined_caption
    • gemini_status
    captions
    gemini_api_key
    gemini_modelmodels/gemini-2.0-flash-exp
    combination_styleAction Summary
    timestamps
    custom_prompt
    output_formatParagraph

    Captioning a video frame-by-frame gives you detail - every still described individually. What it doesn't give you is a story. Ten captions about ten frames say nothing about what happened. Multi-Caption Combiner closes that gap the pragmatic way: it takes your per-frame captions, throws them at Gemini, and asks for a single coherent description with a style you choose. It's the finishing step of the pack's local-captioning chain.

    Inputs: captions is the raw material - comma-separated or newline-separated captions, one per frame. gemini_api_key and gemini_model (default models/gemini-2.0-flash-exp) are entered right on the node, which is a deliberate difference from the pack's settings-based Gemini path: this node takes its key in the widget, so it works standalone. combination_style gives you four merge strategies:

    • Action Summary (default) - what happened, compressed
    • Chronological Narrative - the events in order
    • Movement Flow - focused on how things moved
    • Custom - you supply the instruction in custom_prompt

    Then the presentation knobs: optional timestamps (comma-separated seconds matching each caption, which the node can weave in for a time-coded narrative), and output_format - Paragraph, Bullet Points, or Scene Description. If you want a chunk of prose to drop into a prompt builder, Paragraph is the one; Scene Description nudges it toward the pack's Wan-flavored descriptive style.

    Two outputs: combined_caption (the merged text) and gemini_status (a status string telling you the model, style, caption count, and format - handy confirmation that the run used what you think it did). Under the hood it builds a system prompt from your style choice, sends the captions (plus timestamps) to Gemini, and returns the text - with the same retry-on-overload logic the rest of the pack uses for its API calls.

    Installing

    Part of Swiss Army Knife:

    • ComfyUI Manager: search "Swiss Army Knife", install, restart.
    • Manual:
    cd ComfyUI/custom_nodes
    git clone https://github.com/sammykumar/ComfyUI-SwissArmyKnife.git
    cd ComfyUI-SwissArmyKnife
    pip install -e .
    

    The flow it belongs in

    Frame Extractor → (LLM Studio Picture Describe per frame) → Multi-Caption Combiner. Frames become captions become a story. The timestamps input is the sleeper feature: feed it the frame_timestamps output from Frame Extractor and your combined narrative gains a time axis.

    Gotchas

    • Key is on the node here, not in settings. If you're used to the pack's settings-based Gemini config, this one will bite - a blank key means an auth error even when everything else works.
    • Caption order matters. The node assumes input order = chronological order. Scramble the captions and the "chronological narrative" will be fiction.
    • Timestamps are optional and must match the caption count. Mismatched counts cause the merge to behave oddly - when in doubt, leave timestamps out.
    • gemini_status is your diagnostic friend: if the model came back with the wrong style or format, that's the first place to look.

    It's a small node with a clear job: turn a pile of frame captions into the one paragraph you can actually use. In the pack's ecosystem, it's how per-frame detail becomes a usable prompt.

    CategorySwiss Army Knife 🔪/Utils

    Inputs (7)

    NameTypeDefaultDescription
    captionsSTRINGComma-separated or newline-separated captions to combine (one per frame)
    gemini_api_keySTRINGGoogle Gemini API key for caption combining
    gemini_modelSTRINGmodels/gemini-2.0-flash-expGemini model to use for combining captions
    combination_styleCOMBOAction SummaryStyle for combining captions
    timestampsoptSTRINGComma-separated timestamps (in seconds) corresponding to each caption (optional)
    custom_promptoptSTRINGCustom instruction for combining captions (used when combination_style is 'Custom')
    output_formatoptCOMBOParagraphFormat for the combined caption output

    Outputs (2)

    NameTypeDescription
    combined_captionSTRING
    gemini_statusSTRING