Nodes/ComfyUI/Grok Reference-to-Video
ComfyUI Node Runs on cloud

Grok Reference-to-Video

Grok video steered by reference images and preset voices

By Comfy-Org·Created 4 years ago·Updated about 13 hours ago· 130,663
Grok Reference-to-Video
    • VIDEO
    prompt
    model
    seed0

    Grok Video Reference is the "show, don't just tell" node in the Grok video family. Plain Grok Video takes a prompt (or a single starting image); this one lets you hand the model up to seven reference images to anchor style and subject, and - the part that gets people excited - optional preset voice references so the video can include spoken audio in a voice you picked. It's a built-in partner node, so it's a cloud call: upload references, wait, get a VIDEO back. No GPU work on your side, billed through your Comfy account.

    The way you steer it is a little unusual and worth reading before you complain that "nothing is happening." With the newer model, you refer to your images by name in the prompt using @Image1 through @Image7 (numbered in input order, and a batched image input counts once per image). Voices work the same way: @Audio1, @Audio2, @Audio3 in the prompt. The model isn't guessing which reference you meant - you have to point at it.

    The inputs

    • prompt - the text description of the desired video, including the @ImageN / @AudioN tags that tie references into the scene.
    • model - a dropdown that carries most of the controls with it:
      • grok-imagine-video-1.5 adds reference_images (up to 7), voice_1/2/3 (preset voices only - the API supports these, not custom audio uploads), plus resolution, aspect_ratio, and duration.
      • grok-imagine-video offers reference images plus the same resolution/aspect/duration knobs, without the voice options.
    • seed - the usual partner-node seed: it just triggers a re-run; actual results are nondeterministic.

    What comes out

    A single VIDEO output. Save it or hand it to another node in your graph.

    Gotchas

    • Voices are presets, not uploads. The node description and tooltips are explicit: the API supports only the listed preset voices. If you expected to clone your own voice, this node won't do it.
    • Un-tagged references are wasted. Feed five images but never mention @Image2 in the prompt and the model has no reason to honor it. Tagging is the mechanism, not an optional extra.
    • Watch the reference count. The 1.5 model caps at 7 images; a batched input counts per image. More than that and the request fails.
    • It's a paid call per generation, and adding voices/references can push quality and price up together - budget accordingly.

    The pattern to steal: generate a character or scene locally, then feed a couple of frames plus a voice into this node to get a video that actually matches what you built, instead of rolling the dice on pure text.

    Categorypartner/video/Grok

    Inputs (3)

    NameTypeDefaultDescription
    promptSTRINGText description of the desired video.
    modelCOMBOThe model to use for video generation.
    seedINT00–2147483647Seed to determine if node should re-run; actual results are nondeterministic regardless of seed.

    Outputs (1)

    NameTypeDescription
    VIDEOVIDEO