StreamingT2VRunShortStepAnimateDiff

StreamingT2VRunShortStepAnimateDiff is stage one of the AnimateDiff route through this pack: it turns a prompt into a 16-frame, 256×256 seed video using the AnimateDiff model from StreamingT2VLoaderAnimateDiffModel. The output short_video IMAGE is exactly what the long-extension nodes need to start generating. Simple job, small node - but it's the creative foundation of everything that follows.

Here's the framing that makes the short step make sense. StreamingT2V can't generate a long video from nothing; it extends a short one autoregressively, 8 frames at a time. So somebody has to make the first 16 frames, and this pack offers three ways: AnimateDiff (prompt → video), Modelscope T2V (prompt → video), and SVD (image → video). This node is the AnimateDiff option. AnimateDiff is a 2023-era motion module layered onto SD 1.5 - thoroughly superseded for serious video work these days, but as a 16-frame seed factory it's still perfectly adequate, and it's the most familiar pipeline to anyone who's been around SD 1.5.

How it works

The node runs the AnimateDiff pipeline from the loader: 16 frames, 25 inference steps, guidance 7.5, a hardcoded negative prompt of "bad quality, worse quality," resized to 256×256. Notably, the step count is baked in - you won't find a num_steps input here like on the extension nodes. The only creative knobs are the prompt and the seed, which makes this node refreshingly dumb in the best way. Frames come back as a standard IMAGE tensor.

Inputs that matter

• model - the T2VModel from StreamingT2VLoaderAnimateDiffModel.
• prompt - what the seed shows. This is where you decide what your whole long video is about.
• seed - default 33. Change it to get a different take on the same prompt.

The output, short_video, is a 16-frame IMAGE. Wire it into the long-extension node of your choice - StreamingT2VRunLongStepVidXTendPipeline or StreamingT2VRunLongStep - and you're off. This node is marked as an output node, so it can also stand alone as the end of a short-clip workflow if that's all you want.

Installing it

Pack install:

cd ComfyUI/custom_nodes
git clone https://github.com/chaojie/ComfyUI_StreamingT2V

or ComfyUI Manager → search StreamingT2V → install, restart. The AnimateDiff weights (motion adapter + Realistic Vision V5.1 base) auto-download from HuggingFace on first run - no manual checkpoint. The pack's requirements.txt is heavy (diffusers, pytorch-lightning, modelscope, xformers...), and the first Queue click will sit while the ~1.5GB of AnimateDiff weights come down.

The take

A 16-frame seed is a low-stakes job, and this node keeps it low-stakes. One thing worth internalizing from the README: the short step is explicitly optional - you can feed any 16-frame 256×256 video into the long-extension stage, even one you found online. So use this node when you want a seed that matches your prompt, but don't treat it as precious. The seed only needs to be good enough for the appearance-preservation module to hold onto. If the seed is weak, regenerate it - it's 16 frames, it's cheap.