StreamingT2VRunShortStepModelscopeT2V

StreamingT2V doesn't generate long videos from scratch. It takes a short one and keeps extending it, a few frames at a time, until it's a minute or more. Which means your whole pipeline starts with a seed clip - and this node is the text-to-video way to make one.

What it is

Drop a prompt in, and this node runs it through Modelscope's text-to-video diffusion model (damo-vilab/text-to-video-ms-1.7b) and hands you 16 frames at 256x256 - about two seconds at 8 fps. That's the "short step": the opening chunk that the long-step nodes in this pack (or the all-in-one StreamingT2VRunT2V) grab and keep autoregressively extending.

The pack ships three short-step seed makers - Modelscope, AnimateDiff, and SVD - and the difference is what you feed them. Modelscope (this one) is the pure text-to-video path: no input image, no starting frame, just a prompt and a seed. AnimateDiff is also text-driven but wants an SD 1.5 checkpoint. SVD is the image-to-video one, for when you want the clip to open from a specific still. If you have a character or composition in mind, that's the SVD node's job.

How it works

The model input is a T2VModel produced by StreamingT2VLoaderModelscopeModel - a one-input loader whose entire job is pulling the Modelscope pipeline down from HuggingFace on first run. That download is automatic; you don't place anything by hand. The node then runs 50 diffusion steps (hardcoded - there's no step-count slider here, unlike the big run node) and returns an IMAGE tensor named short_video.

The three inputs that actually matter:

• model - the T2VModel from the loader. Nothing else plugs into this socket.
• prompt - decides what's in your 16 frames.
• seed - the clip is fully seed-driven. Change the prompt, leave the seed alone, and you'll get a very confused cat.

Wire short_video into a long-step node and it becomes the opening of your real video. It's also fine to stop here - the README explicitly says you can use any 16-frame 256x256 clip as the seed, even something you grabbed off the internet, so this node is more "make a seed from scratch" than a mandatory stage.

Installing the pack

This is one of four nodes in chaojie/ComfyUI_StreamingT2V, so install the pack once:

• Easiest: ComfyUI Manager → Install Custom Nodes → search StreamingT2V → install, then restart.
• Or manually: cd ComfyUI/custom_nodes && git clone https://github.com/chaojie/ComfyUI_StreamingT2V, then restart ComfyUI.

Then pip install -r requirements.txt - and this one is heavy. We're talking diffusers, modelscope, xformers, open-clip-torch, fairscale, decord, av, imageio[ffmpeg] and friends, so expect a long first install. On top of that, the long-step half of the pipeline needs the StreamingT2V checkpoint: either the 25.6GB original streaming_t2v.ckpt from PAIR's HuggingFace repo placed in ComfyUI/models/checkpoints, or the 5GB VidXTend model, which auto-downloads. The short-step node itself only pulls the ~5GB Modelscope model, so you can dip your toe in before committing to the big weights.

Gotchas

• 16 frames is not a bug. That's the design - it's a seed, not a finished video. Feed it into the long step.
• First run downloads the Modelscope model automatically, which can look like ComfyUI hanging. It isn't. Give it a couple of minutes on a decent connection.
• This is the lowest-VRAM seed option of the three, and the pack's two-stage design (short step, then long step) exists precisely so you don't have to hold the seed model and the 25GB streaming model in VRAM at once. If your card is under ~10GB, do the two-stage thing instead of the all-in-one run node.

One light aside: the default seed of 33 and the default "A cat running on the street" prompt are a fine first smoke test - you'll know in under a minute whether the whole install actually works.