StreamingT2VLoaderAnimateDiffModel

StreamingT2VLoaderAnimateDiffModel loads one thing and one thing only: the AnimateDiff-based text-to-video model that generates the short 16-frame seed video this pack extends into long videos. It's the "staged" counterpart to StreamingT2VLoaderAnimateDiff, which loads everything at once. If you're building the workflow in stages - short video first, then long extension, then enhancement - this is stage one's loader.

AnimateDiff is the odd-but-fitting choice here. It's the 2023 trick of injecting a temporal motion module into a frozen SD 1.5 UNet so a still-image checkpoint can animate. It's long since superseded for serious video work, but as a seed generator for StreamingT2V it does the job: you only need 16 frames at 256×256, and AnimateDiff is small and fast at exactly that. The pack pairs it with the Realistic Vision SD 1.5 checkpoint for a pleasant default look.

What it loads

Two HuggingFace models, both downloaded automatically on first run in fp16:

• guoyww/animatediff-motion-adapter-v1-5-2 - the motion module.
• SG161222/Realistic_Vision_V5.1_noVAE - the base SD 1.5 checkpoint.

The loader enables model CPU offload and VAE slicing, so it's friendlier to modest VRAM than you might expect for something sitting next to a 25GB checkpoint.

Inputs and output

One input: device (cuda or cpu, default cuda). That's it - there's no checkpoint picker because nothing here comes from your models/checkpoints folder; everything streams from HuggingFace.

The single output is a T2VModel object, which you feed to StreamingT2VRunShortStepAnimateDiff. That run node turns the model plus a prompt into the 16-frame short_video IMAGE that the long-extension stage consumes.

Installing it

Same as every node in this pack:

cd ComfyUI/custom_nodes
git clone https://github.com/chaojie/ComfyUI_StreamingT2V

or ComfyUI Manager → search StreamingT2V → install, then restart. Note this is the pack with the heavyweight requirements.txt (diffusers, pytorch-lightning, modelscope, xformers...), and the first execution of the loader downloads ~1.5GB of AnimateDiff weights, so the first Queue click will hang for a while. That's normal - it's downloading, not frozen.

The catch

This loader is only worth using if you're actually going to run the AnimateDiff seed path. The pack offers three seed generators - AnimateDiff (T2V), Modelscope T2V (T2V), and SVD (I2V, which takes an image) - and you pick one per workflow. Don't load more than one just to see what they look like; each holds a diffusion model in VRAM and there's no benefit to having both resident. If your prompt is where your content comes from, one seed generator is enough, and AnimateDiff's the most familiar of the three.