Nodes/ComfyUI_StreamingT2V/StreamingT2VLoaderStreamModel
ComfyUI Node

StreamingT2VLoaderStreamModel

Load just the autoregressive heart of StreamingT2V

By chaojie·Created 2 years ago·Updated 2 years ago· 36
StreamingT2VLoaderStreamModel
    • stream_cli
    • stream_model
    ckpt_namestreaming_t2v.ckpt
    devicecuda

    StreamingT2VLoaderStreamModel loads the one model this whole pack exists to expose: the StreamingT2V autoregressive long-video model itself. Everything else - seed generators, enhancers - is supporting cast. This node gives you just the heart, as two outputs that feed StreamingT2VRunLongStep: stream_cli (the Lightning CLI object that drives inference) and stream_model (the weights).

    If you've been looking at the pack and wondering what "original model" vs "VidXTend" means, this node is the "original model" story. StreamingT2V is the research release from Picsart AI Research (arXiv 2403.14773) that generates long videos by autoregressive extension: it takes a short 16-frame video, generates the next 8 frames conditioned on the previous chunk, and keeps going - 80, 240, 600 frames, however long you let it run. The magic is in the conditioning: a short-term "conditional attention module" keeps chunk transitions smooth, and an "appearance preservation" module reaches back to the first chunk so the scene doesn't drift into something else. This node loads the weights that do all of that.

    The weight you need to supply

    This loader does not auto-download. It reads streaming_t2v.ckpt from your ComfyUI/models/checkpoints folder (default name streaming_t2v.ckpt), and you have to get that file yourself - it's the ~25.6GB checkpoint from the PAIR/StreamingT2V HuggingFace page. The README is explicit: original model, <15GB VRAM. If you have a 10GB-ish card, the pack points you at the VidXTend route instead, which needs no manual download at all.

    Inputs

    • ckpt_name - the checkpoint to load, chosen from checkpoints. Default streaming_t2v.ckpt.
    • device - cuda or cpu. Note the clever bit in the source: if you select cpu, it flips the internal vram_not_enough flag to true and keeps weights on CPU. CPU inference on a 25.6GB model is not a pleasant experience, but it exists.

    Outputs are the stream_cli and stream_model pair, both required by StreamingT2VRunLongStep.

    Installing it

    The usual:

    cd ComfyUI/custom_nodes
    git clone https://github.com/chaojie/ComfyUI_StreamingT2V
    

    or ComfyUI Manager → search StreamingT2V → install, restart. requirements.txt is heavy (pytorch-lightning, diffusers, modelscope, xformers...), and the model download is on you - this is the one loader in the pack where nothing comes down automatically.

    The take

    This is the node for the StreamingT2V purist: no seed generator, no enhancer, just the extension model, so you can pair it with whatever seed you like (including a downloaded video, which the README explicitly allows for step 1). It's also the node most people should probably not use - the VidXTend loader exists precisely because 25.6GB is an absurd amount of weight for a model whose 2024-era output is charming but dated. If your VRAM is tight, skip the original checkpoint entirely. If you want the real thing, this is the only way to get it.

    CategoryStreamingT2V

    Inputs (2)

    NameTypeDefaultDescription
    ckpt_nameCOMBOstreaming_t2v.ckpt0 options:
    deviceCOMBOcuda2 options: cuda, cpu

    Outputs (2)

    NameTypeDescription
    stream_clistream_cli
    stream_modelstream_model