OpenSoraPlanLoader

OpenSoraPlanLoader is the entry point for a pack that wraps PKU-YuanGroup's Open-Sora-Plan, a 2024-era open text-to-video model, into ComfyUI. It doesn't just load a checkpoint - it loads a whole stack: a diffusion transformer, a video VAE, and a giant T5-XXL text encoder, bundles them into one custom object, and hands that object to the pack's other three nodes. No Loader, nothing else works.

Let's be honest about why you'd reach for it. Open-Sora-Plan is a research-era open video model - the community consensus in 2024 was "fine for open weights, not in the same league as the commercial stuff," and the modern options (Wan, LTX, Hunyuan) have left it behind since. This pack is a time capsule for a specific model. You use it because you specifically want those weights - say, to poke at a piece of video-model history or reproduce an old workflow - not because it's the best video you can get today. Its one genuinely interesting trick: it's a joint image-and-video model, so with a single toggle it'll also generate a plain 512×512 still.

How it works

The node pulls a LatteT2V diffusion transformer from your model_path (fp16), wraps the CausalVAE from the same repo's vae subfolder, loads the T5 text encoder, wires up a PNDM scheduler, and assembles a VideoGenPipeline - the same sampling pipeline the upstream project uses. That's a real chunk of work every time, and on first run it's also a real chunk of downloading.

Where people get burned: the weights are not in the repo. The first load downloads the transformer and VAE from HuggingFace (LanguageBind/Open-Sora-Plan-v1.0.0 by default) and the T5-XXL text encoder from DeepFloyd/t5-v1_1-xxl - roughly 11 GB in fp16 on its own. Worse, the code passes cache_dir='cache_dir', so everything lands in a folder created next to wherever you launched ComfyUI, not in ComfyUI/models. The first run looks like a hang. It isn't; watch the console for progress.

The inputs that matter

Of the five, a beginner really touches three:

• version - the resolution string, <frames>x<height>x<width>. Default 65x512x512 is 65 frames at 512×512; 221x512x512 is 221 frames of the same. This is the single most consequential setting, because it's both your video length and your VRAM bill.
• force_images - flip it to true and the model generates a single 512×512 image instead of a clip. The joint image/video thing I mentioned.
• model_path - the HuggingFace repo id or a local folder containing the weights. Leave the default unless you've pre-downloaded to a local path.

ae (CausalVAEModel_4x8x8) and text_encoder_name (DeepFloyd/t5-v1_1-xxl) are the architecture choices behind the scenes. If you swap the text encoder, the prompt handling changes with it - don't touch either until you know why.

The output is a single OpenSoraPlanModel object. It feeds the other three nodes in the pack: OpenSoraPlanRun (sample + decode in one step), OpenSoraPlanSample (latents out), or OpenSoraPlanDecode (latents in, frames out). It's a pack-internal type - nothing else in ComfyUI knows what to do with it.

Installing it

ComfyUI Manager, search "Open-Sora-Plan", install, restart. Or the manual way:

cd ComfyUI/custom_nodes
git clone https://github.com/chaojie/ComfyUI-Open-Sora-Plan

then restart. The repo vendors the entire Open-Sora-Plan source, so there's nothing else to clone - but it does need the Python deps (diffusers, timm, einops, accelerate, imageio and friends) present in your ComfyUI environment. If the node renders red after install, that's why; pip install the missing ones into the same environment ComfyUI runs in.

One warning worth shouting: the repo's docker folder pins ancient versions - torch 2.0.1, diffusers 0.24, transformers 4.32. Do not pip install -r that file into a working ComfyUI. It'll drag your whole install back to 2023 and break every modern pack you own.

Common issues

• First load sits for ages - it's downloading. Watch the console, check your cache_dir folder for disk space. T5-XXL alone is ~11 GB.
• version mismatch errors - the version string must exist as a subfolder inside model_path. If you point model_path at a local folder, the matching 65x512x512 subfolder has to be in it.
• The node is there but red on load - missing Python deps, not a broken install. Install diffusers and the rest into the ComfyUI environment.

There are two force_images toggles in this pack - one here, one on the sampler/run nodes - and they do slightly different jobs. Leave them both alone unless you're deliberately going for a still image.