(Down)Load OpenSora Model

This is the first node you reach for in any ComfyUI-OpenDiTWrapper workflow. It loads the STDiT3-XL/2 diffusion backbone - the ~2.8-billion-parameter transformer that actually turns noise into video frames. Everything else in the pack exists to feed this model or render its output, so if a workflow crashes with a missing opendit_model, this is where you start.

The pack is kijai's ComfyUI port of OpenSora v3, hpcaitech's open text-to-video / image-to-video model. If you've seen OpenSora mentioned and wondered how to actually run it inside ComfyUI instead of fighting a research repo, this is the answer. Fair warning up front: it's a mid-2024 port that kijai built and then largely left as "WORK IN PROGRESS" while he moved on to his Wan and LTX video wrappers. Expect a working but rough-around-the-edges tool, not a polished release.

How it works

On the first run, the node downloads the checkpoint from HuggingFace (hpcai-tech/OpenSora-STDiT-v3) straight into ComfyUI/models/opensora, skipping the *ema* weights so you don't pull gigabytes you'll never use. It then builds the STDiT3 model and parks it on ComfyUI's offload device - system RAM, typically - so it only climbs onto the GPU when the sampler actually asks for it. That offloading is the whole reason this pack fits in 15GB where the original OpenSora repo wanted far more.

One nice detail: the loaded model is cached on the node instance. Re-running the queue doesn't reload the weights from disk each time, which on a multi-gigabyte model saves you real minutes.

The two inputs and one output

• model - a single choice, hpcai-tech/OpenSora-STDiT-v3. There's one option and it's the right one, so you leave it alone.
• precision - bf16 (default), fp16, or fp32. bf16 is the sweet spot; fp32 is there for correctness at roughly double the VRAM and half the speed. Just use bf16 unless you have a specific reason not to.

The lone output, opendit_model (type OPENDITMODEL), plugs into OpenDiTSampler's opendit_model input. That's the node's entire job: load once, hand the model off.

Installing

ComfyUI Manager → Install Custom Nodes → search "ComfyUI-OpenDiTWrapper", or:

cd ComfyUI/custom_nodes
git clone https://github.com/kijai/ComfyUI-OpenDiTWrapper

Then install the Python deps and restart ComfyUI:

pip install -r requirements.txt
pip install xformers --no-deps

On the Windows portable build, the same commands become:

python_embeded\python.exe -m pip install -r .\ComfyUI\custom_nodes\ComfyUI-OpenDiTWrapper\requirements.txt
python_embeded\python.exe -m pip install xformers --no-deps

That requirements.txt is the real dependency list - transformers, diffusers, accelerate, timm, sentencepiece, protobuf, omegaconf and friends. The xformers line isn't optional flavor; the node code checks for flash attention at import time and prints a loud warning that it's "using much slower torch SDP attention" when it's missing.

Common issues

The big one is VRAM. The model alone is ~5.5GB of weights in bf16 before a single frame is rendered, and it shares the card with the T5 encoder and the VAE. The README's numbers are the realistic floor: 48 frames at 768x512 fits in 15GB, smaller or lower-res runs squeeze into 10GB. If you're below that, cut the frame count and resolution first - the sampler defaults to a tiny 426x240 for a reason.

Also expect the first queue run to stall while the model downloads. That's normal, not a hang - it's pulling a multi-gigabyte snapshot into ComfyUI/models/opensora before anything renders.