Jolly Cosmos3 Text-to-Video

Let's get the expectations out of the way, because they're the thing people hit hardest. Cosmos3-Nano's text-to-video quality is not great. That's not a knock on this node - it's the community's verdict on the model itself, repeated across the release threads: smearing, artifacts, object shapes that warp. NVIDIA built Cosmos3 as a world model for synthetic data and robotics, where physics and scene understanding matter more than cinematic polish. If you want gorgeous text-to-video, Wan 2.2 and LTX remain the friendlier defaults. If you want a 16B model that reasons about motion and physics and fits the Cosmos3 architecture, this is it.

JollyCosmos3TextToVideo takes a cosmos3_pipe from JollyCosmos3ModelLoader plus a prompt and produces frames (an IMAGE tensor, wire it into a Video Combine / VHS node to save an mp4) and audio (an AUDIO dict at 48kHz, only populated if you enable sound). The mechanism is the spec-compliance core of this whole pack: your prompt goes through the Qwen2 chat template, gets the duration template ("The video is X.X seconds long and is of Y FPS.") and resolution template appended, text and vision tokens are joined in a unified 3D mRoPE sequence, and sampling runs under UniPCMultistepScheduler(flow_shift=10.0) - matching how the model was trained, which is precisely what ComfyUI's stock Cosmos 1/1.5 nodes can't do.

The inputs that matter

• num_frames - default 189, and the tooltip spells out the arithmetic: 189 at 24 fps is ~7.9 seconds, per spec. Range is 5–400. Shorter clips are dramatically faster and lighter.
• fps - 24 default, 10–30 allowed.
• width / height - 1280x720 default, but 832x480 (480p) is where Nano lives comfortably.
• steps / guidance_scale / seed - 35 steps and CFG 6 are the shipped defaults and they're fine. The seed feeds the generator directly, so 0 is a fixed seed - you'll get the same clip every run until you change it.
• generate_sound - off by default. The tooltip is the author's own warning: it only works on "sound-capable checkpoints," and the loader tells you whether yours is by checking for a sound tokenizer. Nano supports it; leave it off until you're on hardware with headroom.

A practical note on prompts: the node accepts plain English, but it's built to receive the structured JSON from JollyCosmos3StructuredPrompt - the temporal_caption field is what carries the motion, and the model responds to it. And since this is a flow-matching model with an LLM encoder, the negative_prompt field is a weaker lever than you're used to from SDXL - don't sink hours into negative prompt engineering here.

Installing and the VRAM wall

Same as every node in the pack:

cd ComfyUI/custom_nodes
git clone https://github.com/JOLLYKRONK/jolly-cosmos3-comfyuinodes
pip install -U 'diffusers @ git+https://github.com/huggingface/diffusers.git'
huggingface-cli download nvidia/Cosmos3-Nano --local-dir ComfyUI/models/diffusers/cosmos3_nano

Restart and the nodes show up under Cosmos3. The diffusers git-main install is mandatory - released diffusers has no Cosmos3OmniPipeline.

The wall is VRAM. Nano is ~32GB of weights in BF16; people with 2x3090 generate 1280x720 clips fine but OOM on the video decode step. The pack patches the VAE decode to clear the CUDA cache before each call to dodge fragmentation, and you should still expect to drop resolution or frames on anything short of a 5090-class card. Start at 832x480, get a clip that works, then push up. That's the whole troubleshooting flow, honestly.