Jolly Cosmos3 Image-to-Video

If you're going to run NVIDIA's Cosmos3-Nano at all, this is the node. The community's early verdict on the model is remarkably consistent: text-to-video quality is shaky and smeary, but image-to-video comes out "totally acceptable." Cosmos3-Nano is a 16B world model aimed at synthetic data for robotics, not at winning art contests - and I2V is where that trade-off actually lands in your favor. You feed it a still frame and it makes it move, with optional synchronized audio.

Cosmos3 is a different architecture from the Cosmos your ComfyUI already ships. The built-in EmptyCosmosLatentVideo and friends implement Cosmos 1/1.5 (T5 cross-attention, EDM sampling). Cosmos3 uses a Qwen2 LLM encoder, unified 3D mRoPE position embeddings, and flow matching - that's why this pack exists, and why this node needs a cosmos3_pipe from JollyCosmos3ModelLoader rather than anything stock. The loader builds the diffusers Cosmos3OmniPipeline with a UniPCMultistepScheduler(flow_shift=10.0), and the node routes your prompt through the Qwen2 chat template, anchors frame 0 to your image via VAE encode, and applies a conditioning mask so frame 0 stays put while the rest denoises.

The inputs that matter:

• image - your starting frame, straight from a Load Image node. Any ComfyUI IMAGE works.
• num_frames - default 189, which is ~7.9s at the default 24 fps. 5–400 allowed; fewer frames = faster and less VRAM.
• width / height - default 1280x720, but if your card groans, 832x480 (the official 480p tier) is where Nano is comfortable.
• steps / guidance_scale / seed - 35 steps and CFG 6 are the sensible defaults. Note the seed is wired straight into the generator, so 0 is a fixed seed - same seed, same clip. Bump it manually when you want variety.
• generate_sound - off by default; only does anything if the checkpoint actually has a sound tokenizer (the loader detects this for you).

Outputs are frames (an IMAGE tensor - feed it to a Video Combine / VHS node to actually save an mp4) and audio (an AUDIO dict at 48kHz). If generate_sound is off, audio comes back empty, so don't wire it and expect silence to appear.

Installing it

Same story as the whole pack: JollyCosmos3ModelLoader first - this node does nothing without its pipe. Get the pack, restart ComfyUI, grab the model.

cd ComfyUI/custom_nodes
git clone https://github.com/JOLLYKRONK/jolly-cosmos3-comfyuinodes
pip install -U 'diffusers @ git+https://github.com/huggingface/diffusers.git'
huggingface-cli download nvidia/Cosmos3-Nano --local-dir ComfyUI/models/diffusers/cosmos3_nano

The diffusers git-main install is non-negotiable: Cosmos3OmniPipeline doesn't exist in released versions. The Nano weights are roughly 32GB, and the pack's README is upfront that this is RTX 5090 / RTX 6000 Pro territory - people running 2x3090 via vLLM still hit OOMs decoding 1280x720 clips.

Where people get burned

VRAM is the whole game. One community tester with 2x3090 could generate 1280x720 fine but OOM'd during video decode - the pack works around exactly this by patching the VAE decode to clear the CUDA cache before each call, but you can still hit the wall. Drop to 832x480 or trim frames first. And don't fight the model's nature: negative prompts are a weak lever on flow-matching LLM-encoded models, so don't obsess over the negative_prompt field - the Structured Prompt node's generated JSON negative is more about schema compliance than visible effect.

Reach for this over the built-in Cosmos nodes when you actually want Cosmos3's physics-leaning world model. If you just want pretty motion, Wan 2.2 and LTX remain the friendlier defaults - this is the niche pick, and within it, I2V is the right mode.