DreamX First Frame AV Latent

DreamX-Creator is an image-and-prompt-to-talking-video model: you give it one still frame, it gives you a moving clip with synchronised sound. DreamX First Frame AV Latent is the node where that request gets turned into shapes the sampler can actually chew on. You hand it a picture and a duration; it hands back a conditioned latent holding both a video stream and a silent audio stream, plus the exact numbers it chose.

It's the node where most "why is my clip the wrong length" questions die.

Inputs

You wire up positive and negative (from two CLIP Text Encode nodes fed by the UMT5 loader or bundle), the vae - which must be the Wan 2.2 48-channel video VAE - and start_image. Then four widgets:

• duration - seconds, 0.25 to 60, default 5.
• fps - default 24. This is the frame rate of the output clip, and it decides what "5 seconds" means.
• target_spatial_tokens - default 880, and this is the one people don't read. It's a target count of 32×32 pixel patches, not a resolution. 256 tokens is a 512×512 square; the pack's own quick-start profile uses 256 and gets 512×512 out of a square input. 880 is the released 5-second preset and lands around 950×950 square. Push it up and you buy detail with VRAM you may not have.
• batch_size - fixed at 1, with the tooltip spelling out why: DreamX emits one synchronised video/audio pair per execution. Set it to anything else and the node raises. Want variants? Set ComfyUI's queue batch count.

Outputs

positive and negative come back out with a dreamx_fps value baked into the conditioning. latent is the packed thing you send to the sampler. And then a quiet gift: width, height, frames and fps as separate scalars.

Wire those into your video-saving chain and you stop guessing. frames and fps are exactly what Create Video wants, and they match what you'll actually get - no divider maths, no "why is my MP4 1.875 s".

What it does underneath

Three things, in order. First it picks a resolution: it searches multiples of 32 near your source image's aspect ratio until it hits your token target, then bicubic-resizes the first frame to exactly that. Square in, square out, roughly; a 2:1 source stays near 2:1.

Then it encodes that frame with the Wan VAE and checks the result is (1, 48, 1, H/16, W/16). If it isn't, you get a clear message naming the 48-channel requirement - which is exactly what you'll see if you plugged a Wan 2.1 VAE (16 channels) into this graph.

Finally it builds the pair. The video latent is a zeroed Wan-format tensor with your encoded first frame written into the first temporal position and a noise_mask that zeroes that same position - that's the mechanism that pins frame one to your image and lets the sampler invent the rest. The audio latent is a zero tensor with a noise mask of all ones, sized ceil(seconds × 50) frames wide, because the audio VAE's hop length gives you exactly 50 latent frames per second at 48 kHz.

Frame count gets snapped, not rounded: DreamX truncates your requested frames and floors to a 4N+1 count the Wan VAE can encode. Ask for 5 seconds at 24 fps and you get 117 frames - 4.875 s. Ask for 2 seconds and you get 45, which is 1.875 s, and the node tells you so via the frames output instead of letting you find out in a video player.

Install

ComfyUI Manager → DreamX Creator T8, or:

cd ComfyUI/custom_nodes
git clone https://github.com/T8mars/Comfyui-DreamX-Creator-T8.git
cd Comfyui-DreamX-Creator-T8
python -m pip install -r requirements.txt

Weights are separate, ~54 GB, and this node needs all of creator/, audio_vae/ and wan2.2_ti2v_5b/ present:

python -m pip install -U huggingface_hub
hf download t8star/DreamX-Creator-Comfy --local-dir ComfyUI/models/dreamx_creator
python scripts\verify_models.py

Common issues

The clip is always a bit shorter than asked. Working as designed - the 4N+1 snap. Wan's native window is around 81 frames and that limit doesn't evaporate because DreamX wants audio too; troubleshooting.md has the video-side version of it.

You want an exact output size and can't set one. Resolution is derived, not picked. Crop or resize the input image and the node will follow it.

OOM at 880 tokens on a 24 GB card. Try 512, and check what else is resident. The Creator and Refiner are meant to run in separate graph phases, and the text encoder is its own VRAM budget on top.