ControlNext Decode

ControlNext Decode is the boring, necessary tail end of the ControlNeXt-SVD diffusers path. ControlNext Sampler gives you a LATENT; this node turns it into actual IMAGE frames you can feed to a video-combine node and render to mp4. If you've used VAEDecode in normal ComfyUI, the shape of this will feel familiar - with one quirk that bites people.

What it actually does

It decodes the sampled latents with SVD's temporal VAE, exactly the decode_latents step from the ControlNeXt pipeline. The two inputs that matter:

• controlnext_pipeline - the same CONTROLNEXT_PIPE from (Down)Load ControlNeXt that you gave the sampler. It carries the VAE and image processor, so keep one loader feeding both nodes.
• samples - the LATENT output of ControlNext Sampler. Nothing else produces it.

The one knob, decode_chunk_size (default 4), is a pure memory control: it decodes a handful of frames at a time instead of the whole clip at once, so you can trade a little speed for fitting in VRAM. The node even tries chunk 1 automatically if the decode blows up at your setting, so the failure mode is "slow," not "crash."

Output is images (IMAGE) in frame order, ready for VHS_VideoCombine or a save node.

The quirk: your first frame disappears

Look at the source and you'll see the frames get squeezed and sliced with [1:] - the decode drops the first frame before handing back the tensor. That's deliberate: ControlNeXt-SVD is image-to-video, the reference image is frame zero, and the latent sequence starts at frame one. So if your output mp4 is one frame shorter than your pose input, that's expected, not a bug. Same reason your video is one frame shorter than the clip you loaded into ControlNext GetPoses - nobody lost data, the first frame is the reference image.

Why decode separately at all?

Because the sampler returns latents and doesn't commit to a resolution or a decode budget. Splitting sampling and decoding like this is the whole reason you can crank decode_chunk_size down on a small card without touching the sampling settings, and it's also why you can re-decode a single sample with a different chunk size if the first pass OOM'd - the latents are already there.

Installing it

Part of the same pack as the sampler - install kijai/ComfyUI-ControlNeXt-SVD via ComfyUI Manager (search "ComfyUI-ControlNeXt-SVD") or:

cd ComfyUI/custom_nodes
git clone https://github.com/kijai/ComfyUI-ControlNeXt-SVD
# restart ComfyUI

It leans on the pack's diffusers/transformers requirements and the auto-downloaded SVD VAE from models/diffusers/stable-video-diffusion-img2vid-xt-1-1/. If the loader already ran, there's nothing new to download here.

Common issues

• "First frame missing" - expected, see above. Line the reference frame up yourself when comparing pose input to output.
• OOM on long clips - lower decode_chunk_size. The node already falls back to 1 on failure, so if it still dies, you're out of VRAM for the VAE at that resolution; shrink the resolution.
• Noisy/dim frames - check that you used this node and not a generic VAEDecode. SVD's temporal VAE and its scaling factor aren't a normal latent decode, and feeding the ControlNeXt latents through a stock VAE decode is a one-way ticket to garbage.