Mpi Wan Masked Video

Here's the trick this node exists for. A 360 panorama plus a depth map gives you a scene. Set a virtual camera moving through it and you can render a guide: the pixels that camera path actually sees. Some of those pixels don't exist - behind an object, past the edge of what the reconstruction covered - so you get holes. Then a video model fills the holes, and the walk looks real.

Core's Wan conditioning can't do that. WanImageToVideo conditions on one start frame. Matrix-3D's masked-video conditioning hands the model the whole guide video plus a per-pixel mask, so Wan keeps the known pixels and invents only the rest.

MpiWanMaskedVideo is that conditioning, ported from ComfyUI-SplatKit's WanI2VMaskedConditioning (MIT, mickmumpitz), which reproduces Matrix-3D's latent concat. The only change is mask polarity: Cubric Studio renders its guide with the holes white, so that's what this node expects. The README's claim is that the output is identical to SplatKit's node with invert_mask on - if you've been using that one, this is a drop-in with the mask the other way up.

How it works

The guide is cover-cropped to width × height, kept to length frames, and the hole area is painted black, so the video VAE never sees whatever colour the render left behind a hole. Alongside it, a known-mask is built (1 = known) and area-pooled down onto the latent grid - latent frame 0 is pixel frame 0, and each later latent frame is the mean of its four pixel frames - then thresholded at half and inverted into concat_mask, where 1 means "generate here".

Both the latent image and that mask are set on the positive and negative conditioning, plus clip_vision_output if you wire one (Matrix-3D's own graph takes clip vision from the first frame). The latent output is a zeros tensor already sized [1, 16, (length−1)//4+1, height/8, width/8], so you can go straight into the sampler.

That packing matters more than it sounds: it's why the model gets a mask aligned to the frames it actually generates rather than a pixel-space mask smeared across latent steps.

Inputs and outputs

• positive, negative - your CLIP-encoded conditioning. They come back out masked.
• vae - the Wan VAE, used to encode the guide.
• guide - the guide video: what is known along the camera path. Holes can be any colour; they get zeroed.
• holes - one mask per guide frame, WHITE where Wan must invent, black where the guide is known.
• width / height - default 1440 × 720, steps of 16.
• length - default 81, step 4.
• clip_vision_output (optional).

Outputs: positive, negative and latent. Pos/neg go to your sampler's conditioning, latent to its latent input, then a VAE Decode and a video save.

Installing it

cd ComfyUI/custom_nodes
git clone https://github.com/MadPonyInteractive/ComfyUi-MpiNodes
# restart ComfyUI

Beyond the pack you need a Wan 2.1 I2V model and its VAE - the conditioning is written against Wan 2.1's I2V concat path - and, to make it a camera path rather than a shot, the Matrix-3D 360 LoRA. Matrix-3D is SkyworkAI's Wan-based panorama-to-world project; the LoRA is what teaches the model that a guide video is a moving viewpoint. No extra pip dependencies: the guide encode uses core's VAE, and the conditioning helpers are core's.

Where people get burned

Polarity. White means invent. SplatKit's node ships the other convention, and the pack's own README describes the two as equivalent with invert_mask on - so if you reuse a mask built for that node, you'll get Wan inventing everything you wanted it to keep and preserving every hole. This is the number one way to waste a long render.

Frames past the end of the guide are holes by construction. A 60-frame guide with length 81 gives you 21 frames of pure invention at the tail, which is rarely what you meant. Feed length that matches, and keep it on Wan's 4k+1 grid - 81 is the standard clip.

This is the heavyweight node in the pack. A 1440 × 720, 81-frame Wan 14B run is a large latent; you'll be VRAM-bound long before you're CPU-bound, and the guide encode of 81 frames at 720p isn't free either. Batch the rest of your day around it.

A mismatch between holes and guide frame counts isn't fatal - the shorter one just defines the known region and the rest is treated as holes - which is convenient, and also exactly how a wrong-mask bug stays quiet until you watch the output.