VideoLaVITVideoReconstruction

This is the pack's showcase node, and it's a demo of compression, not creativity. Feed it a real video file and it re-renders a chunk of that video from just one keyframe plus its motion vectors - proof that Video-LaVIT's "decompose video into keyframe + motion tokens" idea actually captures enough information to rebuild pixels. It's the most self-contained way to see what makes this model different from a conventional diffusion video model, and it doesn't need a prompt at all.

How it works

The node reads your video_path, then sample_video_clips does the heavy prep. It extracts frames and motion vectors from the compressed bitstream (that's where the motion-vector-extractor package and FFmpeg with MPEG-4 Part 2 re-encoding come in - the video gets re-encoded so motion data can be pulled out). It finds I-frame boundaries, picks a valid starting point at random, and hands the model a keyframe tensor plus stacked motion vectors. The detokenizer then diffuses the clip back out - max_frames (default 24) at a time, in chunks of decode_chunk_size (default 8). Since the start frame is randomly chosen, the same input video gives you a different reconstructed segment each run unless you fix the seed.

The guidance knobs are the interesting part. noise_aug_strength (0.02) adds noise to the keyframe before decoding, which gives the diffusion model room to work; use_linear_guidance (on) ramps guidance linearly through the clip, between min_guidance_scale (1.0) and max_guidance_scale (3.0). More guidance = the reconstruction clings harder to the keyframe; less = it drifts. For a fidelity demo you mostly leave these alone.

The inputs

• model - a VideoLaVITVideoDetokenizer from VideoLaVITVideoDetokenizerLoader.
• video_path - absolute path to the video file.
• width / height - decode resolution, default 576×320. Multiples of 64.
• decode_chunk_size (8), max_frames (24) - chunk and clip length.
• noise_aug_strength (0.02), use_linear_guidance, max_guidance_scale (3), min_guidance_scale (1) - reconstruction fidelity.
• seed (16) - which random segment you get back.

Output is IMAGE - the reconstructed frames, wired to VHS_VideoCombine in the bundled workflow, with VHS_LoadVideo + VHS_FILENAMES_STRING_LaVIT feeding it the path.

Install

Pack install is the usual (ComfyUI Manager search "ComfyUI-LaVIT", or git clone https://github.com/chaojie/ComfyUI-LaVIT into custom_nodes), but this node has the most extra setup: the motion-vector-extractor pip dependency and an FFmpeg build with the MPEG-4 Part 2 codec, both required for motion extraction. And the recurring trap - the pack's root requirements.txt is empty, so install VideoLaVIT/requirements.txt manually.

Where people get burned

The classic errors are environment-level: motion-vector-extractor missing, FFmpeg without the right codec, or the video_3d_unet.bin file missing from the model folder - all of which surface as opaque crashes. Also remember it's a reconstruction demo: the output is the model's lossy re-render of the original, not an enhancement. It'll be softer, slightly warped, occasionally with invented detail - that's the point, that's the tokenizer's fingerprint. It's the best node in the pack to show a friend, and the worst one to build a production workflow on.