Video to Meshes (Batch TRELLIS2)

VideoToMeshes is Trellis2Predict's loud, hungry sibling. Where the single-image node takes one picture and gives you one mesh, this one takes a batch of video frames and runs the full TRELLIS2 reconstruction on each one, producing a TRELLIS_MESH_SEQUENCE - the raw material for the motion side of this pack.

That's the whole pitch of ComfyUI-TRELLIS2_Motion: not "make a 3D model," but "reconstruct a 3D model per frame and then re-render it from a camera that wasn't there." This node is step one of that pipeline. It's also the reason the README recommends 24GB+ VRAM, so have a realistic look at your card before you wire it up.

How it works

The node takes your frames ([N, H, W, C], the standard ComfyUI image batch - load them with a video loader like VHS and you're set), then:

1. Extracts frames according to frame_skip - it grabs every Nth frame rather than processing everything.
2. For each kept frame: optional rembg background removal, then pipeline.run() through TRELLIS2 (the video path pulls the model via the pack's download helper, which targets microsoft/TRELLIS-2-4B).
3. Converts each result into the pack's mesh format and attaches the original frame index so downstream nodes can keep track of where in the timeline each mesh came from.
4. Clears the CUDA cache every ten frames, because a 4B-parameter model reconstructing dozens of frames will eat your VRAM for breakfast if you let it.

The output isn't a video - it's a TRELLIS_MESH_SEQUENCE, a dictionary of meshes, frame_indices, total_frames, and frame_skip. Wire that into MeshSequenceRenderer to turn it back into images, or into MergeMeshSequences if you're combining chunks.

The inputs that matter

• images - required. Video frames as a batch. Nothing else in this node matters as much as the two numbers below, honestly.
• frame_skip - default 5, range 1–30. This is your cost dial. Every frame at 1024 on a 24GB card is roughly the README's ~10s benchmark; at 30fps video that's 300s per second of footage. Skip 5 frames and a 10-second clip becomes 60 reconstructions instead of 300. Start here before you touch anything else.
• max_frames - 0 means no limit. A hard cap so a long clip can't accidentally run for an hour.
• resolution - 512 or 1024. Same trade as the single-image node: 1024 looks better, 512 fits in less VRAM. The difference in quality on meshes is real but not night-and-day.
• remove_background - default true, and you want it on for video too. Backgrounds eat reconstruction budget frame after frame.
• seed - 0 means a different random seed per frame. Set a fixed seed to keep per-frame meshes consistent, which matters more than you'd think when the renderer later re-shoots the whole sequence from a new angle.

Installing

Same as every node in this pack:

cd ComfyUI/custom_nodes
git clone https://github.com/styletransfer/ComfyUI-TRELLIS2_Motion
cd ComfyUI-TRELLIS2_Motion
python install.py --full
pip install git+https://github.com/microsoft/TRELLIS.git
pip install git+https://github.com/NVlabs/nvdiffrast.git

Restart ComfyUI; the nodes land under the TRELLIS2 category.

Common issues

• CUDA out of memory mid-run. The node skips a frame on error and keeps going, which is forgiving - but if you're OOMing at every frame you're just getting garbage. Cut frame_skip upward, drop resolution to 512, or process the clip in chunks with MergeMeshSequences on the other end. Processing 20-frame chunks through two VideoToMeshes nodes and merging is the standard workaround for cards that can't hold a full run.
• Everything fails with "TRELLIS not available." The trellis package isn't installed or isn't findable. Install it (pip install git+https://github.com/microsoft/TRELLIS.git) and make sure it's outside custom_nodes/ - the pack's own code hunts for the repo at sibling paths, and ComfyUI loading it as a node pack is a one-way ticket to confusion.
• It's slow and you can't tell if it's working. It is working; it's a 4B model running once per kept frame. The node shows a progress bar on all long-running work, so watch that rather than the console. And yes, this is the node where the README's own comparison table (TRELLIS2 vs splat-based SHARP, 3–60s per frame vs <1s) stops being academic. If your end goal is real-time, this pack is not that.