ComfyUI-TRELLIS2_Motion
A comprehensive ComfyUI node package that uses Microsoft's TRELLIS2 to transform video footage into 3D mesh reconstructions with advanced rendering effects.
Nodes (9)
Trajectory controls for your 3D render
The tiny utility that stitches your VRAM-limited runs back together
The nvdiffrast renderer with five looks
Re-shoot a whole video from a camera that was never there
The classic animator's trick, in a ComfyUI node
The post-pass that makes a render look finished
One image in, a PBR mesh out — meet the heart of this pack
The one-node 'angle shift' that hides the whole pipeline from you
Run TRELLIS2 over a whole video, frame by frame
ComfyUI-TRELLIS2_Motion
A comprehensive ComfyUI node package that uses Microsoft's TRELLIS2 to transform video footage into 3D mesh reconstructions with advanced rendering effects.
Features
- Image to 3D Mesh: Convert single images to high-quality 3D meshes with PBR materials
- Video Processing: Batch process video frames through TRELLIS2
- Multiple Rendering Modes: PBR, wireframe, cel/toon shading, normal map, depth map
- Onion Skinning: Motion ghosting effect with configurable interval (every 5, 10, 30 frames)
- Camera Control: Orbit, dolly, pan trajectories with animation support
- Lighting Presets: Studio, outdoor, dramatic, flat lighting configurations
- Post-Processing: Bloom, depth of field, motion blur, outlines
- Progress Bars: Visual feedback on all long-running operations
- Auto Model Download: Automatically downloads TRELLIS2 model (~8GB) on first use
Nodes
| Node | Description | |------|-------------| | Trellis2Predict | Convert single image to 3D mesh with PBR materials | | VideoToMeshes | Batch process video frames through TRELLIS2 | | CameraTrajectory | Define camera movement (orbit, dolly, pan, static) | | MultiAngleCamera | Generate multiple viewing angles for grid preview | | MeshRenderer | Render mesh from camera viewpoint with effects | | MeshSequenceRenderer | Render video sequence with consistent camera | | OnionSkinning | Add motion ghosting effect to rendered frames | | RenderingEffects | Apply bloom, DoF, cel shading, outlines | | Trellis2VideoAngleShift | All-in-one convenience node |
Installation
1. Clone the Repository
cd ComfyUI/custom_nodes
git clone https://github.com/yourusername/ComfyUI-TRELLIS2_Motion.git
cd ComfyUI-TRELLIS2_Motion
2. Install Dependencies
python install.py
3. Install TRELLIS2 (Required)
pip install git+https://github.com/microsoft/TRELLIS.git
4. Install nvdiffrast (Required for Rendering)
pip install git+https://github.com/NVlabs/nvdiffrast.git
5. Restart ComfyUI
The nodes will appear under the TRELLIS2 category.
Requirements
- Python: 3.10+
- CUDA: Required (tested on CUDA 11.8+)
- GPU: NVIDIA GPU with 24GB+ VRAM recommended (A100, RTX 4090)
- ComfyUI: Latest version
Workflows
Pre-built workflows are included in the workflows/ folder:
| Workflow | Description |
|----------|-------------|
| single_image_novel_view.json | Basic image to 3D with camera control |
| video_angle_shift.json | Full video processing pipeline |
| onion_skinning_demo.json | Motion ghosting showcase |
| effects_showcase.json | Cel shading and post-processing demo |
| multiview_preview.json | 8-angle grid preview |
Usage Examples
Single Image Novel View
LoadImage → Trellis2Predict → MeshRenderer → SaveImage
↑
CameraTrajectory
Video Angle Shift
VHS_LoadVideo → VideoToMeshes → MeshSequenceRenderer → VHS_VideoCombine
↑
CameraTrajectory
With Onion Skinning
VHS_LoadVideo → VideoToMeshes → MeshSequenceRenderer → OnionSkinning → VHS_VideoCombine
Onion Skinning Parameters
| Parameter | Description | Default |
|-----------|-------------|---------|
| ghost_count | Number of ghost frames | 3 |
| frame_interval | Frames between ghosts | 5 |
| ghost_opacity | Base opacity of ghosts | 0.3 |
| opacity_falloff | linear, exponential, constant | linear |
| ghost_color_mode | original, tinted, silhouette | tinted |
| past_tint | Color for past frames | Blue (#0066FF) |
| future_tint | Color for future frames | Orange (#FF6600) |
Rendering Modes
- PBR: Full physically-based rendering with materials
- Wireframe: Edge visualization
- Cel/Toon: Quantized shading with outlines
- Normal: RGB-encoded surface normals
- Depth: Grayscale depth visualization
Lighting Presets
- Studio: Professional 3-point lighting
- Outdoor: HDR sky with sun
- Dramatic: High contrast single light
- Flat: Even lighting for cel shading
Performance
| Resolution | TRELLIS2 Time | Render Time | Total/Frame | |------------|---------------|-------------|-------------| | 512x512 | ~3s | ~0.1s | ~3.1s | | 1024x1024 | ~10s | ~0.2s | ~10.2s |
Tested on NVIDIA A100 80GB
Comparison with SHARP
| Aspect | TRELLIS2 | SHARP | |--------|----------|-------| | Output | PBR Mesh + Materials | Gaussian Splats | | Processing Time | 3-60s per frame | <1s per frame | | Materials | Full PBR | SH coefficients | | Rendering | nvdiffrast | gsplat | | Quality | Higher fidelity meshes | Real-time splatting |
Troubleshooting
"CUDA out of memory"
- Reduce resolution from 1024 to 512
- Increase frame_skip to process fewer frames
- Use a GPU with more VRAM
"nvdiffrast not found"
pip install git+https://github.com/NVlabs/nvdiffrast.git
"TRELLIS not found"
pip install git+https://github.com/microsoft/TRELLIS.git
Credits
- Microsoft TRELLIS - Core 3D reconstruction model
- nvdiffrast - CUDA mesh rendering
- ComfyUI - Node-based UI framework
License
MIT License - See LICENSE file for details.