ComfyUI-RH-daVinci-MagiHuman

ComfyUI-RH-daVinci-MagiHuman

ComfyUI nodes for daVinci-MagiHuman: generate a talking-head video with synchronized audio from a single reference image and a text prompt. The plugin wraps the official single-stream pipeline into two nodes with automatic resolution, steps, and guidance.

✨ Features

• 15B single-stream Transformer — text, video, and audio are jointly processed by self-attention.
• Talking-head quality — expressive face motion, lip sync, and audio-video alignment.
• Multilingual — Mandarin, Cantonese, English, Japanese, Korean, German, and French.
• INT8 weights — pre-quantised DiT / SR checkpoints for faster load and lower VRAM.
• Two VRAM modes — mid_vram (INT8 + GPU/CPU ping-pong, ~16 GB) and low_vram (INT8 + layer-by-layer offload, ~6.5 GB).
• Distilled model — 8-step generation without CFG.
• Optional super-resolution — second denoising pass to 540p or 1080p.
• VIDEO output — standard ComfyUI VIDEO at 25 fps with embedded audio, compatible with SaveVideo / PreviewVideo.

🛠️ Installation

Clone this repository into ComfyUI/custom_nodes:

cd ComfyUI/custom_nodes
git clone https://github.com/RH-RunningHub/ComfyUI-RH-daVinci-MagiHuman.git
cd ComfyUI-RH-daVinci-MagiHuman
pip install -r requirements.txt

Restart ComfyUI after installation. flash-attn is listed in requirements.txt and may need a matching CUDA / PyTorch build.

📦 Model Download & Installation

This repository does not include model weights. The ComfyUI nodes load INT8 DiT / SR files. Official BF16 DiT shards and official 540p_sr/ / 1080p_sr/ config directories are not required.

Model Directory Structure

ComfyUI/models/
├── MagiHuman/
│   ├── base_int8.pt                      # INT8 Base DiT (~14.25 GiB)
│   ├── distill_int8.pt                   # INT8 Distill DiT (~14.25 GiB)
│   ├── sr_540p_sr_int8.pt                # optional INT8 540p SR (~14.25 GiB)
│   ├── sr_1080p_sr_int8.pt               # optional INT8 1080p SR (~14.25 GiB)
│   ├── t5gemma-9b-9b-ul2/                # text encoder (official)
│   └── turbo_vae/                        # TurboVAE decoder (official)
│       ├── TurboV3-Wan22-TinyShallow_7_7.json
│       └── checkpoint-340000.ckpt
├── audio_checkpoints/
│   └── stable-audio-open-1.0/            # audio VAE
└── Ovi/
    └── Wan2.2-TI2V-5B/                   # video VAE

You need one of base_int8.pt / distill_int8.pt. Add an SR .pt only when sr_model is 540p_sr or 1080p_sr.

Download Methods

Method 1: Download from Hugging Face

cd /path/to/ComfyUI

# Required: pick the DiT you will use
hf download Gluttony10/ComfyUI-RH-daVinci-MagiHuman base_int8.pt --local-dir ./models/MagiHuman
hf download Gluttony10/ComfyUI-RH-daVinci-MagiHuman distill_int8.pt --local-dir ./models/MagiHuman

# Optional INT8 SR
hf download Gluttony10/ComfyUI-RH-daVinci-MagiHuman sr_540p_sr_int8.pt --local-dir ./models/MagiHuman
hf download Gluttony10/ComfyUI-RH-daVinci-MagiHuman sr_1080p_sr_int8.pt --local-dir ./models/MagiHuman

# Official TurboVAE + text encoder
hf download GAIR/daVinci-MagiHuman --local-dir ./models/MagiHuman

# External VAEs
hf download stabilityai/stable-audio-open-1.0 --local-dir ./models/audio_checkpoints/stable-audio-open-1.0
hf download Wan-AI/Wan2.2-TI2V-5B --local-dir ./models/Ovi/Wan2.2-TI2V-5B

Method 2: Download from ModelScope (for China users)

pip install modelscope
cd /path/to/ComfyUI

# Required: pick the DiT you will use
modelscope download --model Gluttony10/ComfyUI-RH-daVinci-MagiHuman base_int8.pt --local_dir ./models/MagiHuman
modelscope download --model Gluttony10/ComfyUI-RH-daVinci-MagiHuman distill_int8.pt --local_dir ./models/MagiHuman

# Optional INT8 SR
modelscope download --model Gluttony10/ComfyUI-RH-daVinci-MagiHuman sr_540p_sr_int8.pt --local_dir ./models/MagiHuman
modelscope download --model Gluttony10/ComfyUI-RH-daVinci-MagiHuman sr_1080p_sr_int8.pt --local_dir ./models/MagiHuman

# Official TurboVAE + text encoder
modelscope download --model GAIR/daVinci-MagiHuman --local_dir ./models/MagiHuman

Audio / video VAEs are still downloaded from Hugging Face as in Method 1.

Method 3: Manual Download

| Model | Link | Description | |-------|------|-------------| | INT8 DiT / SR package | Hugging Face / ModelScope | base_int8.pt, distill_int8.pt, optional SR files | | Official MagiHuman assets | GAIR/daVinci-MagiHuman | T5-Gemma, TurboVAE | | Audio VAE | stabilityai/stable-audio-open-1.0 | Required audio decoder | | Video VAE | Wan-AI/Wan2.2-TI2V-5B | Required video decoder |

Model Selection Guide

| Your GPU VRAM | Recommended setup | Notes | |---------------|-------------------|-------| | ~8 GB | distill + low_vram | Layer-by-layer offload, ~6.5 GB peak without SR | | 16 GB | distill or base + mid_vram | INT8 + GPU/CPU ping-pong | | 24 GB | mid_vram + 540p_sr | Recommended for RTX 4090 class | | 48 GB+ | mid_vram + 1080p_sr | A6000 / A100 class |

🚀 Usage

Example Workflow

Import examples/magihuman_basic_workflow.json into ComfyUI.

The example demonstrates:

1. RH MagiHuman Model Loader — distill + mid_vram + 540p_sr
2. LoadImage — reference face image
3. RH MagiHuman Generate — prompt, duration, seed → VIDEO
4. SaveVideo — write the 25 fps video with audio

Prompt Tips

Structure the prompt in three parts:

1. Main body (150–200 words) — appearance, facial dynamics, vocal delivery, and camera.
2. Dialogue

   Dialogue:
   <character description, language>: "Line content"
   

3. Background sound

   Background Sound:
   <Description or "No prominent background sound">
   

See prompts/enhanced_prompt_design.md for the full prompt specification.

📝 Node Reference

RH MagiHuman Model Loader

| Parameter | Type | Default | Description | |-----------|------|---------|-------------| | model_type | base / distill | — | Base (32 steps) or distilled (8 steps) | | use_turbo_vae | bool | True | Faster video decoding | | vram_mode | mid_vram / low_vram | mid_vram | VRAM strategy | | sr_model | none / 540p_sr / 1080p_sr | none | Optional INT8 super-resolution | | torch_compile | bool | False | First run compiles ~5 min |

RH MagiHuman Generate

| Parameter | Type | Default | Description | |-----------|------|---------|-------------| | image | IMAGE | optional | Reference face image | | prompt | string | — | Speech, expression, and scene | | seconds | int | 4 | Duration (1–20) | | seed | int | 42 | Random seed | | audio | AUDIO | optional | Optional driving audio for lip sync |

Auto-determined parameters:

• Resolution — landscape 448×256, portrait 256×448
• SR resolution — 540p 896×512 / 512×896, 1080p 1920×1088 / 1088×1920
• Steps — base 32, distill 8, SR 5
• Guidance — video 5.0, audio 5.0, SR video 3.5
• CFG — base full_cfg, distill no_cfg

📄 License

This project is licensed under the Apache License 2.0. Redistribution and modified versions must retain the license and attribution notices, including NOTICE.

Upstream daVinci-MagiHuman is also Apache 2.0.

🔗 Links

• Original project
• Paper
• Official demo
• Official models
• INT8 converted weights (Hugging Face)
• INT8 converted weights (ModelScope)
• Plugin repository: ComfyUI-RH-daVinci-MagiHuman

🙏 Acknowledgements

This project is based on daVinci-MagiHuman by SII-GAIR and Sand.ai. It also builds on Wan2.2, Turbo-VAED, and stable-audio-tools.