ComfyUI_DICE_Talk

ComfyUI_DICE_Talk

Use DICE-Talk in ComfyUI，which is a method about 'Correlation-Aware Emotional Talking Portrait Generation'.

Update

• 重新封装，去掉diffuser和transformer的版本限制/ repackage, remove the version limit of diffuser and transformer

1. Installation

In the ./ComfyUI/custom_nodes directory, run the following:

git clone https://github.com/smthemex/ComfyUI_DICE_Talk.git

2. Requirements

pip install -r requirements.txt

3.Models

• 3.1 download checkpoints from EEEELY/DICE-Talk 从抱脸下载必须的模型,文件结构如下图
• 3.2 download openai/whisper-tiny
• 3.3 SVD checkpoints svd_xt.safetensors or svd_xt_1_1.safetensors

--  ComfyUI/models/sonic/
    |-- audio_linear.pth
    |-- emo_model.pth
    |-- pose_guider.pth
    |-- dice_unet.pth #  rename for different from sonic
    |-- yoloface_v5m.pt  #can use sonic 可以用sonic的，不需要复制
    |-- whisper-tiny/  #can use sonic 可以用sonic的，不需要复制
        |--config.json
        |--model.safetensors
        |--preprocessor_config.json
    |-- RIFE/  #can use sonic 可以用sonic的，不需要复制
        |--flownet.pkl
--   ComfyUI/models/checkpoints
    ├── svd_xt.safetensors  or  svd_xt_1_1.safetensors

4.Example

5 .Citation

@misc{tan2025disentangleidentitycooperateemotion,
      title={Disentangle Identity, Cooperate Emotion: Correlation-Aware Emotional Talking Portrait Generation}, 
      author={Weipeng Tan and Chuming Lin and Chengming Xu and FeiFan Xu and Xiaobin Hu and Xiaozhong Ji and Junwei Zhu and Chengjie Wang and Yanwei Fu},
      year={2025},
      eprint={2504.18087},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2504.18087}, 
}