ThinkSound

Most video models come out of the box deaf. Wan, Hunyuan, LTX - they render frames and hand you a silent file, and native audio is rare enough that people write blog posts about the two models that do it. That gap is exactly what ThinkSound is for. It's the ComfyUI wrapper around Alibaba's FunAudioLLM ThinkSound, a "any2audio" framework: give it a video (or even a still image) and a description of what it should sound like, and it synthesizes a sound effect track and muxes it back onto the footage.

This ThinkSound node is the engine of the whole pack. The other three nodes - LoadCaption, LoadCoTDescription, LoadOThinkSoundVideo - just feed it text and a file path. If you're here to add sound to a clip, this is the one that does the work.

How it actually works

ThinkSound isn't a text-to-speech model and it isn't a music generator. It's a rectified-flow diffusion model - same family as Stable Audio and the audio side of the video-audio diffusion crowd - built on a 21-layer multimodal DiT with 14 more fused layers. It conditions on four things at once: T5 text embeddings, MetaCLIP image and text embeddings, and SyncNet features that tie audio timing to on-screen motion. Those come together in a latent space produced by a pretrained VAE, and the model denoises roughly nine seconds of 44.1 kHz stereo audio from noise.

The "Chain-of-Thought" part is the interesting bit. Instead of conditioning on one caption, the pipeline uses two text inputs: a short title and a longer, step-by-step description of the soundscape - "start with printing, then the beep, then the cat paws the machine." That detailed plan is what guides generation, which is why you'll see the CoT description treated as a first-class input rather than an afterthought.

The wrapper's job is plumbing. It writes your title and description into a cot.csv, then shells out to two external scripts: extract_latents.py to pull conditioning features from the video, then predict.py to run the diffusion loop. Finally it calls ffmpeg to glue the generated .wav back onto the clip as an .mp4.

Inputs

All three inputs are required, and all three come from the pack's own loader nodes:

• video - a VIDEO-typed path from LoadOThinkSoundVideo. Despite the type name it's a filesystem path, not a decoded tensor.
• title - the short caption from LoadCaption ("Printer Printing" by default).
• description - the detailed CoT sound plan from LoadCoTDescription.

There are no outputs. This is effectively a terminal node: it writes a combined _with_audio.mp4 into a temp results folder and reports progress to the console.

Installing the pack

ComfyUI Manager has it if you search "ComfyUI-ThinkSound", or clone it by hand:

cd ComfyUI/custom_nodes
git clone https://github.com/Yuan-ManX/ComfyUI-ThinkSound.git

Then the two things the README doesn't warn you enough about. First, pip install -r requirements.txt is a full pinned environment - torch 2.6.0, numpy 1.23.5, gradio, flash-attn, the lot. Run that inside your existing ComfyUI venv and it will happily clobber your torch and numpy versions. Use a dedicated env. Second, the README wants conda install -y -c conda-forge 'ffmpeg<7', and the weights are a separate multi-gigabyte download:

git lfs install
git clone https://huggingface.co/liuhuadai/ThinkSound ckpts

(or grab the same weights from ModelScope). Those must land in ckpts/ inside the pack directory.

Where people get burned

Be honest with yourself before you spend an afternoon on this: as shipped, the ThinkSound node doesn't actually run. The generate method calls generate_audio(), which is a Python generator, and never iterates it - so the body never executes and the node silently does nothing in the graph. It's a port of a Gradio app where the wiring was dropped. Even if you fix that, the two scripts it shells out to, extract_latents.py and predict.py, are not in this repo - you have to pull them from the upstream FunAudioLLM/ThinkSound repo and drop them in the pack root.

Beyond the wiring, expect a real compute job. A 35-layer multimodal DiT doing 250 diffusion steps on ~9 seconds of stereo audio is minutes on a big GPU, not seconds. The upstream project targets a 44.1 kHz stereo latent diffusion run that will comfortably saturate a 16 GB card. If that sounds like a lot of effort for a node that's currently a stub - it is. Watch the repo; the whole thing is genuinely neat once it's actually wired up, and this is the only active ComfyUI path to the model for now.