HunyuanFoley Text Encode

HunyuanFoley Text Encode is where you tell the foley model what a scene should sound like. It's the audio-world cousin of the CLIP text encoder you're used to from image generation, and getting the mental model right matters more than any setting on the node: the prompt "a person walks on frozen ice" is a sound description, not a visual one. You're not describing what the frame looks like, you're describing the acoustic events that should accompany it - ice crunching, footsteps, maybe a distant wind.

The node is part of the Vantage-HunyuanFoley pack, which splits Tencent's HunyuanVideo-Foley into modular pieces so you don't hold every encoder in VRAM at once. This one loads a CLAP model - laion/larger_clap_general, contrastive language-audio pretraining - and projects your text into the same embedding space the audio model conditions on. It's a separate text encoder, not the one in your checkpoint, and it's a single file tree you have to fetch yourself.

How it works

Under the hood it's straightforward: load a tokenizer and ClapTextModelWithProjection from the clap/ folder with local_files_only=True, encode [negative_prompt, prompt] in that order, then truncate both to the model config's text_length of 77 tokens. The two outputs are positive (TEXT_FEAT) and negative (UNCOND_TEXT_FEAT), which feed the Denoiser's positive and negative inputs. The negative gets the same weight as everything else in classifier-free guidance, so "noisy, harsh" (the default) is doing real work keeping the result clean.

The inputs that matter:

• prompt (multiline string) - the default is "A person walks on frozen ice". Keep it short and concrete; anything past 77 tokens is silently dropped.
• negative_prompt (multiline string) - default "noisy, harsh".
• to_gpu / move_back_to_cpu (bools, both default on) - standard offload toggles so the ~200MB-ish CLAP encoder doesn't sit on your card between runs.

Installing it

Same pack install as the rest: ComfyUI Manager → search Vantage-HunyuanFoley → install → restart, or

cd ComfyUI/custom_nodes
git clone https://github.com/vantagewithai/Vantage-HunyuanFoley.git
cd Vantage-HunyuanFoley && pip install -r requirements.txt

Then the model. Create ComfyUI/models/hunyuan_foley/clap/ and download model.safetensors, config.json, merges.txt and vocab.json from laion/larger_clap_general on Hugging Face - the README points at the refs/pr/2 branch, so grab them from there. If any of the four files is missing or the folder path is off, the node errors out before generating anything.

What to watch for

Prompting is where people get confused, and it shows in the output. This is not an image-tag model - "cinematic, octane render, 4k" does nothing for it. You want the language of sound design: "footsteps on gravel, birdsong in the distance", "thunder and heavy rain on a tin roof". When someone complains the result was "a weird christmas-like music" instead of the scene's sound effects, it's almost always that the conditioning didn't reach the model - the video frames weren't hooked up in Visual Encode, or the prompt was describing the image rather than the audio. Keep prompts short (77-token truncation is unforgiving), keep the negative prompt on, and leave move_back_to_cpu enabled on smaller cards.