Nodes/ComfyUI-HunyuanVideo-Foley/Hunyuan-Foley Sampler
ComfyUI Node

Hunyuan-Foley Sampler

The node that turns a silent clip into footsteps, wind, and rain

By phazei·Created 12 months ago·Updated 3 months ago· 113
Hunyuan-Foley Sampler
  • hunyuan_model
  • hunyuan_deps
  • image
  • torch_compile_cfg
  • block_swap_args
  • audio_first
  • audio_batch
frame_rate16.0
duration5.0
promptA person walks on frozen ice
negative_promptnoisy, harsh
cfg_scale4.5
steps50
samplereuler
batch_size1
seed0
force_offloadtrue

This is the whole point of the pack. Hunyuan-Foley Sampler is where a silent clip - or just a text prompt - becomes audio that actually matches the pixels. It's a flow-matching diffusion model under the hood, the same family as MMAudio, but built by Tencent specifically for high-fidelity foley, and it does something MMAudio-style tools rarely advertise: it works with no video at all. Leave the image input empty and you've got a text-to-audio machine with a negative prompt. That's a genuinely nice trick, and it means you can test your prompt before you've even rendered a frame.

Feed it properly and here's the pipeline, roughly: SigLIP2 reads the video's content at 8fps, Synchformer watches for sync at 25fps, CLAP encodes your prompt, then the transformer denoises toward a DAC-VAE latent that decodes into a real waveform. The node runs it in two phases, ping-ponging the extractor models and the main model so nothing sits in VRAM longer than it has to - that's the "ping-pong offloading" the README credits with saving ~1–2GB.

The inputs that matter, in the order you'll touch them:

  • image (optional) - connect an image sequence (frames) for video-to-audio; leave empty for text-to-audio. If your clip is shorter than duration × frame_rate, the pack just holds the last frame rather than failing.
  • frame_rate and duration - the fps of your input and how many seconds of audio to make. The defaults (16fps, 5s) are sane.
  • prompt / negative_prompt - defaults are "A person walks on frozen ice" and "noisy, harsh". They matter more than you'd expect; foley quality lives and dies on the prompt here.
  • cfg_scale (4.5), steps (50) - the README's stock recipe is Euler at CFG ≈4.5, steps ≈50. Trust it. On a 3090, a 10s clip is roughly 20 seconds of generation before block swap.
  • sampler - here's the author's own honest tooltip: the dropdown was inherited from the official repo, "but only Euler seems decent..." So: leave it on Euler.
  • batch_size (1–6) - generates several variations at once. VRAM scales with it, so this is your first OOM lever.
  • force_offload (default on) - shoves models back to CPU after generation to keep VRAM free for whatever's next in the graph. Leave it on.

The two optional inputs are where the pack's headline features plug in: torch_compile_cfg (from Hunyuan-Foley Torch Compile) and block_swap_args (from Hunyuan-Foley BlockSwap Settings). Both are applied lazily, right before denoising, so you can switch them on and off without touching this node.

Outputs: audio_first - an AUDIO object of just the first clip, ready to plug straight into a Preview Audio node - and audio_batch - the full batch, meant for the pack's Select Audio From Batch node so you can audition variations. Wire audio_first into VHS_VideoCombine or SaveAudio and you're done.

Troubleshooting

The common failure is OOM, and the fix order is boring and correct: drop batch_size to 1, cut steps, make sure force_offload is on. If you're on an 8GB card, load the fp8 model and set quantization to match - the sampler itself is well-behaved once the model fits. Text-to-audio runs are dramatically lighter than video-to-audio because there are no extractor passes over pixels, so if you're testing settings, that's the cheap way to do it.

First run needs internet (SigLIP2/CLAP download), and the whole thing wants PyTorch 2.7+. The pack's example workflow - VHS Load Video → Model Loader + Dependencies Loader → this → Select Audio From Batch → Preview Audio → VHS Video Combine - is in example_workflows/ and is the fastest way to a working graph.

Categoryaudio/HunyuanFoley

Inputs (15)

NameTypeDefaultDescription
hunyuan_modelHUNYUAN_MODEL
hunyuan_depsHUNYUAN_DEPS
frame_rateFLOAT16.01–120The framerate of the input image sequence
durationFLOAT5.01–60Duration of the audio to generate in seconds
promptSTRINGA person walks on frozen ice
negative_promptSTRINGnoisy, harsh
cfg_scaleFLOAT4.51–10Classifier-Free Guidance scale
stepsINT5010–100Number of denoising steps
samplerCOMBOeulerThese were included with the official repo, but only Euler seems decent...
batch_sizeINT11–6Number of audio variations to generate at once
seedINT00–18446744073709550000
force_offloadBOOLEANtrueOffload models from VRAM after generation
imageoptIMAGE
torch_compile_cfgoptTORCH_COMPILE_CFGCompile the model blocks with this configuration (applied lazily before denoising).
block_swap_argsoptBLOCKSWAPARGSEnable BlockSwap VRAM optimization during sampling.

Outputs (2)

NameTypeDescription
audio_firstAUDIO
audio_batchAUDIO