Nodes/ControlFoley Official/ControlFoley Simple Generate
ComfyUI Node

ControlFoley Simple Generate

ControlFoley Simple Generate

By YJX-Research·Created 3 months ago·Updated 2 months ago· 8
ControlFoley Simple Generate
  • video
  • video_input
  • images
  • audio
  • sample_rate
  • inference_time_sec
  • peak_vram_gb
  • status
◄controlfoley_source_dircontrolfoley►
◄model_weights_dirpath/to/model_weights►
◄variant▾►
◄deviceauto►
◄precisionbf16►
◄low_vramfalse►
◄compile_encodersfalse►
◄promptA bird sings melodically in a forest►
◄negative_prompt►
◄duration10.0►
◄seed42►
◄num_inference_stepsfixed►
◄guidance_scale4.5►
◄mask_away_clipfalse►
◄cache_video_featurestrue►
◄staged_offloadtrue►
◄clip_batch_size_multiplier40►
◄sync_batch_size_multiplier40►
◄reference_audio_path►
◄image_fps24►
◄enabledtrue►
◄silent_audio_on_errorfalse►
◄auto_fetch_sourcetrue►

ControlFoley Simple Generate is the "just let me hear something" node. Every other path in this pack makes you assemble a chain - Dependencies Loader into Model Loader into a Generate node. This one swallows all of that: it loads the model and generates audio in a single node, so a complete text-to-foley run is literally one node on the canvas.

That's a big deal for your first run, because it's also the cheapest way to smoke-test the whole pack. The default prompt is "A bird sings melodically in a forest" and the default duration is 10 seconds - pull the node in, hit run, and you'll either get a WAV or a clear error. Given the pack's footprint (~16GB of weights plus Hugging Face dependencies on first load), having a one-node test before you build anything elaborate is exactly what you want.

What it packs in

Because it fuses loader and generator, the input list is long. The loader half is on top: controlfoley_source_dir, model_weights_dir, variant (only large_44k exists), device (auto/cuda), precision (bf16/fp16/fp32), low_vram, compile_encoders, and auto_fetch_source. The generation half is everything ControlFoley Generate offers: prompt, negative prompt, duration, seed, num_inference_steps (fixed or 1–100), guidance_scale, reference_audio_path, the two batch-size multipliers, plus enabled and silent_audio_on_error from the advanced node.

The few you'll actually touch:

  • precision - bf16 default; fp16 if your card doesn't do bf16 well, fp32 if you're chasing reference quality and have VRAM to burn.
  • low_vram - leave this false for anything with a video input. The node hard-errors if you combine low_vram=true with a video or reference audio; it's a text-only T2A/TTA path.
  • auto_fetch_source - default on: if it can't find the ControlFoley source tree locally, it shallow-clones the pinned upstream revision into <ComfyUI root>/controlfoley over git. Offline or behind a firewall? Set CONTROLFOLEY_SOURCE_URL to a mirror, or clone xiaomi-research/controlfoley yourself.

Modes it still supports

Despite the name, this isn't text-only. The optional inputs video (CONTROLFOLEY_VIDEO), video_input (native VIDEO), and images (IMAGE + image_fps) all work here too, so you can do V2A, TC-V2A, and AC-V2A from a single node. The reason it's called "simple" is the graph, not the feature set.

Outputs mirror the advanced node: audio (AUDIO), sample_rate (INT), inference_time_sec, peak_vram_gb (FLOATs), and status (STRING). audio feeds Save ControlFoley Audio or the muxer.

The tradeoff, stated plainly

Every generation here re-runs the load path, so this node is the wrong choice once you're past experimenting - it throws away the whole point of the Model Loader's cache, which keeps the runtime in memory across runs. For a one-off test or a quick sanity check it's perfect; for a workflow you run repeatedly, spend the two extra minutes building the loader chain and let Simple Generate retire gracefully. Install the pack through ComfyUI Manager ("ControlFoley Official") or git clone into custom_nodes + pip install -r requirements.txt, and be aware the whole thing is CUDA-only - no CPU, no MPS.

CategoryControlFoley

Inputs (26)

NameTypeDefaultDescription
controlfoley_source_dirSTRINGcontrolfoley—
model_weights_dirSTRINGpath/to/model_weights—
variantCOMBO1 options: large_44k
deviceCOMBOauto2 options: auto, cuda
precisionCOMBObf163 options: bf16, fp16, fp32
low_vramBOOLEANfalse—
compile_encodersBOOLEANfalse—
promptSTRINGA bird sings melodically in a forest—
negative_promptSTRING—
durationFLOAT10.00.7–30Text-only generation uses 10s by default. Video generation follows input length up to 30s.
seedINT420–4294967295—
num_inference_stepsSTRINGfixedUse 'fixed' to keep the default step setting, or enter an integer from 1 to 100.
guidance_scaleFLOAT4.50–20—
mask_away_clipBOOLEANfalse—
cache_video_featuresBOOLEANtrue—
staged_offloadBOOLEANtrueMove encoders to CPU during sampling when the ControlFoley source supports it. The public upstream source does not implement this; the option is then ignored and a console note is printed.
clip_batch_size_multiplierSTRING40Integer 1-80. Use 4-8 on low-VRAM GPUs.
sync_batch_size_multiplierSTRING40Integer 1-80. Use 4-8 on low-VRAM GPUs.
reference_audio_pathSTRING—
image_fpsFLOAT241–120—
enabledBOOLEANtrue—
silent_audio_on_errorBOOLEANfalse—
auto_fetch_sourceBOOLEANtrueWhen the public ControlFoley source tree is not found locally, run 'git clone' (pinned revision) from GitHub into <ComfyUI root>/controlfoley. Set the CONTROLFOLEY_SOURCE_URL environment variable to use a mirror.
videooptCONTROLFOLEY_VIDEO—
video_inputoptVIDEO—
imagesoptIMAGE—

Outputs (5)

NameTypeDescription
audioAUDIO—
sample_rateINT—
inference_time_secFLOAT—
peak_vram_gbFLOAT—
statusSTRING—