Nodes/ControlFoley Official/ControlFoley Simple Generate
ComfyUI Node

ControlFoley Simple Generate

ControlFoley Simple Generate

By YJX-Research·Created 2 months ago·Updated 2 months ago· 6
ControlFoley Simple Generate
  • video
  • video_input
  • images
  • audio
  • sample_rate
  • inference_time_sec
  • peak_vram_gb
  • status
controlfoley_source_dircontrolfoley
model_weights_dirpath/to/model_weights
variant
deviceauto
precisionbf16
low_vramfalse
compile_encodersfalse
promptA bird sings melodically in a forest
negative_prompt
duration10.0
seed42
num_inference_stepsfixed
guidance_scale4.5
mask_away_clipfalse
cache_video_featurestrue
staged_offloadtrue
clip_batch_size_multiplier40
sync_batch_size_multiplier40
reference_audio_path
image_fps24
enabledtrue
silent_audio_on_errorfalse
auto_fetch_sourcetrue

ControlFoley Simple Generate is the "just let me hear something" node. Every other path in this pack makes you assemble a chain - Dependencies Loader into Model Loader into a Generate node. This one swallows all of that: it loads the model and generates audio in a single node, so a complete text-to-foley run is literally one node on the canvas.

That's a big deal for your first run, because it's also the cheapest way to smoke-test the whole pack. The default prompt is "A bird sings melodically in a forest" and the default duration is 10 seconds - pull the node in, hit run, and you'll either get a WAV or a clear error. Given the pack's footprint (~16GB of weights plus Hugging Face dependencies on first load), having a one-node test before you build anything elaborate is exactly what you want.

What it packs in

Because it fuses loader and generator, the input list is long. The loader half is on top: controlfoley_source_dir, model_weights_dir, variant (only large_44k exists), device (auto/cuda), precision (bf16/fp16/fp32), low_vram, compile_encoders, and auto_fetch_source. The generation half is everything ControlFoley Generate offers: prompt, negative prompt, duration, seed, num_inference_steps (fixed or 1–100), guidance_scale, reference_audio_path, the two batch-size multipliers, plus enabled and silent_audio_on_error from the advanced node.

The few you'll actually touch:

  • precision - bf16 default; fp16 if your card doesn't do bf16 well, fp32 if you're chasing reference quality and have VRAM to burn.
  • low_vram - leave this false for anything with a video input. The node hard-errors if you combine low_vram=true with a video or reference audio; it's a text-only T2A/TTA path.
  • auto_fetch_source - default on: if it can't find the ControlFoley source tree locally, it shallow-clones the pinned upstream revision into <ComfyUI root>/controlfoley over git. Offline or behind a firewall? Set CONTROLFOLEY_SOURCE_URL to a mirror, or clone xiaomi-research/controlfoley yourself.

Modes it still supports

Despite the name, this isn't text-only. The optional inputs video (CONTROLFOLEY_VIDEO), video_input (native VIDEO), and images (IMAGE + image_fps) all work here too, so you can do V2A, TC-V2A, and AC-V2A from a single node. The reason it's called "simple" is the graph, not the feature set.

Outputs mirror the advanced node: audio (AUDIO), sample_rate (INT), inference_time_sec, peak_vram_gb (FLOATs), and status (STRING). audio feeds Save ControlFoley Audio or the muxer.

The tradeoff, stated plainly

Every generation here re-runs the load path, so this node is the wrong choice once you're past experimenting - it throws away the whole point of the Model Loader's cache, which keeps the runtime in memory across runs. For a one-off test or a quick sanity check it's perfect; for a workflow you run repeatedly, spend the two extra minutes building the loader chain and let Simple Generate retire gracefully. Install the pack through ComfyUI Manager ("ControlFoley Official") or git clone into custom_nodes + pip install -r requirements.txt, and be aware the whole thing is CUDA-only - no CPU, no MPS.

CategoryControlFoley

Inputs (26)

NameTypeDefaultDescription
controlfoley_source_dirSTRINGcontrolfoley
model_weights_dirSTRINGpath/to/model_weights
variantCOMBO1 options: large_44k
deviceCOMBOauto2 options: auto, cuda
precisionCOMBObf163 options: bf16, fp16, fp32
low_vramBOOLEANfalse
compile_encodersBOOLEANfalse
promptSTRINGA bird sings melodically in a forest
negative_promptSTRING
durationFLOAT10.00.7–30Text-only generation uses 10s by default. Video generation follows input length up to 30s.
seedINT420–4294967295
num_inference_stepsSTRINGfixedUse 'fixed' to keep the default step setting, or enter an integer from 1 to 100.
guidance_scaleFLOAT4.50–20
mask_away_clipBOOLEANfalse
cache_video_featuresBOOLEANtrue
staged_offloadBOOLEANtrueMove encoders to CPU during sampling when the ControlFoley source supports it. The public upstream source does not implement this; the option is then ignored and a console note is printed.
clip_batch_size_multiplierSTRING40Integer 1-80. Use 4-8 on low-VRAM GPUs.
sync_batch_size_multiplierSTRING40Integer 1-80. Use 4-8 on low-VRAM GPUs.
reference_audio_pathSTRING
image_fpsFLOAT241–120
enabledBOOLEANtrue
silent_audio_on_errorBOOLEANfalse
auto_fetch_sourceBOOLEANtrueWhen the public ControlFoley source tree is not found locally, run 'git clone' (pinned revision) from GitHub into <ComfyUI root>/controlfoley. Set the CONTROLFOLEY_SOURCE_URL environment variable to use a mirror.
videooptCONTROLFOLEY_VIDEO
video_inputoptVIDEO
imagesoptIMAGE

Outputs (5)

NameTypeDescription
audioAUDIO
sample_rateINT
inference_time_secFLOAT
peak_vram_gbFLOAT
statusSTRING