Nodes/ControlFoley Official/ControlFoley Video-Audio Muxer
ComfyUI Node

ControlFoley Video-Audio Muxer

Put the foley back on the picture

By YJX-Research·Created 2 months ago·Updated 2 months ago· 6
ControlFoley Video-Audio Muxer
  • controlfoley_model
  • video
  • audio
  • video_file
  • video_path
output_filenamecontrolfoley/output.mp4
modereplace

Generating foley is only half the job - you want it on the video, in one file you can actually watch and ship. ControlFoley Video-Audio Muxer is that last step: it takes the generated audio, stamps it onto the original video, and writes an MP4 where your foley replaces the clip's original audio track.

The one mode it currently has

  • mode - replace (default) is the only implemented option. mix_planned is in the enum, and it's a lie you should know about: the code raises NotImplementedError if you select it. Mix mode (generated audio blended with the original track) is planned, not shipped. Use replace.

Everything else is straightforward:

  • controlfoley_model - yes, the muxer takes the model object, not just audio and video. It needs the runtime because it re-runs the video through the pack's load pipeline (the same cached feature path the generator uses) to produce the frames for the output. So the model has to be loaded when this runs - keep the loader in the chain.
  • video - the CONTROLFOLEY_VIDEO from ControlFoley Video Loader.
  • audio - the AUDIO output. If it comes from Save ControlFoley Audio, the muxer reuses that node's filename stem so the .wav and .mp4 land side by side with the same numeric suffix. Otherwise it generates a filename from output_filename (default controlfoley/output.mp4).
  • output_filename - base name for the muxed file when there's no stem to inherit.

Outputs: video_file (CONTROLFOLEY_VIDEO_FILE) and video_path (STRING, the written path). It's an output node, so you get an inline video preview too.

The plumbing it does quietly

Two things in here are worth knowing because they're the difference between a file that "works" and one that plays correctly. First, if the audio is multichannel it gets downmixed to mono for the mux, and the output duration is clamped to the shorter of video and audio so you don't get a dead tail. Second, there's a container-timing fix: the upstream muxer pins frame PTS values but never sets packet durations, so ffmpeg guesses ~1/24s for the final frame and the MP4 reports a longer duration than the WAV from the same waveform. The node rewrites the muxed file's packet durations so the container matches reality. That's the kind of jank you'd otherwise chase for an hour wondering why your MP4 says 12 seconds when your WAV says 9.

Gotchas

The output lives under ComfyUI/output/controlfoley (or wherever the save-stem points). Keep both the Save Audio and Muxer nodes enabled in video workflows - the pack's templates do exactly that so you get the standalone WAV and the paired MP4. And remember the whole pipeline is CUDA-only, and generated output inherits the weights' CC BY-NC 4.0 non-commercial terms. Install the pack via ComfyUI Manager ("ControlFoley Official") or git clone into custom_nodes + pip install -r requirements.txt.

CategoryControlFoley

Inputs (5)

NameTypeDefaultDescription
controlfoley_modelCONTROLFOLEY_MODEL
videoCONTROLFOLEY_VIDEO
audioAUDIO
output_filenameSTRINGcontrolfoley/output.mp4
modeCOMBOreplace2 options: replace, mix_planned

Outputs (2)

NameTypeDescription
video_fileCONTROLFOLEY_VIDEO_FILE
video_pathSTRING