Nodes/ComfyUI_FunCineForge/FunCineForge_SM_KSampler
ComfyUI Node

FunCineForge_SM_KSampler

The 'KSampler' that isn't one — this is where the dubbing happens

By smthemex·Created 6 months ago·Updated 6 months ago· 21
FunCineForge_SM_KSampler
  • model
  • conditioning
  • audio
infer_dir/video/ComfyUI_00014_
seed0

The name is a lie. There's no diffusion loop, no denoising schedule, no CFG scale hiding in here. FunCineForge_SM_KSampler is the inference node for the FunCineForge dubbing model, called a "sampler" only because that's the closest familiar slot in a ComfyUI graph. What it actually does is take the loaded model plus the prepared data, synthesize the dubbed speech, and then quietly mux it back onto the video for you.

Inputs. Three that matter:

  • model - straight from FunCineForge_SM_Model. It carries the LLM, flow-matching model and vocoder you selected there.
  • conditioning - from FunCineForge_SM_Predata. This is the JSONL plus the precomputed face/lip features that make the new speech land on the right mouths at the right times. Feed it; don't make the node fall back.
  • seed - your reproducibility dial. Same model, same conditioning, same seed → same audio.

There's also infer_dir, and it's a trap-shaped default. If you don't provide conditioning, the node looks for an input_data.jsonl and a video named after the folder inside that directory - and the default value (/video/ComfyUI_00014_) is clearly from the author's own machine. In practice you always wire conditioning from Predata and ignore this input.

The output. audio - the dubbed speech as a waveform at a 24kHz sample rate. The example workflow wires it two ways: into a SaveAudio node to keep just the voice, and into a CreateVideo node to mux it back onto the original clip, then SaveVideo. Worth copying that second path, because the KSampler also writes a dubbed_video.mp4 into the work folder behind your back - check your output directory even if you only saved the audio.

Behind the curtain. It runs the three-stage chain - the Qwen2-0.5B LLM turns your text plus clue into tokens, the flow-matching model shapes them, the vocoder renders the waveform - then stitches the per-clip wavs onto the video timeline at your segment start times.

Setup. Identical to the rest of the pack:

cd ComfyUI/custom_nodes
git clone https://github.com/smthemex/ComfyUI_FunCineForge
pip install -r requirements.txt

Restart, load the ~13GB of models into ComfyUI/models/funcineforge, and remember the shared gotchas: numpy needs to stay below 2.0 after pyannote.audio installs, and the source video has to be 25 fps.

Where people get burned:

  • First run is slow. Loading a 13GB stack plus inference is not a snappy experience. Subsequent runs in the same session are better but still not instant.
  • Empty combos upstream kill it. If the Model node's dropdowns were empty, nothing ever got loaded and this node fails immediately. Models first, always.
  • Alignment issues point upstream, not here. If the speech drifts from the mouths, it's almost always the 25 fps rule being violated in Predata. The pack's update notes specifically fixed audio-lip misalignment and two-speaker accent drift by splitting segments per speaker - so keep your Segments grouped by spk and your source video at 25 fps.
CategoryFunCineForge

Inputs (4)

NameTypeDefaultDescription
modelMODEL
infer_dirSTRING/video/ComfyUI_00014_
seedINT00–2147483647
conditioningoptCONDITIONING

Outputs (1)

NameTypeDescription
audioAUDIO