Nodes/ComfyUI_FunCineForge/FunCineForge_SM_Predata
ComfyUI Node

FunCineForge_SM_Predata

The unglamorous middle node that makes dubbing work

By smthemex·Created 6 months ago·Updated 6 months ago· 21
FunCineForge_SM_Predata
  • videos
  • conds
  • conditioning
  • full_report
video_type独白
formatauto
codecauto
filename_prefixvideo/ComfyUI

If you're coming from a normal diffusion workflow, this is the node that will surprise you. Predata is the stage manager of the FunCineForge pack: it takes your video plus the script you built with Segments nodes and turns them into the JSONL data file the dubbing model actually consumes. It's also why this pack drags in a small army of speech models - this is the node that uses most of them.

What happens when you run it. Three things, in order:

  1. It re-saves your video into the ComfyUI output folder. The format, codec and filename_prefix inputs are the standard ComfyUI video-save controls (the tooltips on them say exactly that), so Predata doubles as the save step for your source clip.
  2. It runs the front-end analysis pipeline on that saved clip: speaker diarization via pyannote segmentation-3.0, voice-activity detection, speaker embeddings, and the visual side - face detection, active-speaker detection, lip features - all condensed into face-embedding files. This is the part that makes lip-synced dubbing possible: the model later uses these features to fit the new audio to the mouths on screen.
  3. It slices the clip at your segment timestamps and writes input_data.jsonl - one JSON record per utterance - plus the clipped video/audio pairs into a working directory.

The video_type combo (独白 monologue, 旁白 narration, 对话 dialogue, 多人 multi-speaker) tells the downstream model what kind of scene it's dubbing, which changes how it handles speaker turns.

The outputs. conditioning is a CONDITIONING that bundles the JSONL path, the parsed items, the video path and the work dir - wire it straight into FunCineForge_SM_KSampler. full_report is a STRING containing a preview of the generated JSONL, which the example workflow pipes into a PreviewAny node so you can eyeball the data before burning GPU time on inference. That's a habit worth copying: check the report, then run the sampler.

Setup. Same as the rest of the pack:

cd ComfyUI/custom_nodes
git clone https://github.com/smthemex/ComfyUI_FunCineForge
pip install -r requirements.txt

Restart, then get the models into ComfyUI/models/funcineforge. Predata is the node that actually needs the supporting models - the ModelScope speech models (FSMN VAD, the Paraformer ASR, CAMPPlus speaker verification, the punctuation model) and pyannote's segmentation-3.0, all of which the author bundles into one cloud-drive zip. If diarization throws a missing-model error, this is the stage to blame.

Where people get burned:

  • 25 fps, no exceptions. The README's lip-sync fix note is blunt: the input video must be 25 fps. Feed it a 24 or 30fps clip and the timing drifts. Re-encode first.
  • It's the slow stage. Face and speaker analysis on a full clip takes a while and hammers the CPU/GPU. It's normal for this to run far longer than the actual inference.
  • The numpy trap bites here too. pyannote.audio drags numpy to 2.x on install; the README tells you to downgrade it back below 2.0, or this node - and everything after it - misbehaves.
  • Don't skip it. The conditioning output carries precomputed face features; there's no shortcut where you feed the KSampler a raw video instead.
CategoryFunCineForge

Inputs (6)

NameTypeDefaultDescription
videosVIDEO
condsCONDITIONING
video_typeCOMBO独白4 options: 独白, 旁白, 对话, 多人
formatCOMBOautoThe format to save the video as.
codecCOMBOautoThe codec to use for the video.
filename_prefixSTRINGvideo/ComfyUIThe prefix for the file to save. This may include formatting information such as %date:yyyy-MM-dd% or %Empty Latent Image.width% to include values from nodes.

Outputs (2)

NameTypeDescription
conditioningCONDITIONING
full_reportSTRING