Comfyui-Sequential-VideoLoading

ComfyUI Sequential Video Loading

LoadVideoSequentially is a custom ComfyUI node that:

• Loads a video by time segments (turn-based)
• Extracts all frames in the selected segment (unless select_every_nth > 1)
• Extracts matching audio for the same segment
• Runs speaker diarization + transcription
• Maps detected speakers to your configured position labels (A/B/C/D)
• Optionally translates dialogue lines with an LLM (Qwen/Gemma-compatible Transformers folders)

What The Node Outputs

• images: image batch for the selected time segment
• audio: audio for the selected time segment
• turn_used, start_second, end_second, frame_rate
• speaker position passthrough strings
• dialogue_text: formatted transcript lines, e.g.
  • 0:03 Center left character says: ...

Time Segment Logic

For each run:

• start_second = (turn - 1) * duration_seconds
• end_second = start_second + duration_seconds

Example:

• duration_seconds = 20
• turn = 2
• Node loads second 20 to 40

Inputs (Important)

• video: source video from ComfyUI input folder
• duration_seconds: segment length per turn
• turn: segment index (1-based)
• select_every_nth: frame skipping (use 1 for all frames)
• hf_token: Hugging Face token for pyannote diarization
• whisper_model: Whisper model for transcription
• merge_consecutive_speaker: merges adjacent lines from same speaker
• continue_without_hf_token: allow fallback transcription without diarization
• translation_language: target language (none disables translation)
• llm_model: model folder from ComfyUI/models/LLM (must include config.json)
• speaker_a_position ... speaker_d_position: labels used in output text

HF Token Prompt Behavior

If hf_token is empty, a popup appears:

HF Token is not present see node readme for instructions. Continue YES//NO

• NO: queue is canceled
• YES: node proceeds without diarization (transcription fallback)

How To Create A Hugging Face Token (for pyannote)

1. Create/login account at https://huggingface.co
2. Open token settings: https://huggingface.co/settings/tokens
3. Create a new token (Read access is enough)
4. Accept model access terms (once) for:
   • pyannote/speaker-diarization-3.1
5. Copy token (starts with hf_...)
6. Paste into node input hf_token

If the token is missing/invalid, diarization may fail and fallback transcription will be used (if allowed).

LLM Translation Notes

• Translation is done line by line.
• Only utterance text is sent to the model.
• Timeline and speaker label prefix are preserved exactly.

Input line:

0:03 Center left character says: Bonjour tout le monde

Translated output:

0:03 Center left character says: Hello everyone

Installation

1. Copy this folder to:
   • ComfyUI/custom_nodes/Comfyui__Load_Video_Sequentially
2. Restart ComfyUI
3. Hard refresh browser (Ctrl+F5) to load JS extensions

Requirements

Python packages used by this node include:

• opencv-python
• numpy
• torch
• pyannote.audio
• openai-whisper
• transformers

Install missing ones in your ComfyUI Python environment.

Troubleshooting

• Too few frames / accelerated output:
  • Set select_every_nth = 1
• Single speaker for multi-speaker clip:
  • Check valid HF token and pyannote model access
• Translation skipped:
  • Ensure selected llm_model folder is a valid Transformers model directory (config.json present)