Gemini Diarisation (Vertex AI)

Drop a podcast or interview recording in, get four separate audio tracks back, one per speaker. That's the pitch, and it's the rare node that delivers it with almost no setup - because the heavy lifting happens on Google's side. It's the audio equivalent of this pack's whole philosophy: no local models, no VRAM, just a Vertex AI call.

What it actually does (the honest version)

This is speaker diarization: figuring out who spoke when, and splitting the audio accordingly. What it is not is audio source separation - the node doesn't isolate each voice like a stem-splitting tool. Instead, Gemini transcribes the audio, timestamps every utterance, and assigns each one to a speaker. The node then cuts the original recording at those timestamps and reassembles each speaker's segments into their own track. Overlapping speech gets messy, but for a clean two-person interview it's remarkably effective. num_speakers (1–10) tells Gemini how many distinct voices to look for, which is the single most important input here - get it wrong and the whole split is off.

How it works

The node converts your AUDIO input to a mono WAV, then sends Gemini a carefully worded prompt that demands a strict JSON list of utterances with start_timestamp, end_timestamp, and speaker labels. It parses that JSON, sorts speakers by when they first speak, and builds up to four output tracks by copying the relevant slices of the original audio. The audio_timestamp toggle (optional, default off) asks the API for finer timestamp precision, at the cost of some speed. There's also a thinking toggle and budget for when the timestamping gets tricky.

The inputs that matter

The credential trio again: project_id, location, service_account - and the pack-wide reminder that service_account wants the JSON contents pasted in, not a file path.

• audio - an AUDIO input, so it pairs naturally with this pack's TTS node or any audio-loading node.
• num_speakers - the count Gemini must find. Set it before you run; it's not a suggestion.
• model - defaults to gemini-2.5-flash; the dropdown mirrors the chat node. Flash is fine for speech.
• seed and temperature - the usual dials; low temperature keeps the transcription consistent.

Outputs

speaker_1 through speaker_4 - four AUDIO outputs. If your recording has two speakers, speaker_3 and speaker_4 come back silent, which can look like a bug the first time you see it. Wire each to an audio preview, a save node, or further processing.

Installing

cd ComfyUI/custom_nodes/
git clone https://github.com/Aryan185/ComfyUI-VertexAPI.git
cd ComfyUI-VertexAPI
pip install -r requirements.txt

Restart ComfyUI, or install ComfyUI-VertexAPI via ComfyUI Manager.

Where people get burned

The censor-with-asterisks behavior is baked into the prompt - if your audio has strong language, the transcription censors it, so don't expect verbatim captions on salty content. Timestamp drift is the other failure: if Gemini mis-times an utterance, a chunk of the wrong person's speech lands on the wrong track. And one more quirk worth knowing: this node leans on Gemini's transcription, so heavy accents or crosstalk degrade it the same way they'd degrade any ASR. For a clean two-person recording it's a one-shot win; for a chaotic five-way call, set expectations low and check the output before you trust it.