Replicate cjwbw/seamless_communication

This wraps Meta's SeamlessM4T - a single model that does speech and text translation across roughly a hundred languages, in any combination. Feed it audio, get translated audio or text. Feed it text, get translated speech or text. It also does plain transcription (ASR). It's genuinely impressive tech, and this node lets you call it from a ComfyUI graph through ComfyUI-Replicate.

It runs on Replicate's cloud, not locally. That's the point - you're not hosting a multilingual speech model on your own hardware.

How it works

SeamlessM4T is one unified model handling the whole matrix of speech↔text translation, rather than a pipeline of separate ASR, translation, and text-to-speech stages. You pick a task, hand it the right input (audio or text) and a target language, and it does the conversion end to end. The node ships that to Replicate and returns the result. No local compute, pay per run.

Inputs and outputs that matter

There are two outputs: STRING (the translated/transcribed text) and AUDIO (translated speech, for the speech-output tasks).

• task_name - the mode, and the first thing you set. Choices: S2ST (speech→speech), S2TT (speech→text), T2ST (text→speech), T2TT (text→text), and ASR (transcription). Everything else depends on which of these you pick.
• input_audio - an AUDIO input; required for the speech-input tasks (S2ST, S2TT, ASR).
• input_text - text input, for the text-input tasks (T2ST, T2TT).
• Target-language pickers - target_language_with_speech (36 languages that can produce spoken output) for tasks that return audio, and target_language_text_only (95 languages) for text-only targets. There's also input_text_language for when you're starting from text.

max_input_audio_length (60s default) caps how much audio it processes. force_rerun forces a fresh run on identical inputs.

The rule of thumb: match your input to your task. ASR/S2TT/S2ST want input_audio; T2TT/T2ST want input_text; use the speech-capable target list only for tasks that end in speech.

How to install it

ComfyUI Manager: search ComfyUI-Replicate, install, restart. Or:

cd ComfyUI/custom_nodes
git clone https://github.com/replicate/comfyui-replicate
cd comfyui-replicate
pip install -r requirements.txt

Restart, and set your token first:

export REPLICATE_API_TOKEN="r8_************"; python main.py

Token: replicate.com/account/api-tokens.

Common issues

Read this before you rely on the node: the pack's README says it targets models returning simple text or image, and explicitly warns that a model returning "audio, video, JSON objects or a combination of outputs" will not work as expected. SeamlessM4T returns both a string and audio - a combination - which is exactly the case the README flags. So the text output may come through while the audio side is unreliable, or the node may behave oddly on the speech-producing tasks. If you only need transcription or text→text translation (ASR, S2TT, T2TT - the string-only outputs), you're on the safest ground here.

The other usual things: a missing REPLICATE_API_TOKEN is the first-run killer, wrong task/input pairing (audio task with no audio wired in) will error, and every call costs money on Replicate.