Qwen Omni Combined🐼

If you've ever wanted ComfyUI to look at something and then tell you about it - not with a caption, with an actual voice - this is the node. Qwen Omni Combined is the centerpiece of the ComfyUI-Qwen-Omni pack: a single node that runs Alibaba's Qwen2.5-Omni multimodal model locally and returns both a text answer and (optionally) a spoken one. You feed it text, an image, an audio clip, a video path, or any mix; it reads everything at once and replies. No API, no key, no cloud. The README's "end-to-end multimodal interaction" is marketing fluff for something genuinely useful: one node in your graph that actually understands what it's looking at.

Who's it for? The obvious cases: captioning or answering questions about a reference image, summarizing a voice note, or asking "what's in this video?" and wiring the answer into the rest of your workflow. People on r/comfyui actually point to this pack when someone asks for a voice-to-text prompt node or a way to transcribe audio tracks locally. If you want an audio clip turned into a text description you can feed onward, this is about the shortest path that exists in ComfyUI right now.

How it works

This runs the real Qwen2.5-Omni model (Qwen2_5OmniForConditionalGeneration) loaded from ComfyUI/models/Qwen/ on your machine. It builds a chat-style conversation from whatever you connected, applies the model's chat template, and calls generate() - the same transformer call a script would make, wrapped in a node. The "Omni" trick is that text, image, audio, and video all get tokenized together, so the model reasons across modalities instead of you chaining separate captioner nodes.

A few details worth knowing before you expect too much:

• Videos are decimated hard: down to at most 15 frames, each resized to 384×384. Good for "what's happening here," not frame-accurate analysis.
• Images get thumbnailed to a 1024px max before being fed in.
• Speech comes back at 24 kHz; you pick Chelsie (female, warm) or Ethan (male, calm).

The inputs that matter

The defaults are sane enough to just hit Queue, but here's what you'll actually touch:

• model_name - Qwen2.5-Omni-3B (default) or -7B. Start with 3B; reach for 7B only if you want better answers and have the VRAM.
• quantization - 4-bit (default, VRAM-friendly), 8-bit, or none (original fp16 precision). Unquantized 7B will eat 14GB+ of VRAM before you add any context.
• prompt - your actual question, plain language, Chinese works too. This is what you're asking the image/audio/video about.
• audio_output - None, or one of the two voices. Turn it on and the audio output starts producing.
• max_tokens, temperature, top_p, repetition_penalty - standard generation knobs, and the tooltips are unusually good. The author's guidance: temperature 0.1–0.3 for structured/technical answers.

The optional inputs are where it gets fun: image (PNG/JPG), audio (MP3/WAV), and video_path (MP4/WEBM - wire in this pack's Video Uploader node). audio_source decides whether to use a video's built-in track or a separate audio file. Outputs are text (STRING) and audio (AUDIO) - text wires into anything that takes a string, audio goes to a save or playback node.

Installation

cd ComfyUI/custom_nodes/
git clone https://github.com/SXQBW/ComfyUI-Qwen-Omni.git
cd ComfyUI-Qwen-Omni
pip install -r requirements.txt

Or search "ComfyUI-Qwen-Omni" in ComfyUI Manager and let it handle the clone. Restart either way. On first run the model auto-downloads to ComfyUI/models/Qwen/ - the node even speed-tests Hugging Face against ModelScope and grabs whichever is faster. That's a multi-GB download for the 3B alone.

Where people get burned

The requirements file is the trap. It pins a preview build of transformers - git+https://github.com/huggingface/[email protected] - plus bitsandbytes, modelscope, and a Windows-targeted triton-windows line. Custom nodes share one Python environment with zero isolation, so that pinned transformers can silently break other nodes expecting a different version, and triton-windows is a known pip failure on Linux. If an install or load fails, that requirements file is the first thing to suspect.

Other classics: it hard-requires CUDA (raises a RuntimeError if it can't see a GPU), and if the auto-download flakes out, the error tells you exactly where to drop the model manually: ComfyUI/models/Qwen/Qwen2.5-Omni-3B/. The model also stays resident in VRAM after the node runs - not something to leave floating in every workflow if your card is small.