Nodes/ComfyUI-Qwen2_5-Omni/QWEN2_5_OMNI_Run
ComfyUI Node

QWEN2_5_OMNI_Run

Ask your images, videos, and audio questions — this node is the actual assistant

By MakkiShizu·Created about a year ago·Updated about a year ago· 0
QWEN2_5_OMNI_Run
  • QWEN2_5_OMNI_model
  • image
  • video
  • audio
  • BatchImage
  • text
  • audio
text
video_decode_methodtorchvision
use_audio_in_videotrue
return_audiotrue
Voice_TypeChelsie
seed1

QWEN2_5_OMNI_Run is the node that makes the whole pack worth installing. Everything else in ComfyUI-Qwen2_5-Omni is setup; this is the payoff. You feed it the model from DownloadAndLoadQWEN2_5_OMNIModel, plus any combination of an image, a video, an audio clip, and a text prompt - and it answers in text, or talks back to you in a voice, at 24kHz audio. Same graph you use for SDXL is suddenly hosting a fully local multimodal assistant that understands what it sees and hears.

People use it for everything the cloud chat models do, just offline: describe a reference image in detail, caption a video including its dialogue, ask "what chord is this playing?" against an audio track, or batch-analyze a set of images for consistency before a LoRA run. Community reports on Qwen2.5-Omni back up the hype - someone ran it against a live guitar feed on a 3090 and it called out chords correctly in real time. It's genuinely good at hearing, not just seeing.

The inputs that matter

The text field (multiline) is your instruction. The QWEN2_5_OMNI_model input is required and must come from the loader node. Everything else is optional, and you can mix them: image and video and audio in one prompt, all described together. Four settings to know:

  • return_audio - true means you get speech back, and the Voice_Type dropdown (Chelsie, a warm female voice, or Ethan, bright and male) picks who says it. Set it false and you get text-only, faster. One catch: if the loader node was configured with use_audio_output off, this is forced to false anyway.
  • video_decode_method - torchvision (default), decord, or torchcodec. The default needs nothing extra. decord comes with the pack's requirements. torchcodec is a separate install and not worth it unless you're chasing video performance.
  • use_audio_in_video - if your video has a soundtrack and you want the model to hear it, keep this on. Turn it off to analyze just the frames.
  • seed - for reproducible outputs. Bump it when the model gets weirdly repetitive.

The outputs

Two: text (a STRING - wire it to a ShowText / preview node, or pipe it onward as a prompt) and audio (an AUDIO object, which any audio preview or save node will take). Under the hood it builds the Qwen chat template, hands your files to qwen_omni_utils for decoding, and calls generate() with your chosen speaker - which is why the audio only shows up when return_audio is on.

Installing and gotchas

Same install as the rest of the pack (Manager, or git clone + pip install -r requirements.txt, with transformers>=4.52.3 as a hard floor). The bigger "gotcha" is the whole-stack reality: this is a real 3B/7B model, not a toy. On modest cards it's slow and heavy - the 7B unquantized wants serious VRAM, and people realistically run the 3B or a quantized load (GGUF builds of Omni-7B exist and work). First runs also spend a while on model download/load before the node even wakes up.

The one thing to remember: if the loader disabled audio output, don't fight the node trying to get speech back. It won't. Match the two - audio off at load means text-only here, and that's fine, because text-only responses are a lot faster anyway.

CategoryQwen2_5-Omni

Inputs (11)

NameTypeDefaultDescription
textSTRING
QWEN2_5_OMNI_modelQWEN2_5_OMNI_MODEL
video_decode_methodCOMBOtorchvision3 options: torchvision, decord, torchcodec
use_audio_in_videoBOOLEANtrue
return_audioBOOLEANtrueThe model can batch inputs composed of mixed samples of various types such as text, images, audio and videos as input when return_audio=False is set.当设置 return_audio=False 时,该模型可以将由各种类型的混合样本(如文本、图像、音频和视频)组成的 Importing 进行批处理作为输入。In order to obtain a flexible experience, we recommend that users can decide whether to return audio when generate function is called. If return_audio is set to False, the model will only return text outputs to get text responses faster.为了获得灵活的体验,我们建议用户可以决定在调用 generate 函数时是否返回音频。如果 return_audio 设置为 False,则模型将仅返回文本输出以更快地获得文本响应。
Voice_TypeCOMBOChelsieChelsie:Female, A honeyed, velvety voice that carries a gentle warmth and luminous clarity.甜美、天鹅绒般的嗓音,带着温柔的温暖和明亮的清晰度。Ethan:Male, A bright, upbeat voice with infectious energy and a warm, approachable vibe.明亮、乐观的声音,具有感染力和温暖、平易近人的氛围。
seedINT11–18446744073709550000
imageoptIMAGE
videooptVIDEO
audiooptAUDIO
BatchImageoptBatchImage

Outputs (2)

NameTypeDescription
textSTRING
audioAUDIO