Qwen2 Audio Q&A

Hand an AI a clip of audio and ask it what it hears. That's the whole pitch: this node runs Alibaba's Qwen2-Audio-7B-Instruct, a genuinely multimodal LLM that takes audio and text as input and answers in text - "what instrument is playing at 0:23?", "transcribe the speech", "does this sound like a room recording or a treated studio?" If you've seen Qwen's image models (Qwen-Image and friends, the same Tongyi/Qwen family), this is the audio sibling, and the ComfyUI community hasn't exactly fallen over itself shipping audio-aware LLMs - so this one is rarer than it should be.

It's worth being clear about what this is not: it's not a lightweight utility. It's a 7B parameter language model, quantized to int4 to fit in consumer VRAM, and it needs a Hugging Face token because the weights are gated. The first run downloads several GB. If that sounds like a lot for "ask about a sound," you're right - but it's the difference between hardcoding audio analysis and just asking.

How it works

The node pulls Sergei6000/Qwen2-Audio-7B-Instruct-Int4 into ComfyUI/models/LLM/Qwen2-Audio-7B-Instruct-Int4 on first use, then loads it through transformers with device_map="auto" and bf16 if your GPU supports it (fp16 otherwise). Audio arrives in the standard ComfyUI AUDIO format and gets converted: resampled to the model's native sampling rate, mixed to mono, then fed into a chat-template conversation alongside your text prompt. It generates up to max_tokens tokens and returns the model's reply as a plain STRING - wire it to a text display node, a prompt field, or SaveText.

The force_offload toggle (default on) unloads the model after generation so it's not squatting on VRAM while the rest of your graph runs. Leave it on unless you're doing a batch of audio questions and want to skip the reload cost.

The inputs that matter

• text - your question or instruction. "Describe the mood of this music" or "transcribe and summarize."
• audio - optional AUDIO input. This is the star of the show: connect any audio source and the model hears it. Leave it empty and you just get a text-only LLM chat.
• max_tokens - default 256, up to 2048. Answers are usually short; raise it only if you're asking for transcriptions.
• seed - 0 rolls random sampling; a fixed seed makes generation deterministic.
• force_offload - keep it on.

The single output is STRING - the model's answer.

Installing

The node lives in the drmbt/comfyui-dreambait-nodes pack. Install the pack (ComfyUI Manager, search comfyui-dreambait-nodes, or git clone https://github.com/drmbt/comfyui-dreambait-nodes into custom_nodes and restart). The heavy lifting - transformers, accelerate, bitsandbytes, torchaudio, librosa - is all in the pack's requirements, and the model itself downloads on first run.

The auth wall, and why most people bounce

This is the #1 failure mode, and the node is upfront about it: the model is gated on Hugging Face. First run without a token throws a ValueError that tells you exactly what to do - accept the license on the model page, create a token at https://huggingface.co/settings/tokens, then either set the HF_TOKEN environment variable or run huggingface-cli login in ComfyUI's Python environment. It's a free account and a free model, but it is a wall, and if you're on a machine where you can't easily get a token, this node simply won't work - there's no anonymous path.

Two smaller gotchas: the download is big and looks hung on first run (it isn't). And if your audio input is stereo with wildly different channels, the mono mix can lose detail - for analysis purposes that's usually fine, but don't expect it to diagnose a hard-panned stereo image. When it works, though, it's the closest thing ComfyUI has to just asking an audio file what it is.