Qwen2 AQA

This is the whole point of the pack: a Qwen2-Audio 7B instruct model, int4-quantized, running fully local inside your graph. The name "AQA" is "audio question answering," and that's exactly what it does - feed it an audio file and a question like "what do you hear in this clip," and it returns a plain-text answer. No API calls, no keys, no cloud. You pay for it in VRAM and one big first-run download instead.

The genuinely useful trick is that it doesn't need audio at all. Leave source_audio_path empty and it's just a local 7B chat model, which makes it handy for brainstorming prompts or rewriting a caption mid-workflow. But where it shines is the audio path: captioning sound files for audio-to-video pipelines, describing what's happening in a clip you can't listen to right now, or turning a scene's audio into a prompt for a video model. That's the use case people actually reach for it - someone in r/comfyui recommended this exact pack to turn audio into a descriptive breakdown that an LLM then reshapes into a video-generation prompt.

How it works

Under the hood it's a standard transformers pipeline: Qwen2AudioForConditionalGeneration plus the matching AutoProcessor, with sdpa attention, bfloat16 on Ampere-and-newer GPUs (compute capability 8+), fp16 otherwise, and device_map="auto" so it'll spill to CPU RAM rather than hard-fail when VRAM runs tight. Audio is loaded with librosa at the processor's sampling rate and woven into the chat template. Generation is capped at 256 tokens - plenty for a caption, short for an essay.

The inputs that matter

• text (multiline) - your question or instruction. The one you'll actually type.
• source_audio_path (optional PATH) - wire this from the pack's AudioLoader node. Leave it unconnected for text-only chat.
• keep_model_loaded (boolean, default false) - this is the trap. On false, the model is deleted and VRAM freed after every run, so a follow-up question reloads the whole 7B. If you're chaining several queries, flip it to true and it stays warm until you want it gone.
• seed (int, default -1) - -1 means unseeded/random; set a number to reproduce an answer.
• model - a dropdown with exactly one choice (Qwen2-Audio-7B-Instruct-Int4). Not much of a choice.

Output is a single STRING with the model's response - wire it into any text display or downstream node.

Installing and the first-run wall

Install the pack via ComfyUI Manager (search Qwen2) or cd ComfyUI/custom_nodes && git clone https://github.com/IuvenisSapiens/ComfyUI_Qwen2-Audio-7B-Instruct-Int4, then pip install -r requirements.txt and restart. Dependencies include transformers>=4.45.0, bitsandbytes, librosa, huggingface_hub, and torch. Note bitsandbytes is what makes int4 load at all, and it wants a working CUDA install.

On your first run the model (Sergei6000/Qwen2-Audio-7B-Instruct-Int4 on HuggingFace) auto-downloads into ComfyUI/models/prompt_generator/ - around 5 GB, so go make tea. Where people get burned: they see a hang on first run and assume it's broken; it's just downloading. Then budget ~7–8 GB VRAM to run it comfortably, and remember the default keep_model_loaded=false means every run after that pays a reload penalty too. If your GPU can't hold it, device_map="auto" doesn't crash - it just offloads and makes you wait.

The catch, if there is one: at 256 tokens of output it's a captioning tool, not a conversational assistant, and the int4 quantization trades a bit of quality for that VRAM win. For describing audio it's hard to beat on a local-only setup.