Ke-Omni-R Audio2Text

Ke-Omni-R Audio2Text is the pack's local audio-understanding node: it runs the 3B Ke-Omni-R reasoning model on your own GPU, listens to an audio clip, and writes you a structured analysis - genre, instruments, tempo, mood - with a <think> block and an <answer>. It's the "no uploads" alternative to the API audio node, aimed squarely at music/audio LoRA training data, and the author clearly uses it for tagging tracks.

Reach for it when you're building an audio training set and you don't want your music sent to a cloud API. The model is an open reasoning audio model from the KE-Team built on the Qwen2.5-Omni architecture, and the pack feeds it a carefully crafted prompt (the same one the MultiLinePromptKOR node carries) that demands a strict JSON output: simplified and expanded tag lists, a descriptive sentence, synonyms, use cases, and an analysis object with genre/instruments/tempo/mood. That structured output is exactly what you'd caption a dataset with.

How it works

The node takes an AUDIO clip (or every file in an audios_dir folder), caches it to a temp WAV, and runs it through Qwen2_5OmniThinkerForConditionalGeneration + Qwen2_5OmniProcessor - the Qwen2.5-Omni Thinker path, which is what Ke-Omni-R's reasoning weights ride on. It appends an instruction to output thinking inside <think> </think> and the final answer inside <answer> </answer>, then extracts just the answer. It batches (batch_size) and can unload the model from VRAM after each run. Output is a text STRING - the JSON blob from the <answer>, which is what you want for a caption file.

The inputs that matter

• prompt (STRING, required, forceInput) - this input is force-connected, meaning you must wire it from another node. The intended source is the pack's MultiLinePromptKOR node, which holds the big audio-analysis prompt template. You can wire any text node, but that's the one the author built for it.
• audio (AUDIO, optional) - a single clip from the canvas.
• audios_dir (STRING, optional) - a folder path; it finds .wav/.mp3/.flac files and processes them in a batch. Give the node one of audio or audios_dir, not neither - otherwise it raises a ValueError.
• max_new_tokens / batch_size / seed - generation and batching controls.
• save_audio_prompt_to_folder - optional output folder for the prompt files.
• unload_model - frees VRAM after each run.

Installing it

Part of ComfyUI_Prompt-All-In-One (billwuhao) - Manager search Prompt-All-In-One or clone + pip install -r requirements.txt. The model is a manual download: KE-Team/Ke-Omni-R-3B into ComfyUI/models/LLM/Ke-Omni-R-3B, and here's the trap the README is very upfront about - the model is missing an index file. The pack ships model.safetensors.index.json in its repo root; copy that file into the model folder or the load will fail.

Where people get burned

This is the highest-effort node in the pack, and the community signal for it is essentially nil - you're an early adopter, so expect to fight setup. The three known traps: (1) the missing index file above; (2) torchaudio is imported but not in requirements.txt - if you hit No module named torchaudio, install the version matching your torch; and (3) the qwen_omni_utils dependency needs a version compatible with your transformers build. It's also a ~3B model running through the Omni processor, so it's not instant and it wants real VRAM while resident. If the API audio node is good enough and you don't care about uploads, it's a lot less friction. But for private, local, structured audio tagging, this node (with MultiLinePromptKOR wired into its prompt input) is the pack's payoff.