AuK Encoder Loader

Most ComfyUI workflows treat a text encoder as plumbing you forget about. Here it's half the system. AuK's encoder is a converted Qwen2.5-Omni model - the same checkpoint reads your instruction text and listens to your reference audio. That's the trick voice cloning and every edit task rest on: the model isn't given a text description of the reference clip, it hears it.

It also carries the language-model head used by AuK Prompt Enhance, which turns a sloppy request like "make her sound excited and say welcome home" into a canonical AuK instruction. So one 3B-ish file is the ears, part of the brain, and a small local LLM. That's why the node exists on its own instead of hiding inside the sampler.

What you actually set

• encoder_name - a dropdown of your ComfyUI/models/text_encoders/ contents. Sizes are 8.07 GB bf16, 4.67 GB int8, 3.18 GB w4a8; the language head always stays BF16, so W4A8 doesn't shrink it as much as the ratio suggests. One encoder can feed any number of AuK models, so if you keep Base and Flash both loaded you still only load one encoder.
• precision - compute dtype. auto is bf16 on GPU, fp32 on CPU. Weight format comes from the file, same as the model loader.

Output is a single AUK_ENCODER socket. Wire it into AuK Instruction Encode (.encoder) and, if you use it, AuK Prompt Enhance (.encoder).

Formats can be mixed freely - a bf16 encoder with an int8 diffusion model is fine, and mixing is exactly how you'd trade quality against fit.

The "no sidecars" part is genuinely nice

Qwen's config, tokenizer and processor files are bundled inside the node pack under assets/qwen2.5-omni-3b/ and loaded locally. So unlike most encoder swaps, you are not hunting for a tokenizer.json next to your weights - the README is explicit that you download only the weights and that no sidecars belong in models/text_encoders/. Nothing here reaches the network at runtime either; the processor is loaded with local files only.

Install

ComfyUI Manager → ComfyUI-AuK, or:

cd ComfyUI/custom_nodes && git clone https://github.com/Saganaki22/ComfyUI-AuK
cd ComfyUI-AuK && python -m pip install -r requirements.txt

The dependency list is small (transformers>=5.3,<6, safetensors, PyYAML, tqdm) for what this does - a pleasant surprise if you've been living inside the TTS node-pack dependency wars, where the standing complaint is that adding one model breaks three others through transformers/torch/tokenizer conflicts. This pack does reuse ComfyUI's own model management and attention paths instead of dragging in its own runtime, which is the reason the list stays short. Restart ComfyUI after install.

Troubleshooting

"Select a converted AuK Qwen encoder." You pointed the dropdown at a raw upstream Qwen checkpoint. Only files converted by the pack's tools/convert.py carry the auk_component: encoder metadata the loader checks for. Grab the qwen_omni_*.safetensors files from the Hugging Face repo instead.

"Checkpoint does not match the AuK encoder architecture." The converter and the pack drifted out of sync, or the file was cut from a different Qwen size. Re-run tools/convert.py --component encoder on the original Qwen directory with the current version.

Prompt Enhance complains about a missing language head. Some conversions were made without the LM head. Instruction Encode doesn't care - it only needs hidden states - but Prompt Enhance does, and it fails loudly rather than producing nonsense.

It's the VRAM you didn't budget for. This is the encoder-as-its-own-budget problem that concepts.md describes for image models, except here it's audio: treat the encoder file as a second model, not a footnote. The w4a8 encoder exists for exactly this, and a quantized model + encoder pair has been reported at roughly 8 GB peak. Report, not requirement.

One last thing: because this file also powers Prompt Enhance, an int8 or w4a8 encoder is doing double duty as a small LLM. Quantized language heads occasionally drop a quote or a comma when emitting JSON - the Prompt Enhance node has salvage code for that, but if you see malformed replies, that's the mechanism, not a mystery.