AuK Encoder Loader X

Why you'd reach for it

AuK doesn't take a prompt string. It takes an instruction - "Say the following with the same voice: '…'" or "Replace 'Tuesday' with 'Friday'" - and it has to hear the audio it's editing. So it uses a text-and-audio encoder: a converted Qwen2.5-Omni model that reads both the instruction and the reference clip, and whose hidden state becomes the conditioning the diffusion model samples against.

AuK Encoder Loader X is that encoder. It's the unglamorous middle of every AuK graph, and there's one rule that matters: encoder_name expects the converted Qwen2.5-Omni text/audio encoder, and "converted" is doing work in that sentence.

How it works

Both inputs are combos:

• encoder_name - the converted encoder checkpoint, from models/text_encoders/.
• precision - auto means bf16 on GPU, fp32 on CPU. As with the model loader, the weight format (bf16, int8 or w4a4) comes from the checkpoint itself; this setting only decides compute dtype.

The tooltip carries the one thing you can actually break: the encoder's config and tokenizer files must stay beside the weights. That's why the pack bundles the Qwen config, tokenizer, preprocessor and chat-template files under auk/assets/qwen2.5-omni-3b/ and loads them from there - you don't need sidecars in models/text_encoders/, and you shouldn't improvise your own. If you downloaded a raw Qwen directory and pointed the loader at the wrong shard, this is what fails.

One output: WORKFLOWX_AUK_ENCODER, and the tooltip's note is worth reading twice - one encoder can feed any number of AuK models. You don't need a second loader when you switch from Base to Flash. Wire it to AuK Instruction Encode, and to AuK Prompt Enhance if you're using the local prompt enhancer.

Sizes, if you're budgeting disk: the released encoder files are 8.07 GB (bf16), 4.67 GB (int8) or 3.18 GB (w4a4). The full original Qwen checkpoint is around 12 GB; this integration keeps the text/audio encoder plus the language-model head that Prompt Enhance needs. The head itself always stays bf16.

Install

ComfyUI Manager → search WorkflowX Configurator, or:

cd ComfyUI/custom_nodes
git clone https://github.com/haroonaslam/WorkflowX-Configurator
cd WorkflowX-Configurator
pip install "transformers>=5.3,<6" safetensors PyYAML tqdm

The Transformers version is not a suggestion - the AuK nodes need 5.3.x or a compatible 5.x. Then grab qwen_omni_bf16.safetensors (or the int8/w4a4 variant) from drbaph/AuK-comfyui into ComfyUI/models/text_encoders/, and restart. Only the weights: the pack ships the config and tokenizer it expects.

Where it goes wrong

Transformers version conflicts are the default failure mode in ComfyUI audio. The community-maintained TTS suites say it out loud - adding one model and breaking three others because of transformers, torch or tokenizer clashes. If AuK worked yesterday and doesn't today, the first thing to check is whether something else rewrote your Transformers version. The pack's own dependency list is deliberately narrow for exactly this reason.

If the encoder isn't the converted one, expect an error downstream at AuK Instruction Encode rather than here - the loader is happy with any safetensors it can read, and it's the encode step that discovers the shape is wrong.

Prompt Enhance needs the language head. All the released encoder files include it, but if you converted your own encoder without the head, Prompt Enhance is the node that errors, and its tooltip names the fix: re-run the conversion with --component encoder against the original Qwen directory.