AuK Instruction Encode X

Why you'd reach for it

Most ComfyUI audio workflows are: text encoder in, conditioning out. AuK looks similar and isn't. Its conditioning has to carry two things - a task instruction written in a fairly rigid template, and, for anything but plain speech synthesis, the audio being worked on. If you don't hand the model the source clip at the encoding stage, it has nothing to clone from or edit.

AuK Instruction Encode X is where that happens. It's the node that turns "Say the following with the same voice: '…'" plus a reference recording into the CONDITIONING that AuK Generate / Edit X samples against.

How it works

You need model here as well as at generation - wire AuK Model Loader X into both. That's not redundancy: Base and Flash carry their own learned Qwen layer-fusion weights, so the model participates in encoding, and both must feed this node. encoder comes from AuK Encoder Loader X.

Then instruction, which is the whole point. It's not a prompt in the image sense - it's a sentence in one of a set of known task shapes, and the model was trained to follow those shapes specifically. The tooltip gives the pattern with real examples:

• Say the following with the same voice: "Your text". - voice cloning
• Replace 'old words' with 'new words'. - content editing
• Raise the pitch by 2 semitones. - pitch
• Extract only the singing voice and remove accompaniment. - separation

The quoting is load-bearing. Anchors and quoted words in an edit instruction must actually occur in the recording, or there's nothing to edit.

audio is optional and it's the switch between two entirely different jobs. Leave it empty and you get text-only speech - describe the voice, specify the words, done. Connect it and every cloning and editing task becomes available. For editing, the tooltip says connect the prepared audio from AuK Prompt Enhance X if you're using that node; Prompt Enhance applies upstream's whisper RMS targets to the clip for whisper conversion, which matters because reduced volume alone isn't a successful whisper.

One output: CONDITIONING. That goes to AuK Generate / Edit X, and nothing else.

A detail that saves time: a connected STRING overrides what you typed in the widget. So when AuK Instruction Builder X is wired in and your edit doesn't happen, check the connection before you rewrite your text - you may be editing text nobody reads.

Install

ComfyUI Manager → search WorkflowX Configurator, or:

cd ComfyUI/custom_nodes
git clone https://github.com/haroonaslam/WorkflowX-Configurator
cd WorkflowX-Configurator
pip install "transformers>=5.3,<6" safetensors PyYAML tqdm

This node does nothing without the three AuK weight files: an AuK checkpoint in models/diffusion_models/, the converted Qwen encoder in models/text_encoders/, and the unquantized auk_vae.safetensors in models/vae/ - all from drbaph/AuK-comfyui, weights only. Restart ComfyUI after adding them, since model folders are scanned at startup.

Where it goes wrong

Encoding is once, before denoising. It's easy to imagine the instruction being re-read each step and to conclude that a wrong instruction will "partially" apply. It won't - you get the conditioning you built, and the sampling loop is downstream of it.

Vague instructions underperform. "Make it better" isn't a task. If you're not sure which task you mean, that's the signal to use the instruction builder rather than to write freer prose.

The model's own limits. AuK's documentation is upfront that prompt adherence isn't strong: cloning, TTS and some edits work, others are unstable, and that's the model rather than the node pack. Voice continuity and word preservation can vary, so listen rather than assume.

Nothing downstream. If generation fails immediately after this node, check that the encoder is the converted Qwen2.5-Omni file. The loaders will happily accept a similar-looking safetensors; this stage is where a wrong-shaped one shows up.