AuK Prompt Enhance X

Why you'd reach for it

AuK wants its instructions in one of a handful of rigid templates. You want to type "make her sound excited and say welcome home". AuK Prompt Enhance X bridges that gap - a small local language model reads your loose request, picks the closest AuK task, fills the template, and hands back a properly shaped instruction.

And it does two more jobs that are easy to miss until you read the outputs. It calculates a task-aware duration, and it prepares the audio levels for whisper conversion. So it's not a prompt-quality toy; it's three of the fiddly parts of an AuK graph in one node.

How it works

encoder is the same converted Qwen2.5-Omni encoder you already loaded - but with one hard requirement: the checkpoint must include the language head. All the released encoder files do; if you converted your own, that's the node that errors and the tooltip names the fix (re-run tools/convert.py --component encoder on the original Qwen directory).

instruction is your loose request. context is optional extra text, and the obvious thing to put in it is the transcript from AuK Whisper Transcribe X - a transcript helps the model understand what's in the recording you're about to edit. audio is the source/reference clip. max_new_tokens (default 256) caps the answer length.

Everything runs in-process, as a local port of upstream's classify-and-render stage, and nothing here touches the network. That's the trade: you lose the upstream recommendation to route classification through a bigger hosted model, so classification quality can differ, and you gain a fully offline node.

Four outputs, and the tooltips are explicit that you should wire all the applicable ones:

• instruction → AuK Instruction Encode X's instruction input
• seconds → AuK Generate / Edit X's seconds
• task → the task the model picked, useful through a text display so you know what it decided
• prepared_audio → AuK Instruction Encode X's audio input

This is the local-LLM-in-the-graph pattern, and it's worth being clear about why it's local: not quality - a 3B model isn't writing better English than a frontier API - but privacy, offline operation and zero cost per call. The pattern's two characteristic failure modes both show up here in a mild form. The first is dirty output: a chat model emits chat, and this one gets constrained by asking for minified JSON and parsing it into a task and parameters rather than trusting prose. The second is drift: an enhancer that adds detail you never asked for. Narrowing the model's job is the community's standard defence - this node narrows it as far as it can, to picking one task from a fixed catalogue and filling that template's slots.

The task output is the node being honest about being a classifier. Before you trust a run, look at what it chose. If you asked for "say it softer" and it landed on a volume decrease when you wanted a whisper conversion, you'll see that in the task string rather than in a failed clip.

The whisper level matching

The prepared-audio half exists because whisper conversion has a specific failure mode: turning the volume down doesn't make speech whispered, and a model that just gets quieter output instead of a whisper conversion has failed even though the waveform looks right. This node applies the upstream whisper RMS targets to the clip before encoding, which is the part that actually gives the model something to convert from. If you route audio straight into Instruction Encode and skip prepared_audio, you're skipping that.

Install

ComfyUI Manager → search WorkflowX Configurator, or:

cd ComfyUI/custom_nodes
git clone https://github.com/haroonaslam/WorkflowX-Configurator
cd WorkflowX-Configurator
pip install "transformers>=5.3,<6" safetensors PyYAML tqdm

Plus the encoder in models/text_encoders/ - qwen_omni_bf16.safetensors or its quantized siblings from drbaph/AuK-comfyui. Restart after dropping files in.

Where it goes wrong

Malformed JSON from the model. The code is defensive about this, and for a good reason: quantized language heads drop a quote or a comma now and then. It repairs and salvages fields before giving up, but if you see a "language model returned malformed JSON" error, the recovery is usually a rerun or a simpler request rather than a settings change.

Disconnected seconds. The node's duration estimate is good for most edit tasks, but for a difficult edit the AuK task guide recommends setting the output 0.2–0.5 seconds shorter than the source for better instruction adherence. That means unplugging the seconds link and typing the number - which is a deliberate move, not a workaround.

Audio-dependent behaviour. Edit-task duration calculation needs audio connected. No audio, no duration estimate - which is fine for text-only speech, and confusing if you didn't intend text-only.