AudioX Enhanced Video to Audio

The basic AudioX Video to Audio node works, but it gives you one CFG dial to balance two competing signals: your text prompt and the video frames. When "the visuals say one thing and the audio obeys the other," you're stuck. This enhanced node is the fix - it splits the balance into separate text and video CFG scales plus separate conditioning weights, which turns video-to-audio from a coin flip into something you can actually tune.

Mechanically it's the same AudioX diffusion pipeline (250-step dpmpp-3m-sde, video frames as IMAGE conditioning). What changes is how conditioning is assembled: the node builds enhanced video conditioning with both the text prompt and the video frames, attaches weight metadata for each, then computes an effective CFG scale as a weighted average of text_cfg_scale and video_cfg_scale. So when you push text_weight up, you're telling the model to lean on your description; push video_weight and it leans on what's on screen. If your video reads "rain on pavement" but the audio keeps going ambient-musical, raise video_cfg_scale/video_weight and drop the text side - that's the whole point of this node.

Inputs that matter

• model - from the AudioX Model Loader (AUDIOX_MODEL).
• video - frame batch (IMAGE) from your video loader.
• text_prompt - the description that steers generation.
• text_cfg_scale / video_cfg_scale - 0.1–20, both default 7. Text CFG controls prompt adherence; video CFG controls how hard the model listens to the frames. This pair is the headline feature.
• text_weight / video_weight - 0–2, default 1. Fine-tunes the balance between the two conditioners, on top of the CFG pair.
• steps / seed / duration_seconds - standard: 250 steps, -1 seed, 10s.
• negative_prompt (optional) - what to avoid. Caveat below.
• prompt_template (optional) - 6 presets (nature_forest, urban_traffic, action_footsteps, etc.) appended to your prompt.
• enhance_prompt (optional, default true) - auto-appends audio keywords.

Output is audio (AUDIO), ready for the combiner or a save node.

The honest caveats

• Negative prompting here is mostly decorative. The enhanced text nodes' tooltips literally say negative prompting is "currently logged only - implementation pending," and in this node the negative prompt gets stored in the conditioning as metadata rather than driving a true negative/positive CFG split. It won't hurt to fill it in, but don't expect it to veto "wind" the way you'd expect.
• It's one effective CFG, not two. The "separate CFG scales" become a weighted average before sampling. That still gives you the text-vs-video balance control - just know the model itself isn't running two independent guidance passes.

Install & troubleshooting

Same pack ritual as everywhere:

cd ComfyUI/custom_nodes
git clone https://github.com/lum3on/ComfyUI-StableAudioX.git
cd ComfyUI-StableAudioX
pip install -r requirements.txt

Weights + config.json from HKUSTAudio/AudioX on Hugging Face into ComfyUI/models/diffusion_models/ (weights renamed AudioX.ckpt), with ffmpeg/libsndfile at the system level.

What actually bites people:

• First run is slow. The heavy generation stack imports lazily on first queue, and the pack auto-installs missing deps at startup. Give the first run minutes.
• VRAM. The README's spec is a single 4090 / ~16GB VRAM. Shrink duration_seconds, cut steps, or both.
• Mushy results = conditioning imbalance. Before blaming the model, push video_cfg_scale up and text_weight down, and write a tighter prompt. This node exists to win that fight.
• Frames must actually be frames. Feed it a video file path and you'll get a type error - it wants IMAGE.

This is the pack's most capable node and the one worth building a real workflow around. It's still early local v2a (expect rough edges on hard cases), but with two dials per signal instead of one, it's the version you can actually steer.