LTX-2 MLX Audio to Video (Song Mode)

The display name says "Song Mode", which conjures up dropping a three-minute track in and getting a music video out. What you actually get is one audio-conditioned shot: your music goes in, a clip comes out, frame-level timing included. You cut the shots together yourself.

That's still the best version of this feature anyone has locally. LTX-2 generates video and audio in the same diffusion pass, and driving that with an audio track is what made lipsync and music-video work LTX's strongest community use case. On an Apple Silicon Mac, this pack is how you get there at all - the official LTX nodes want CUDA-only fp8 kernels that MPS doesn't have.

How it works

The audio plumbing is unglamorous and worth understanding, because it explains the odd bits of behaviour.

Your audio input is a ComfyUI AUDIO tensor. The node takes the first item in the batch, transposes it to samples-by-channels, writes a temporary WAV with soundfile, and measures its duration. That WAV path goes upstream to the MLX pipeline, along with an audio_max_duration derived from the frame count it settled on. Afterwards the temp WAV and any temp conditioning PNG are deleted; only the mp4 survives.

Then there's match_audio_length, and this is the one setting that matters. With it on (the default), video length is derived from the audio's duration - and snapped down, not to the nearest value. With it off, num_frames is used and snapped to the nearest valid 8k+1 count, the same way LTX2MLXGenerate does it.

That asymmetry is deliberate and it's the fix for a real crash. Upstream sizes its audio-token RoPE tables from the video duration you request, regardless of how much audio actually exists - so asking for more video than you have audio blows up with a broadcast_shapes mismatch inside the audio attention step. Rounding down guarantees your video is never longer than your track. Padding the audio with silence was tried during development and reverted. Keep video ≤ audio, which this toggle does for you.

The inputs and outputs

Required: pipeline (from LTX2MLXAudioModelLoader - a different socket type than the T2V pipeline, so you can't mis-wire it), prompt, audio, height (default 480), width (default 704), frame_rate (default 24, an integer here because it's used in the duration maths), match_audio_length, num_frames (default 97, used only when the toggle is off), seed, and filename_prefix (defaults to ltx2mlx/a2v).

There's one optional input: image. Connect it and you're doing audio-plus-image-to-video - the clip starts from your still but stays conditioned on the track.

Output is a VIDEO socket, and like its T2V sibling this node saves and previews the mp4 itself, so no SaveVideo node is needed.

The maths you should actually do

num_frames maxes out at 257. At 24fps that's a shade over 10.7 seconds, and the manual slider defaults to 97 - about four seconds. That ceiling tells you the scale this node is built for: shots, not songs. With match_audio_length left on, the derived frame count comes from your audio rather than the slider, so a long track is asking for a very long render and a lot of memory; you'll have a much better time feeding it 8-12 second sections and cutting them together in an editor.

Also, because the number is floored to 8k+1, a 12-second section at 24fps derives 288 frames, floors to 281, and you're 7 frames (about 0.29 seconds) short of the audio. Trim the audio tail or nudge in post. Nothing is broken.

Installing it

Manager: search LTX-2 MLX (Apple Silicon). Manually:

cd ComfyUI/custom_nodes
git clone https://github.com/tanis2000/comfyui-ltx2-mlx.git
cd comfyui-ltx2-mlx
pip install -r requirements.txt

macOS on Apple Silicon only - MLX has no CUDA, Windows, or Linux backend - plus Python 3.11+ and a ComfyUI with the V3 node API. Weights arrive on first run: your chosen dgrauet/ltx-2.3-mlx tier plus a Gemma 3 12B 4-bit text encoder. Run hf auth login first; anonymous Hugging Face downloads get throttled hard, and you're pulling tens of gigabytes.

Where people get burned

You disabled match_audio_length on a long track. Now you're either waiting out a monster render or, if you set num_frames larger than your audio can cover, hitting the RoPE shape mismatch. Leave it on unless you specifically know your clip is shorter than the track.

The output looks right but sounds wrong. Audio quality here is conditioned on the prompt, the same way visuals are. "Wizard in forest" gets you near-silent room tone; "Wizard in forest, low whispered chant, ember crackle, distant owl hoot" gets you a soundscape with the elements you named, timed to the picture. Describe the sound the way you describe the shot - this is documented LTX-2.3 behaviour, not something this node can paper over.

Your clip is a couple of frames short of the beat. Floor snapping, covered above.

The machine is thrashing. Drop the tier to q4, turn low_ram on in the loader, and keep sections short. Above 32GB of unified memory you're comfortable at these resolutions; at 16GB, low_ram isn't optional.

The node refuses to run at all. Non-Apple hardware, or a Python running under Rosetta as x86_64. The loaders check platform.machine() and tell you plainly.