Nodes/Mickmumpitz-Nodes/Audio Timestep Override
ComfyUI Node

Audio Timestep Override

The one knob that unfreezes audio-driven mouths

By mickmumpitz·Created 8 months ago·Updated 9 days ago· 47
Audio Timestep Override
  • model
  • model
audio_timestep_scale0.93

If you've run an audio-to-video lipsync workflow and gotten a video where the mouth just… doesn't move, the culprit is usually not the model - it's a mask. Audio-driven video models gate their audio cross-attention on a denoise mask, and the standard "protect the audio with SetLatentNoiseMask(0)" trick also zeroes the audio mask the model sees, which shuts the gate and freezes the mouth. AudioTimestepOverride exists to reopen that gate.

The source explains the failure precisely: with an input noise mask of 0, the model sets a_timestep = 0 for all audio tokens, and the audio-to-video cross-attention gate (computed from that value) closes. The mouth freezes even though the audio signal is intact. This node installs a model wrapper that replaces the audio denoise mask with a uniform non-zero value, so the model sees a_timestep = scale * sigma - audio participates in denoising again - while the sampler's inpainting blend still uses the original mask (preserving 100% of the audio signal). It's surgical: one model, one knob.

How it works

Two inputs:

  • model - the MODEL whose audio gate you want to open. Wire it before the sampler.
  • audio_timestep_scale - the gate-opening amount. Default 0.93, and the source recommends ~0.93. The tooltip is the spec: 1.0 = model sees audio at the same noise as video; 0.0 = original behavior (gate closed).

Output: model, the patched clone. Set the scale to 0 and the node returns the model unchanged - a clean bypass when you want to A/B test whether the gate is actually your problem.

That single number is the whole interface, which is the right design: the failure is binary (gate open or closed), so the knob is a dial on how fully open. Values near 1 are the sensible range; the default is a good starting point, and the tooltip's example workflow runs it at 0.93.

Where it fits

In the audio branch of a Wan audio-to-video (or similar) lipsync pipeline: Load Audio → mask → model → AudioTimestepOverride → KSampler. It's the difference between "mouth frozen" and "mouth moves with the track," and it replaces the fiddly approach of reworking the mask just to keep the audio gate open. Pair it with the pack's other Lipsync-category nodes (like the DWPose mouth-mask extractor) for a complete audio-driven face pipeline.

Installing it

Part of Mickmumpitz-Nodes:

cd ComfyUI/custom_nodes
git clone https://github.com/mickmumpitz/ComfyUI-Mickmumpitz-Nodes.git

or ComfyUI Manager → search "Mickmumpitz" → install → restart. No downloads from this pack; it needs whatever audio-to-video model you're already running.

Troubleshooting

  • Mouth still frozen - your audio branch may not be using SetLatentNoiseMask(0), so the condition this node fixes isn't the one you have. It targets the specific "mask-zeros-the-gate" case; verify that's your setup.
  • Audio too strong / overdriven - lower audio_timestep_scale from 0.93 toward 0.5. It's a dial, not a switch.
  • Scale 0 does nothing - correct: that's the bypass position. Use it to confirm the node is actually what changed your result.
  • "Why a wrapper and not a parameter?" - because the bug is in how the model processes the mask, not in a sampler setting. A wrapper is the only place that can intercept it.
CategoryMickmumpitz/Lipsync

Inputs (2)

NameTypeDefaultDescription
modelMODEL
audio_timestep_scaleFLOAT0.930–1Uniform scale for the audio denoise mask seen by the model. 1.0 = model sees audio at same noise as video. 0.0 = original behaviour (gate closed).

Outputs (1)

NameTypeDescription
modelMODEL