Nodes/ComfyUI-Advanced-ControlNet/Latent Keyframe πŸ›‚πŸ…πŸ…’πŸ…
ComfyUI Node Runs on cloud

Latent Keyframe πŸ›‚πŸ…πŸ…’πŸ…

Per-frame ControlNet strength in ComfyUI

By KosinkadinkΒ·Created 3 years agoΒ·Updated 8 days agoΒ· 993
Latent Keyframe πŸ›‚πŸ…πŸ…’πŸ…
  • prev_latent_kf
  • LATENT_KF
β—„batch_index0β–Ί
β—„strength1.000β–Ί

If you're generating a batch of latents - an AnimateDiff clip, a batch of frames, a batch of variations - and you want the ControlNet to grip one specific frame harder or softer than the rest, this is the node. A Latent Keyframe says "for the latent at batch index N, apply the ControlNet at strength X." Everything else in the batch is left alone.

That's the whole idea, and it's smaller than it sounds. A regular ControlNet applies one strength across the entire batch. Latent Keyframes let you vary that strength per latent instead. The pack's author puts it plainly: a latent keyframe achieves the same effect as a uniform mask with the chosen strength value - it's just a much cheaper way to say "this frame gets 0.9, that frame gets 0.3" than painting masks.

Where you'd actually reach for it

This node earns its keep in batch and video work. The classic move: in an AnimateDiff-Evolved workflow, pin the ControlNet hard on your first frame (batch_index 0, strength 1.0) so the composition locks, then let it fall off across later frames so the motion module has room to breathe. Or the reverse - control only the middle of a clip and let the ends drift. On a single image, honestly, don't bother; with one latent there's nothing to schedule.

Advanced-ControlNet is Kosinkadink's pack, the same dev behind AnimateDiff-Evolved, so it's no accident that the scheduling nodes and the sliding-context video nodes play nicely together. That pairing is the reason latent keyframes exist.

Inputs and outputs that matter

Only two inputs do real work:

  • batch_index - which latent in the batch this keyframe targets. It's the "key." If that index never shows up during sampling, the keyframe simply has no effect (no error, no warning - it's a no-op). Negative indices aren't supported here; for those you want Latent Keyframe Group with a latent hooked up.
  • strength - the ControlNet strength applied to that one latent. 1.0 is full, 0.0 is off.

The optional prev_latent_kf is how you build a schedule: chain one Latent Keyframe into the next and each adds its own index/strength. If two keyframes in the chain claim the same batch_index, the one carried in from prev_latent_kf wins.

The single output, LATENT_KF, goes one of two places: into the latent_keyframe input of a Timestep Keyframe (if you're also scheduling across sampling steps), or straight into the latent_kf_override input on Apply Advanced ControlNet if latent scheduling is all you need.

Installing the pack

Grab it through ComfyUI Manager - Install Custom Nodes, search "ComfyUI-Advanced-ControlNet," install, restart. Or from a terminal: cd ComfyUI/custom_nodes && git clone https://github.com/Kosinkadink/ComfyUI-Advanced-ControlNet, then restart. No extra Python dependencies; it's pure ComfyUI. Two things new users miss: the pack ships no preprocessors (canny/depth/openpose live in Fannovel16's comfyui_controlnet_aux), and it ships no ControlNet models - you still need the actual .safetensors in ComfyUI/models/controlnet.

Common gotchas

The big one is the silent no-op. Set batch_index to a frame that doesn't exist in your batch and nothing happens - no crash to tell you why the strength you dialed in did nothing. Double-check your batch size and count from zero.

Second: chaining one node per frame gets old fast. If you're setting more than three or four, switch to Latent Keyframe Group (index=strength strings and python-style ranges) or Latent Keyframe Interpolation. And remember the strength here stacks with the strength on the Apply node - they multiply, so a 0.5 keyframe under a 0.8 Apply is really 0.4.

CategoryAdv-ControlNet πŸ›‚πŸ…πŸ…’πŸ…/keyframes

Inputs (3)

NameTypeDefaultDescription
batch_indexINT0-9007199254740991–9007199254740991β€”
strengthFLOAT1.0000–10β€”
prev_latent_kfoptLATENT_KEYFRAMEβ€”

Outputs (1)

NameTypeDescription
LATENT_KFLATENT_KEYFRAMEβ€”