Cosmos Reference Conditioning

The name undersells it. Cosmos Reference Conditioning isn't an IPAdapter-style "show me a style image" node - it's the control channel for Anima, the Cosmos-Predict2-based anime base that still doesn't have a production ControlNet. If you caught the LoRA drop on r/StableDiffusion, this is the pack that shipped with it: the author trained an overnight scribble/canny/depth LoRA for Anima and built these nodes so the control strength is actually adjustable. The example workflow in the repo is literally named "Anima Control."

Here's the shape of it. Anima's control situation is thin: as of the KB's last read it only has experimental LLLite sample weights, with pose and depth deliberately weak. This pack is one of the few real levers for steering with an image, and it's the lever Anzhc built for their own gens when they "needed some control but was not satisfied with existing solutions."

How it works

You feed the node your positive and negative conditioning plus a reference_latent - in practice the VAE-encoded control image (the repo's workflow runs a scribble/canny/depth render through VAEEncode and LatentMultiply before this node). The node stamps metadata onto the conditioning: which latent to attach, at what strength, and with which method. Then the patched model (that's the pack's other node) reads it during sampling.

Two control methods, and the difference is worth understanding:

• Temporal mask (default): the reference latent is concatenated as extra temporal content and the model's padding mask is the control signal. Strength scales the reference mask - lower values suppress it, higher values expose it more strongly.
• KV gating: reference tokens enter through the attention keys and values. Strength under 1.0 uses a reduced subset of tokens, exactly 1.0 uses the reference once, and above 1.0 repeats tokens to push influence higher.

The author's own recommendation: KV gating for more linear scaling, temporal mask for a smoother effect.

The inputs that matter

• positive, negative - your CONDITIONING from CLIPTextEncode.
• reference_latent - a LATENT; VAEEncode your control render.
• strength - FLOAT, 0 to 3, default 1.0.
• control_method - the enum above.

Outputs are positive and negative CONDITIONING, straight into the KSampler.

The trap: it needs its training

Both methods require specialized training to work. That's the README's own warning, and it's the thing that trips people up. Without the companion LoRA (CabalResearch/Anima-Canny-Scribble-Adjustable-Control-LoRA on HuggingFace), KV gating produces visible artifacts and temporal mask has little to no effect. The LoRA is what taught the model to interpret the reference latent as a control signal - the node just carries it.

Install

ComfyUI Manager, search "Anzhc-ComfyUI-Cosmos-Reference", or:

cd ComfyUI/custom_nodes
git clone https://github.com/Anzhc/Anzhc-ComfyUI-Cosmos-Reference

Restart ComfyUI. There's no requirements.txt - the code only uses torch, einops (bundled with ComfyUI) and ComfyUI's own internals, so this is one of those rare clean installs. Nothing to download from the pack; you bring the Cosmos model and the control LoRA yourself.

Troubleshooting

• "Strength does nothing" - the real reports on this one are almost always wiring. The conditioning metadata only matters if the patched model is in the sampler path, so this node must be paired with ApplyCosmosReferenceModelPatch. And on temporal mask without the LoRA, a flat result is expected behavior, not a bug.
• Visible artifacts / garbage - you're on KV gating without the trained LoRA. The README called this.
• A RuntimeError about concat_padding_mask=True - temporal mask only works on Cosmos models that expose that flag (the Anima / Predict2-2B line). It won't run on SDXL or Flux.
• Latent shape mismatch errors - the reference latent's channels and spatial size must match your target; resize the control render first.

Anzhc is a known quantity in this space - they've shipped FLUX VAE and latent-cleanup tools on that famous 4060 Ti - and this pack is an improved fork of Mirumo0u0/ComfyUI-Cosmos-Reference. For Anima users wanting image control without waiting on an ecosystem that hasn't arrived, this is the one.