Load LLLite

You reach for this node for one reason: you want ControlNet-style conditioning on SD 1.5 or SDXL, but you don't want to load a second UNet-sized encoder alongside your checkpoint. That's the whole pitch. ControlNet-LLLite is kohya-ss's experimental take on control conditioning where the "control net" is a LoRA-shaped module injected into the existing UNet's attention layers, not a copied encoder joined through zero convolutions. It's a fraction of the VRAM and disk cost of a real ControlNet. It is also, in the author's own words, experimental, with a community reception that never got past "mostly curious." If you want reliable pose or depth conditioning on SDXL, a proper ControlNet is still the better tool - this is the lightweight experiment you try when the weight is the problem.

Don't confuse this pack with the newer ComfyUI-Anima-LLLite node that ports the same idea to Anima's DiT architecture. Same author, same technique, different pack, different models. This one is the original, built on laksjdjf's IPAdapter-ComfyUI, and it only targets the classic UNet checkpoints.

How it works

The conditioning image isn't fed to a control network. Instead, it's pushed through a small stack of strided convolutions that downsample it into a conditioning embedding at latent resolution, then normalized to -1..+1. That embedding gets attached to each LLLiteModule placed at every transformer block in the UNet. When attention runs, the module takes the incoming q, k or v, passes it through a LoRA-style down → concatenate-with-cond-embedding → mid → up path, and adds the result back, scaled by strength. The loader clones your model and registers this as an attention patch on both self-attention (attn1) and cross-attention (attn2), so the condition nudges spatial structure through the attention projections themselves. There's no copied encoder anywhere - that's the whole point, and why the models are so small.

The inputs that matter

model_name is a dropdown that lists every .safetensors in the pack's own models folder - nothing else, so your LLLite file has to live there or it won't show up. model is your checkpoint's MODEL, and cond_image is the control map (canny, depth, whatever), usually passed through an ImageScale node first so it matches your generation resolution. strength is the effect strength: 1.0 default, 0.0 is no effect, and you'll rarely go above it.

The three step controls are the clever bit. Because the node can't see how many total steps your sampler is running, you tell it: set steps to the sampler's step count, then start_percent/end_percent (0-100) define the window during denoising when the effect applies. Leave them at 0 and it applies for the whole run; set steps to 36, start_percent to 20 and end_percent to 80, and it only kicks in for the middle steps. The console prints start LLLite: step X / end LLLite: step Y so you can check the actual window.

The output is a single patched MODEL, and this is the wiring gotcha that trips everyone: it goes straight into the KSampler's model input, not into a ControlNet conditioning path. Wire it like a LoRA loader, not like a ControlNet.

Install

Easiest via ComfyUI Manager - search "ControlNet-LLLite" and install. Or the manual route:

cd ComfyUI/custom_nodes
git clone https://github.com/kohya-ss/ControlNet-LLLite-ComfyUI
# restart ComfyUI

There's no requirements.txt - the node only uses ComfyUI's own imports and torch. The actual work is the model download: grab sample models from kohya-ss/controlnet-lllite on HuggingFace and drop the .safetensors files into ControlNet-LLLite-ComfyUI/models/. The repo ships a sample SDXL workflow (lllite_workflow.json): load it, and you'll see the Canny map resized to 1024×1024 through ImageScale before hitting the loader.

Where people get burned

• Weak or broken output. The community's verdict on the SDXL depth model ranges from "totally hosed" (beige latent renders with artifact outlines) to "barely does anything," and openpose was reported as a no-op in some setups. LLLite is a research experiment with reference-quality weights, not a production pipeline. Expect to fiddle, and don't judge it by ControlNet standards.
• Size mismatch. The conditioning is embedded at latent resolution, so the cond image effectively needs to be square at your generation size. People hit "only accepts perfectly square input images at particular resolutions." Scale it with ImageScale like the sample workflow does.
• Wrong pack. Searching "LLLite" in Manager can surface the Anima port, which wants a different model directory and a different base model. Check you're on the original ControlNet-LLLite-ComfyUI if you're generating SDXL.

Honest bottom line: it's a neat proof of concept that costs almost nothing in VRAM, and the step-window controls are genuinely nice. If you need dependable structure control, use a real ControlNet. If you're curious about kohya's lighter take, this is the easiest door into it.