Ideogram 4 Regional LoRA

If you've ever loaded two character LoRAs into one Ideogram 4 image and watched both faces come out as the same smeared average of the two, this node is the fix. Ideogram 4's JSON prompt gives every element a bounding box, but those boxes are just text tokens to the model - it has no spatial handle to hang a per-region LoRA on. Load two character LoRAs normally and both deltas land in the weights globally, so everyone in the frame wears both identities at once. Ideogram 4 Regional LoRA applies each LoRA's delta only to the image tokens inside its own box, in a single forward pass, and leaves the base weights alone.

One scope note: this is for the local Ideogram 4 DiT - the ideogram4_*.safetensors weights, e.g. ideogram4_fp8_scaled - not the cloud API nodes. API users, wrong pack.

How it works

Ideogram 4 runs one packed [text | image] sequence where the image tokens form a row-major (gh, gw) grid. That layout is the trick: a bbox maps exactly onto a set of token positions, so the node can turn your drawn rectangle into a mask over actual tokens. At forward time it reads the sequence layout off the model wrapper's own arguments, then registers a forward hook on each target Linear that adds the LoRA's low-rank delta weighted per token:

out = base_out + w_token · B(A(x))

The delta lives in activation space (bf16), never baked into the fp8 weight, which keeps ComfyUI's own weight-patching and fp8 casting path untouched. The hooks are installed per forward and removed in a finally, so a clone stays cheap and nothing leaks between runs. Measured on ideogram4_fp8_scaled against a stock global LoRA, you get roughly 88% of the in-bbox strength with about 3× less off-target change; the residual is bake-vs-bypass on quantized weights, not a real difference.

The inputs that matter

You get up to 8 LoRA slots, each a trio of widgets: lora_N (pick the file, or leave at None to skip), strength_N, and region_index_N - which placed bbox that LoRA applies to. That last one is where people get burned - covered below.

After the slots sit the three tuning knobs:

• feather_cells - softens every region edge in latent cells, 1 cell ≈ 16 px. Default 1; 0 is a hard edge.
• overlap_mode - what happens where two boxes intersect: normalize scales down to sum 1, sum stacks and can over-drive, first_wins lets the earlier slot occlude.
• mask_adaln - includes the broadcast adaln_modulation deltas, worth 19–26% of a face LoRA's effect. Defaults on; turning it off saves a ~150 MB/block transient tensor at 4k tokens.

Regions come from either the bboxes output of Ideogram 4 Prompt Builder KJ (with width/height wired in) or the caption prompt JSON string itself, which takes precedence. Ideogram 4 does CFG with two models, so patch the unconditional one too: wire it into model_uncond. Both model and model_uncond outputs feed a DualModelGuider. The demo workflow in examples/IDEOGRAM4_REGIONAL_LORA_demo.json (a face + hands pairing) mirrors a standard Ideogram 4 path but replaces the usual pair of throttled-to-0.2 global LoRA loaders with one of these nodes.

Install

Via ComfyUI Manager (search "Ideogram 4 Regional LoRA"), or the manual way:

cd ComfyUI/custom_nodes
git clone https://github.com/kat3ri/ComfyUI-ideogram4-regional-lora
# restart ComfyUI

No extra Python dependencies - the one requirement is a ComfyUI recent enough to include comfy/ldm/ideogram4/ (local Ideogram 4 support). If you're running the model, you already qualify.

Where people get burned

• region_index is not the prompt builder's numbering. It counts placed boxes in editor order, zero-based - the KJ builder's own preview tags boxes 01, 02 one-based and counts unplaced elements too. Typing one of those numbers silently selects the wrong region. That mismatch is exactly what the pack's sibling node, Ideogram 4 Region Preview, exists to solve - run it and read the numbers off the image.
• Don't also load these LoRAs in a Power Lora Loader. That applies them globally and the whole masking becomes moot.
• Leaving model_uncond unwired is a choice, not a default. It keeps the uncond at base weights, which makes CFG pull each region back toward base identity - a legitimate A/B that amplifies the delta, but not what you want by accident.
• Attention crosses the seam by design. Tokens in the bbox still exchange keys and values with the rest of the image at every layer, so the delta is concentrated in the box, never perfectly confined. The leakage is diffuse drift, not a boundary halo; feathering softens it but won't remove it.
• It fails closed, not wrong. If the token layout can't be resolved, the node skips to base weights rather than applying a character LoRA to the whole image. Symptom: your LoRA does nothing at all - the safe failure, and a useful diagnostic.