RS Character LoRA

Here's the setup that used to be impossible without a lot of pain. Character A has a LoRA. Character B has a different LoRA. They need to be in the same picture, wearing the right faces. Stack both LoRAs on the model and cross-attention happily paints A's face on B's body, because a LoRA is a global patch - it doesn't know where it is.

RS Character LoRA is the third piece of the modular pipe, and it's the one that makes regional LoRA actually usable: a LoRA and model/CLIP strengths per slot, loaded and applied per-character instead of globally.

How the per-character part works

The sampler loads each distinct LoRA file once, builds a hook for it, and blends the resulting masked predictions so each region samples with the weights that belong to it. It goes through ComfyUI's native hooks rather than some private patch to the UNet, which is why the pipe travels intact into the detailer too - the detailer re-uses the matched character's LoRA when it re-renders that character's crop.

The cost is real and the author says so: distinct LoRA combinations require separate predictions. Two characters with two different LoRAs is not the same compute as one LoRA for both. On supported Anima/SDXL linear adapters there's a shared-layer path that keeps ordinary LoRAs from ballooning the expanded-weight storage, but don't expect a free lunch when every character has a different file.

Files are matched by architecture, as with any LoRA. An SDXL LoRA on a Krea 2 checkpoint does nothing useful, and Anime/Anima LoRAs are their own thing. Put the right family's files in ComfyUI/models/loras/ and refresh before you open this node - the dropdown is built from what ComfyUI can see at definition time, and a missing file is a hard error at sample time, not a warning.

The fields

character_count mirrors the other role nodes: it governs how many rows are visible. Key detail from the tooltip - the prompt node's count decides how many regions actually get generated; this node's count only decides which slots get LoRA settings. Set it to 2, then:

• character1_lora - a dropdown of your installed LoRAs, defaulting to None.
• character1_lora_strength - the model weight, −10 to 10, step 0.05.
• character1_lora_clip_strength - the CLIP weight, same range.

Anything you don't list stays at None with strength 1, which literally means "this character samples without a LoRA". That's the boring-but-correct default; a scene with three characters where only one is a specific design is a completely normal use.

The optional building_pipe input is what makes the assembly order arbitrary - connect Prompt, LoRA and Position into Finalize in whatever order the canvas makes prettiest. If you skip this node entirely, Finalize fills the slots with None and you just don't get per-character LoRAs.

Output is building_pipe, feeding Finalize (or another role node). The pack applies all of this through ComfyUI's native hooks, and it reports which LoRAs qualified for the faster shared-layer path in the run log.

Install

One pack, one install. Manager → search RS MultiCharacter Sampler, or:

cd ComfyUI/custom_nodes
git clone https://github.com/raspie10032/ComfyUI_RS_MultiCharacter_Sampler.git

Restart ComfyUI afterwards. The LoRA node itself needs nothing extra - no training stack, no diffusers, just the torch/NumPy/Pillow your ComfyUI already has and the lora files you already downloaded. The heavy optional dependencies (opencv-python, scipy, timm, huggingface-hub) belong to the detailer and tagger paths only.

Where it bites

Strengths for a regional LoRA want to be lower than you'd use globally - the character only influences its own region, so 1.0 on both model and clip often over-fires and gives you a pasted-looking face with weird neck shading. Try 0.7–0.9 and go up if the likeness is weak.

More importantly: two LoRAs, two predictions, and regional masking on top of that is the most fragile configuration in this pack. If a two-LoRA scene produces swapped outfits or a character wearing the wrong face, the fix is usually positions and box sizes rather than weights - the regions are the scaffold, and a bad scaffold beats a good LoRA every time.

And remember that a LoRA can't be attached to a text region. If Finalize complains about a text slot with a LoRA, that's the validation saying "spelling doesn't have a character design".