Apply ControlNet (Ray)

ControlNet on a multi-GPU setup has a problem the normal pipeline never faces: the full ControlNet weights, which can be gigabytes, would normally get serialized into every conditioning packet and shipped to every worker. RayControlNetApply is the Raylight way around that. Instead of embedding the whole model into your conditioning, it attaches a lightweight reference, and the workers restore the real ControlNet from their local cache during sampling. The source docstring says it plainly: "Works like ApplyControlNet but uses a lightweight reference instead of embedding the full model in the conditioning data."

The inputs that matter. Six required, one optional:

• positive and negative - your conditioning pair; both come back modified, ControlNet-injected.
• ray_control_net - the lightweight handle from RayControlNetLoader. Not the weights, just the reference.
• image - the conditioning image (canny edge map, depth, pose, whatever your ControlNet consumes).
• strength - how hard the ControlNet pushes, default 1.0, range 0–10. Same semantics as core ApplyControlNet.
• start_percent and end_percent - the sampling window where the ControlNet is active, defaults 0.0 to 1.0. Narrowing this is the classic trick for "let ControlNet nail the structure early, then let the model breathe."
• ray_vae - optional. Needed when the ControlNet's architecture requires VAE weights on the workers (the Flux-style ControlNets are the case you'll actually meet). If your workflow errors on a missing VAE context, this is the port to fill.

Outputs. positive and negative, both CONDITIONING, both ControlNet-applied. These feed your sampler exactly like core ApplyControlNet output would.

The constraints that matter. The README is explicit: only one ControlNet model per workflow. You can't stack two different ControlNets the way you can in core ComfyUI. What you can do - and the README calls this out - is hang multiple RayControlNetApply nodes off a single RayControlNetLoader to apply the same model with different images or strengths. That's the intended way to do multi-zone control. And remember this whole ControlNet feature is experimental, added "by avtc," so budget a little patience.

Where it fits. This is the distributed twin of ControlNetApplyAdvanced-style wiring. In the Raylight world you'd use it in USP/FSDP workflows where the sequence is split across GPUs - the ControlNet conditioning has to land consistently on every rank, which is exactly why the "reference + worker-local restore" design exists.

Install. Part of the pack:

cd ComfyUI/custom_nodes
git clone https://github.com/komikndr/raylight
cd raylight
pip install -r requirements.txt

or ComfyUI Manager → search "raylight" → Install, restart. ControlNet model files go in the usual ComfyUI/models/controlnet folder; xfuser is the heavy dependency; Windows means WSL2.

The one-ControlNet-per-workflow rule is the thing that'll trip you up if you're used to stacking three of them. Design around it - one model, multiple applies - and the rest of the node behaves exactly like the core one you already know.