Local Forge UI Controller (InstantID + Canny)

ForgeUIController is a single node that hands your generation off to a separately running local Forge UI (or Automatic1111) instance and brings the finished image back into your ComfyUI graph. Despite "ForgeAPI" in the pack name, there's no cloud API and no key: it talks to Forge's local /sdapi/v1/txt2img endpoint over HTTP on your own machine.

Why would you bother? The author's premise is that Forge's InstantID implementation renders better than the ComfyUI equivalents. InstantID - the January 2024 InstantX face adapter that works zero-shot from one photo - is SDXL-era, and Forge baked InstantID/ControlNet support into its built-in extensions. If you already run Forge for face work but want to stay in ComfyUI, this saves rebuilding the whole InstantID + Canny stack as ComfyUI nodes. It's a niche glue node and honest about it: one image in, one image out.

How it actually works

The interesting part is the two-way handshake. When the node loads, it calls three Forge endpoints - /controlnet/model_list, /sdapi/v1/loras, and /sdapi/v1/sd-models - and uses the results to populate your dropdowns. So the model pickers you see are a live listing of what's inside Forge's model folders, not ComfyUI's.

At generate time it builds an A1111-style txt2img payload with three ControlNet units always attached:

• InstantID IP-Adapter (InsightFace (InstantID) module) - instantid_image supplies identity.
• InstantID face keypoints (instant_id_face_keypoints module) - IdentityNet-style landmark conditioning that locks face geometry to the reference (the half of InstantID that FaceID lacked).
• Canny - edge structure, active over roughly the first half of steps (canny_start_step 0.0 → canny_end_step 0.51).

Your two image inputs get base64-encoded into PNGs, posted to {forge_url}/sdapi/v1/txt2img, and the returned image is decoded back into a normal ComfyUI IMAGE tensor. It blocks until Forge finishes - a 300-second timeout - so slow renders don't die on you. If you set forge_url to disabled, none, or skip, it skips the request and returns a black placeholder - handy for bypassing the node without rewiring your graph.

The inputs that matter

The ones a beginner actually touches:

• forge_url - where Forge lives. Default http://127.0.0.1:7860 is right if you're running it on the same machine.
• sd_model - your checkpoint, pulled from Forge's list. The default sample is zep7_v2, a Juggernaut-family SDXL model, which tells you the intended neighborhood.
• steps - the README recommends 8–12 (8 for speed, 12 for quality). That low step count is the whole trick: the default lora_model is dmd2_sdxl_4step_lora_fp16, a distilled 4-step LoRA, paired with LCM sampler, Exponential scheduler, and cfg_scale at 1.0.
• instantid_image and canny_image - your face reference and your edge/composition reference. Both optional, but this is a face node; feed it a face.
• The three weight sliders (instantid_weight, instantid_keypoints_weight, canny_weight) - the InstantID panel's community consensus band is 1.0–1.3, which matches the node's 1.0 defaults.

Two quirks worth knowing: the LoRA isn't loaded as a separate file - it gets injected into your prompt as <lora:dmd2_sdxl_4step_lora_fp16:1.1>, A1111-style. And if you only provide a canny_image, it also gets applied to the face-keypoints ControlNet unit. Pick None and tune weights to opt out.

The single output is IMAGE - wire it into a Save Image / Preview node like any generator.

Installing it (the setup is 80% Forge, not ComfyUI)

The node itself installs like any custom node: ComfyUI Manager (search "ForgeAPI for InstantID") or

cd ComfyUI/custom_nodes
git clone https://github.com/kreonxv/ComfyUI-ForgeAPI-for-InstantID-workflow

then restart ComfyUI. No heavy deps - it only needs requests, numpy, PIL, and torch, which ComfyUI already ships.

The real work is on the Forge side. You need Forge running with the API flag:

python launch.py --api

and these files in Forge's models/ControlNet/ folder (not ComfyUI's):

• ip-adapter_instant_id_sdxl.safetensors - the InstantID IP-Adapter
• control_instant_id_sdxl.safetensors - keypoints
• diffusers_xl_canny_full.safetensors - Canny

Plus an SDXL checkpoint and the dmd2_sdxl_4step_lora_fp16 LoRA in Forge's models/Lora. Getting this wrong is the #1 "nothing works" cause: the node runs in ComfyUI but reads Forge's disk.

Troubleshooting

• Empty or default-only dropdowns. The model lists are fetched when the node loads, so start Forge before ComfyUI, or reload the browser tab after Forge is up. Classic gotcha.
• "Failed to connect to Forge UI" - Forge isn't running, or wasn't started with --api. Verify the API is alive:

  Invoke-WebRequest http://127.0.0.1:7860/controlnet/model_list | Select-Object -Expand Content

• No images returned - a required model isn't in Forge's models/ControlNet, so the ControlNet unit silently fails. Confirm all three appear in the dropdowns.

One honest caveat: this is an SDXL-only tool because InstantID is an SDXL-only adapter - it never got a Flux port, and the reference repo has been dormant since mid-2024. On SDXL it still works and pairs well with Canny - exactly the combo this node hardcodes. On anything newer, this node is a museum piece. If you live on SDXL and love Forge's face results, it's a tidy bridge; otherwise there's little reason to reach for it.