ComfyUI Agent Bridge

<p align="center"><img src="docs/icon.png" width="96" alt=""></p>

ComfyUI-Agent-Bridge

A human-in-the-loop channel between the ComfyUI canvas and local AI agents: Claude Code, MCP clients, scripts, or anything else that can make an HTTP request.

MCP servers such as comfy-mcp let an agent build and run workflows, but the agent cannot put a graph in front of the person, and cannot see what that person picked, liked or changed. This pack adds that return path:

• Propose. An agent proposes a workflow. Every open ComfyUI tab asks the user; on Confirm it opens in a new tab, saved under workflows/agent_bridge/, so Ctrl+S writes back to it.
• Agent Variant Switch. An N-way switch with labelled radio buttons. Only the chosen branch runs, and the pick is recorded for the agent.
• Agent Feedback. Thumbnails with 👍/👎, keep/change chips, a rating, a push/pull slider and free text. A Send to agent button delivers them without running anything.
• Agent Report Sink. Forwards any node output, such as JSON reports, strings or numbers, to the agent.
• Events. One long-poll endpoint returns proposal decisions, feedback, reports and finished runs, together with the pick and the feedback of each run.
• Canvas sharing (opt-in). While the user turns it on in Settings, a local agent can read the live graph.
• Notify. The agent can show a toast in the user's open tabs.

It was built for model-bending research (ComfyUI-Model-Bending), but nothing in it is specific to bending.

Install

It needs ComfyUI ≥ 0.18 and frontend ≥ 1.40, because it uses the V3 node API with autogrow and match-type inputs. There are no dependencies beyond ComfyUI itself. Install it any one of these ways, then restart ComfyUI:

• ComfyUI-Manager: search for Agent Bridge and install.
• comfy-cli: comfy node install comfyui-agent-bridge
• Manually:

  cd ComfyUI/custom_nodes
  git clone https://github.com/abuzreq/ComfyUI-Agent-Bridge
  

After the restart, the three nodes are under agent_bridge, Settings → Agent Bridge holds the two switches (open proposals without asking, share canvas), and the example appears under the templates of this pack.

Nodes (category agent_bridge)

| node | inputs | outputs | |---|---|---| | Agent Variant Switch | candidates (grows as you connect; any type; lazy), selected (1-based), labels (one per line) | selected, index, label | | Agent Feedback | images (optional batch), labels, session_id, round, rating 0–5, push −1…1, direction, plus hidden votes/chips managed by the UI | feedback_json, chosen_index | | Agent Report Sink | reports (grows; any type), channel, session_id | none (output node; shows what it sent) |

Everything the nodes learn is also written to the node's UI output. It therefore appears in ComfyUI's own /history, for agents that only speak the core API.

HTTP API

Routes live under /api/agent_bridge/ (also mounted without /api).

| route | caller | purpose | |---|---|---| | GET info | anyone | version, capabilities, sharing state. Local non-browser callers also get token_file | | POST propose {name, workflow, message?, session?, round?, save?} | agent | ask the open tabs to open a graph (frontend format) | | GET proposal/{id} | agent | pending / opened / dismissed | | GET events?since=&session=&kinds=&wait= | agent | long-poll. Kinds: proposal, proposal_status, feedback, report, run | | GET canvas | agent | the user's active graph (403 unless sharing is on) | | POST notify {message, severity?} | agent | toast in the open tabs | | GET proposals/pending, GET proposal/{id}/workflow, POST proposal/{id}, POST feedback, PUT/DELETE canvas | this pack's frontend | same-origin only |

TOKEN=$(cat ComfyUI/user/agent_bridge/token)
curl -s -H "X-Agent-Bridge-Token: $TOKEN" "http://127.0.0.1:8188/api/agent_bridge/events?since=0&wait=30"

A run event looks like this:

{"id": 17, "kind": "run", "session": "demo", "data": {"prompt_id": "…", "status": "success",
 "agent": {"session": "demo", "round": 1},
 "switch_picks": [{"node": "27", "index": 2, "label": "B · scale in.mid ×0.7", "labels": ["…"]}],
 "feedback": [{"votes": {"B · scale in.mid ×0.7": "up"}, "chips": {"palette": "keep"}, "rating": 4,
               "direction": "B's frame, but keep A's headland", "chosen_index": 2}],
 "reports": [{"node": "28", "channel": "bends", "count": 3}], "errors": []}}

Put extra.agent (any JSON object) in the workflows you propose. It survives the user's edits and saves, and comes back in every run event, so the agent can match runs to its own rounds.

Security

ComfyUI has no authentication, and is often started with --enable-cors-header="*", which lets any website you visit call its API. This pack therefore uses two route classes:

• Agent routes need a token. It is generated on first start in ComfyUI/user/agent_bridge/token and sent in the X-Agent-Bridge-Token header.
  • A local process can read the file; a web page cannot.
  • The header is not in ComfyUI's CORS allow-list, so browsers will not even send it cross-site.
• Browser routes are same-origin only. They are checked with Origin / Sec-Fetch-Site, so a foreign page or a plain script cannot post fake feedback or read proposals.
• The user stays in control:
  • Proposals never open without the user's confirmation, unless they turn on Settings → Agent Bridge → Open agent proposals without asking.
  • Canvas sharing is off until the user turns it on.
  • The event log stays on your machine: user/agent_bridge/sessions/<session>.jsonl.

If ComfyUI is exposed on a network (--listen), treat the token like a password.

Example

example_workflows/agent_bridge_prompt_variants.json (also in ComfyUI's template browser, under this pack). It uses v1-5-pruned-emaonly.safetensors; pick any SD1.5 checkpoint you have.

1. Three candidate prompts are rendered on the same seed.
2. They are shown in Agent Feedback for voting.
3. They are routed through Agent Variant Switch into a 4-seed render of the pick.

Load it, press Run, then pick, vote and press Send to agent.

Tests

python_embeded/python.exe ComfyUI/custom_nodes/ComfyUI-Agent-Bridge/tests/test_bridge.py

This runs 15 tests covering the security matrix, the event store and long-poll, every route, the three nodes (including the switch's lazy evaluation) and the release metadata. If the pack does not sit in custom_nodes, set COMFYUI_DIR. CI runs them on CPU against ComfyUI v0.18.2 and master.

Releasing

1. Bump version in pyproject.toml and VERSION in agent_bridge/bridge.py (a test checks they match), and add a CHANGELOG.md entry.
2. Push a tag v<version>. The Publish to the Comfy Registry workflow publishes it, using the repository secret REGISTRY_ACCESS_TOKEN (a publishing key of the abuzreq publisher on https://registry.comfy.org).

Pull requests are also checked with Comfy-Org's node-diff, which flags input and output changes that would break saved workflows.

License

MIT