🖥️ Web ROI Capture

WebROICapture turns whatever is on your screen - a game, a 3D viewport, a video, a drawing app - into a live IMAGE tensor inside your workflow, with a drag-and-crop region of interest. No screenshots, no file exports, no OBS, no virtual camera, no Python packages to babysit. You hit START on the node, your browser asks what to share, and from then on every frame gets pushed into the graph. Pair it with Auto Queue and a fast model and you've got real-time screen-to-image - the same genre as ScreenDiffusion and the Stream Diffusion crowd, except it lives entirely inside ComfyUI's own canvas.

The part most people miss: the capture is client-side, not server-side. The node is a shell - the real work happens in js/web_roi.js, which calls the browser's native navigator.mediaDevices.getDisplayMedia() and gets your stream from the browser itself. Which means it works on headless cloud boxes. AutoDL, RunPod, Colab - the server never needs a GUI, a display driver, or a virtual display. As long as you can see the ComfyUI web page in your browser, you can stream your screen into it.

How it works

Under the hood it's a tidy little handshake between the JS front-end and a Python node. On every frame, the JS draws the shared video onto the node's preview, lets you drag a yellow selection box, and on release locks it as a red ROI. That selection is stored as normalized 0–1 proportions, so it survives you resizing the node or the capture resolution changing. The selected region is then cropped through a hidden canvas and exported as a JPEG data URL (quality 0.7, so expect mild compression - fine for restyling, not for pixel-perfect work) into a hidden widget.

The Python side (web_roi.py) just waits for that string. It strips the data:image/jpeg;base64, prefix, decodes it, opens it with PIL, and converts to a float32 IMAGE tensor in the standard ComfyUI range. If there's no data yet, it returns a black 512×512 image instead of erroring - that's why you'll see black frames until you actually hit START and grant the share. Not a bug, it's the node keeping your graph alive while it waits.

The inputs that matter

Most of the node is mouse-driven, so you barely touch the schema:

• image_data (STRING, required) - you don't set this. The JS fills it every frame; the widget is hidden on the node.
• trigger_always (INT, optional) - the honest take: nothing in the shipped code actually reads this. It's exposed as a re-trigger hook, but the value is ignored, so you can leave it alone. Don't hunt for a meaning that isn't there.
• Output image (IMAGE) - this is a plain ComfyUI image tensor, so it wires into anything that takes an image: img2img, a VAE Encode into the sampler, an image-to-video model, a ControlNet preprocessor. Your screen becomes a source node.

Installing

Trivial, and there's no model download and no requirements.txt - the only Python deps are torch, numpy, and PIL, which ComfyUI already ships. Either search ComfyUI-WebROI in ComfyUI Manager, or:

cd ComfyUI/custom_nodes/
git clone https://github.com/reeoi/ComfyUI-WebROI.git

Then restart ComfyUI and add the node via search - it's listed as "🖥️ Web ROI Capture". That's the whole install.

Where people get burned

The README's one CRITICAL warning is real and it's the thing that'll trip you up on a rented box: browsers block getDisplayMedia on insecure (HTTP) origins. http://localhost works because localhost counts as a secure context, but http://your-server:8188 is dead on arrival - the START button will just fail. On cloud, get an HTTPS link: RunPod's proxy link, a Cloudflare Tunnel, or ngrok. The Chrome chrome://flags/#unsafely-treat-insecure-origin-as-secure escape hatch exists but it's for testing only; don't ship a workflow that depends on it.

A few smaller ones worth knowing:

• Don't collapse the node while streaming - the JS bails out of drawing (and therefore capturing) when the node is collapsed, and your image goes stale.
• Keep the browser tab alive. The video element lives in the page; switch to another tab or stop sharing and the stream dies. This is a "it's right in front of you, browser-based" tool, not a background daemon.
• A tiny click (under 10px either way) is treated as a cancel, not a crop - drag a real box. Double-click or the RESET ROI button clears it.

For the real-time loop: connect the output to your img2img-ish graph, enable Auto Queue in the ComfyUI menu, and the frames stream continuously. It's not a production pipeline, and the per-frame base64 handshake isn't winning any speed awards - but for "point ComfyUI at your screen and mess around live," it's about the lowest-friction option that exists.