HF Image Segmentation

The name sounds like it should download a model and eat your VRAM. It doesn't. HF Image Segmentation is a ComfyUI node that ships the heavy lifting to Hugging Face's hosted Inference API: you type a model name into a text box, feed it an image, and colored, labeled masks come back over the internet. No weights on disk, no GPU required, nothing to quantize. If you've ever wanted a model to tell you what's in your image and where - "person at 0.98, chair at 0.91" - this is the cheapest way to get that answer inside a workflow.

It's one of the image nodes in the bitaffinity/ComfyUI_HF_Inference pack, which is exactly what it looks like: a thin, unofficial wrapper around Hugging Face's Inference API. The repo is a single commit from mid-2024, Apache-2.0, and has been quiet since. That quiet is a feature here - the only Python dependency is requests, so install is trivial and it can't drag your environment into the dependency hell that eats so many ComfyUI setups.

What it does

The inputs are about as minimal as ComfyUI allows:

• endpoint (STRING) - the Hugging Face model ID, like facebook/detr-resnet-50-panoptic (the README's pick). It can also take a full URL.
• images (IMAGE) - any image tensor: from Load Image, from a VAE decode, from whatever you just generated.

Under the hood the node converts each image tensor to PNG bytes and POSTs them to https://api-inference.huggingface.co/models/{endpoint}. The server answers with JSON: a list of detected things, each with a label and a mask (a base64-encoded PNG). The node decodes every mask and pastes it onto the original image, so you see each object tinted over the source.

Because the class is marked as an output node with no return types, there's nothing to wire out of it. The result shows up as a preview on the node itself.

Where it fits

This is a panoptic segmentation model, not a background remover. If you're used to BiRefNet or rembg-style cutouts - clean subject versus background - those give you a matte you can composite with. This gives you a labeled, multi-class overlay. It's an inspection tool more than a cutout tool: good for tagging, for counting objects in a frame, for sanity-checking a generated scene before you commit to it.

Installing it

The README's path works for every node in this pack:

git clone https://github.com/bitaffinity/ComfyUI_HF_Inference custom_nodes/ComfyUI_HF_Inference
cd custom_nodes/ComfyUI_HF_Inference
pip install -r requirements.txt

Then restart ComfyUI. ComfyUI Manager can do the clone for you - search "ComfyUI_HF_Inference" - but you still need the token setup below either way.

Every node in the pack reads the HF_AUTH_TOKEN environment variable. A free Hugging Face token works; you're just on the rate-limited serverless tier:

HF_AUTH_TOKEN=hf_yourtokenhere python main.py

Where people get burned

The pack's own README is refreshingly honest: the serverless Inference API requires a model 10GB or below "and fails for random reasons on different models." Cold starts are the most common surprise - the first call to a model that's been unloaded can take a couple of minutes while HF spins up a container. The pack handles that gracefully (it reads the estimated_time from the error response, waits, and retries once), so you usually see a long pause rather than a crash.

One thing to know before you rely on it: the node loops over every image in the batch, but only the last image makes it into the preview. Feed it a single image and it's fine. Feed it a batch and expect the earlier masks to vanish.