Load OWL Model

This is the node everything else in the OWL-ViT ComfyUI pack hangs off of. Load OWL Model pulls in a real object-detection model - Google's OWL-ViT / OWLv2 - so the rest of your graph can look at an image and tell you where the objects are.

If you've never met OWL-ViT, quick version: it's an open-vocabulary object detector. Unlike the Stable Diffusion checkpoints you're used to, which generate pixels, this one understands them. You feed it an image, it runs a vision transformer over it and outputs bounding boxes. "Open-vocabulary" is the selling point - it can find categories it was never explicitly trained on, because it matches against a text encoder rather than a fixed class list. The name is a lie in the best way: no API, no key, no cloud. It's a Hugging Face checkpoint running entirely on your machine.

What it actually does

The node is a thin wrapper over transformers' Owlv2Processor and Owlv2ForObjectDetection. When you hit Run, it calls from_pretrained on whichever checkpoint you picked, shoves the model onto CUDA if it's available (CPU otherwise), and hands you the whole thing as a single OWL_MODEL output. That custom type is the only thing the rest of this pack knows how to talk to.

There's one input, and it's optional:

• model_name - your pick of two Hugging Face checkpoints. The default is google/owlv2-large-patch14-ensemble; the other option is google/owlv2-base-patch16-ensemble. Large is the stronger detector and roughly a couple of GB on first download; base is under a gigabyte and noticeably lighter on VRAM. Start with base unless you have the card for it.

The output, OWL_MODEL, is the entry point - wire it into OWL_Objectness_Inference and nothing else.

Installing it

It's a small pack from Jannled, and like most small packs it's a work in progress - the README is cookiecutter boilerplate and the tests still reference a node class that no longer exists, so treat it as experimental and keep backups. The lazy path is ComfyUI Manager: search for "OWL-ViT ComfyUI" and hit install, then restart. The manual path is the same as always:

cd ComfyUI/custom_nodes
git clone https://github.com/Jannled/owl-vit-comfyui

Then restart ComfyUI. The dependencies are declared in pyproject.toml - torch, transformers, scipy, matplotlib, and opencv-python - and Manager will pull those in for you. Manual installs may need pip install -e . from inside the folder if imports fail.

Where people get burned

• The first run downloads the model. No error, just a long pause while a few GB come down from the Hugging Face Hub. If you're behind a proxy or in a region where HF is throttled, this is where it hangs.
• It reloads every time you Run. The code has a loaded_models cache dict, but nothing ever writes to it, so the "return cached model" branch never fires. Each queue run rebuilds the model from disk. Not a bug that breaks anything, just a few seconds you can't avoid per run.
• fp32 weights. Neither checkpoint is quantized, so the large one leans on your VRAM. It still fits on a normal card, but don't stack it alongside a diffusion model in the same graph without watching your memory.

The takeaway: this node is the boring start of a clever pipeline. It's not where the action is - the Objectness node is - but nothing runs without it.