VLM Detections from JSON

Every detection node in gokayfem's VLM_nodes pack speaks one dialect: the VLM_DETECTIONS socket, backed by a versioned JSON schema (comfyui-vlm/detections, version 1). That's great while you stay inside the pack. The moment you want to save detections to disk, send them to another program, or load boxes that came from somewhere else, you need the two boundary nodes - and VLM Detections from JSON is the one that reads JSON back in.

It's the inverse of VLMDetectionsToJSON. One serializes a typed socket to a string; this one parses a string back into the socket. Unremarkable work, but it's the difference between a workflow that can persist its analysis and one that forgets everything between runs.

How it works

It takes one input, json_text - a multiline string holding a detections document. The parser is strict: it checks the schema and version, validates the media dimensions, frame count, and the box records, and produces a proper DetectionSequence on the detections output. Everything downstream that takes VLM_DETECTIONS - filters, renderers, mask converters, the SAM2.1 node - can then consume it as if a detector had just produced it.

The default value in the widget is an empty stub document, which is useful: you can paste real output from VLMDetectionsToJSON, from the json output of the open-vocabulary detector, or from a completely different system that implemented the same schema, and it just works.

Inputs and outputs

• json_text (STRING, multiline) - the only input. Paste or wire the JSON in.
• detections (VLM_DETECTIONS) - the only output.

One note on scope: this socket carries box/label/score data plus polygon or quad records, but not dense mask tensors. The pack deliberately leaves masks in-process - they'd balloon any JSON export by hundreds of megabytes. So if your pipeline needs the masks, keep them in the graph rather than round-tripping through JSON.

Installing this pack

Same install for the whole pack. ComfyUI Manager, search VLM_nodes, install. Or:

cd ComfyUI/custom_nodes
git clone https://github.com/gokayfem/ComfyUI_VLM_nodes
cd ComfyUI_VLM_nodes
python -m pip install -r requirements.txt

Run pip with ComfyUI's Python. This node downloads nothing.

Common issues

The error you'll actually hit is a validation failure from malformed JSON - usually a hand-edited document or output from a model that wrapped the JSON in markdown fences. Strip the fences before feeding it here (the pack's Text Clean node can do that for you). Version mismatches are the other one: if you saved data with an older schema version, the strict parser will reject it rather than silently guessing. That's a feature - silent re-interpretation of spatial data is how wrong masks get made.