VLM Temporal Events From JSON

Small vision-language models are genuinely good at watching a video and saying "the person picked up the cup, then walked left." They are terrible at doing it in the same JSON format twice. VLM Temporal Events From JSON is the enforcement layer: it takes a VLM's messy textual output plus the exact frame selection you fed it, and produces a validated, timestamped, typed VLM_EVENTS list that downstream logic can actually depend on.

It's part of the pack's adaptive video-intelligence layer. The one-node path (VLMVideoTemporalReasoner) wraps this whole contract internally; this node exposes the same contract for people running their own local or hosted VLM and wanting the strict parsing without the rest.

How it works

You give it three things: text (the model's raw response - plain or fenced JSON), selection (the VLM_VIDEO_SELECTION from the sampler, which maps sampled images back to source frame indices and timestamps), and max_events. The parser walks the model's claimed events and validates every one: timestamps must be in range, evidence frame references must resolve to supplied images, confidence must be finite, no duplicate evidence, no malformed JSON. Violations fail validation rather than silently corrupting your data.

One subtle behavior worth knowing: small VLMs often report evidence as a position in the image batch you handed them rather than the source index. The parser accepts that form only when every value is an unambiguous valid batch position, then maps it back to the immutable source selection and records the normalization mode. You get clean source timestamps out, no matter how the model sloppily reported them.

Inputs and outputs

• text (STRING, multiline) - the VLM response.
• selection (VLM_VIDEO_SELECTION) - must come from the same sampling run the model saw, or validation can't map evidence.
• max_events - cap on parsed events, default 256.

Outputs: events (VLM_EVENTS), summary (STRING), and normalized_json (STRING) so you can see the cleaned, standardized result.

Installing this pack

It's the usual one-pack install. ComfyUI Manager: search VLM_nodes. Or:

cd ComfyUI/custom_nodes
git clone https://github.com/gokayfem/ComfyUI_VLM_nodes
cd ComfyUI_VLM_nodes
python -m pip install -r requirements.txt

ComfyUI's Python for pip; this node downloads nothing.

Common issues

The failure that bites is a mismatched selection: if the text came from a run that sampled different frames than the selection you're wiring in, evidence positions won't resolve and validation fails. Keep the selection socket and the model's input from the same sampler output. And don't fight the strictness - when the parser rejects a response, it's telling you the model hallucinated a frame number or a timestamp, and the correct fix is a better prompt (or the pack's VLMVideoReasoningPrompt, which is built to elicit exactly the format this parser wants).