VLM Persistent Scene State

Object tracking gives you a timeline: at frame 12 object 3 was here, at frame 14 it was there, and so on. That's enormously useful and also not what you actually want when you're building anything that reasons about a scene - you want to know "is the box still there, how sure are we, and is it moving toward the door?" VLM Persistent Scene State is the node that compresses a noisy track timeline into that compact world-state summary.

It lives in the video-intelligence layer of gokayfem's VLM_nodes pack, and its whole point is to keep expensive generative inference out of the per-frame loop. You've already done the expensive work - tracking, detection, maybe temporal reasoning. This node just makes the result speak like a scene instead of a spreadsheet.

How it works

You feed it a VLM_TRACKS socket (from VLMTrackDetections, VLMSAM2VideoSegmentation, or the SAM3 adapter) and optionally a VLM_EVENTS socket (from VLMVideoTemporalReasoner or VLMEventsFromVideoJSON). It walks each durable object track and computes, per object: first and last observation time, current bounding box, confidence, state, and pixel velocity. The result is a VLM_SCENE_STATE object - a compact, versioned, persistent summary rather than a raw frame-by-frame dump.

Three outputs come out: the typed scene_state socket, scene_state_json (serialized for logging or persistence), and a human-readable summary string you can wire straight into a text viewer. It's pure computation - no model, no VRAM, instant.

Inputs and outputs that matter

• tracks (VLM_TRACKS, required) - without this there's nothing to summarize. Feed it from any tracking node in the pack.
• events (VLM_EVENTS, optional) - include it when you've run temporal reasoning, so the scene state reflects not just where objects are but what happened.

That's the entire input surface. Outputs are scene_state, scene_state_json, and summary.

Installing this pack

Same as every node in this pack - it's one repo, gokayfem's ComfyUI_VLM_nodes. ComfyUI Manager: search VLM_nodes. Or:

cd ComfyUI/custom_nodes
git clone https://github.com/gokayfem/ComfyUI_VLM_nodes
cd ComfyUI_VLM_nodes
python -m pip install -r requirements.txt

Use ComfyUI's Python for the pip step. This node downloads nothing on first run.

Common issues

The usual failure mode is upstream wiring, not this node: if tracks comes from a tracking session that was cut short (the pack is explicit that separate queue runs are separate tracking sessions, and IDs are only durable within one), your scene state reflects only that slice. For very long media, bound your slices, reseed each one, and keep the mapping in the caller - this node doesn't pretend IDs are globally stable across separate queues. Also, if you expected events and got none, check that you actually ran a temporal-reasoning node; without it the scene state is purely track-derived, which is still useful but has no "what happened" dimension.