VLM SAM3 Track Adapter

SAM3 already did the hard work. It tracked objects through your video and gave you dense per-object masks - so why would you want to run segmentation again? The VLMSAM3TrackAdapter exists to answer exactly that: it's not a second SAM3 loader, it's the translation layer that takes ComfyUI's core SAM3_TRACK_DATA and exposes it as the lightweight VLM_TRACKS format this pack's tracking nodes speak.

The problem it solves is one of incompatible vocabularies. If you've built with SAM3 before, you know its output is the core ComfyUI type SAM3_TRACK_DATA - a bit-packed bundle of mask data that's great for what it is, but only useful to other core SAM3 nodes. Meanwhile, everything in the VLM_nodes vision/tracking category - track reports, track-aware crops, scene-state building - wants VLM_TRACKS. This node is the handshake between the two worlds.

How it works

Mechanically it's honest about being an adapter. It takes your track_data, validates the SAM3 layout, and walks through the packed masks frame by frame, computing each object's bounding box, timestamp (only if you supply a real fps - set fps to 0 and timestamps stay unknown), and per-object score. It does not unpack the full mask volume or duplicate it in memory or JSON; the dense masks stay where SAM3 put them. The track_data output is a straight passthrough of what you fed in, untouched.

The two optional inputs, seed_detections and seed_tracks, are worth knowing about if you've already run a VLM detection pass over the same video. They attach label, text, and score metadata to the SAM3 objects so your VLM_TRACKS come out labeled instead of anonymous object IDs. Skip them and the tracks are still fully usable - you just get bare IDs.

What wires where

The two outputs are the whole point of the node:

• tracks (VLM_TRACKS) - feed this into VLMTrackReport for a readable summary, VLMTrackAwareCrops for per-object crops, or VLMBuildSceneState for a persistent scene-state.
• track_data (SAM3_TRACK_DATA) - the passthrough. Keep it connected to core SAM3_TrackPreview or SAM3_TrackToMask when you actually need a dense mask rendered. The pack's own guidance is to only reach for SAM3_TrackToMask when you need the actual mask, not out of habit.

Install

The node ships in ComfyUI VLM Nodes (gokayfem/ComfyUI_VLM_nodes). In ComfyUI Manager, search "VLM Nodes" and install, or clone manually:

cd ComfyUI/custom_nodes
git clone https://github.com/gokayfem/ComfyUI_VLM_nodes
python -m pip install -r ComfyUI/custom_nodes/ComfyUI_VLM_nodes/requirements.txt

Run the pip command with ComfyUI's Python, and never let the repo install or replace torch - ComfyUI's own installer owns that.

The gotcha that bites everyone

SAM3 weights sit under Meta's SAM License, and the upstream facebook/sam3 repository requires you to accept access terms and share account info before download. The ComfyUI checkpoint is marked sam-license too. The example workflow names sam3.1_multiplex_fp16.safetensors - if you can't get it (or won't accept the license), don't substitute an unrelated checkpoint and hope. The pack's own advice is blunt: use the SAM2.1 workflow instead, which lives in the same pack (VLMSAM2VideoSegmentation) and produces VLM_TRACKS directly. Better a working SAM2.1 track than a broken SAM3 import.