Strata 视频分析 / Video Analyze

Most "ask a model about a video" setups quietly pretend the model watched it. This node doesn't: it takes sampled frames plus the manifest that describes them, splits them into segments, and requires every event in the answer to cite frame IDs from the segment it came from. Cite a frame you weren't given, or claim a time range outside the frames you referenced, and the answer is rejected.

The workflow

Load Video → StrataT8VideoSample → StrataT8VideoAnalyze, with both outputs of the sampling node wired in: frames to frames, manifest_json to manifest_json. The analysis node then splits those frames into segments (8 by default), asks about each segment, and merges the segment summaries into one final answer. All of it happens inside a single engine lease, so a 64-frame clip doesn't reload the model per segment.

It's a vision-language job under the hood, which is worth being clear-eyed about: the KB's captioning section is blunt that every VLM in this class struggles with fine-grained attribution (the KB's llm-in-comfyui notes § 2). The pack's answer isn't to pretend otherwise - it's to make the model state its evidence and its limits, and to fail loudly when it makes things up.

Inputs

• connection - from StrataT8Connection.
• frames - the IMAGE batch, 1–64 frames.
• manifest_json - force-input from the sampling node. Contract version 1, dimensions must match the tensor.
• task - overview, timeline (chronological events with evidence frame IDs), storyboard (shots, composition, camera movement, distinguishing apparent motion from certainty), question, or prompt (reverse-engineer an English video-generation prompt).
• question - your actual question, used when you pick the question task; 4KiB max.
• mode - frames (default) sends stills to the existing vision API; native needs the service to declare video-protocol capability and will error outright if it doesn't. In frames mode segments are 2–8 frames; native allows 2–16 and pairs odd segments internally by duplicating the last frame.
• max_pixels - 65536–1048576, default 589824.
• max_tokens - default 2048, hard ceiling 4096 - different from the text nodes, and enforced.
• Sampling, plus an optional transcript. Note what that is: user-supplied text, never audio. The node doesn't load ASR weights and won't infer sound or music, and the report keeps your transcript separate from visual evidence.

Outputs

report (the prose report - or the prompt itself when task: prompt), analysis_json (the full document: report, events, shots, prompt, limitations, plus every segment's raw answer, source info and coverage), evidence_json (the frame records with IDs and timestamps), coverage_json (sampling window and interval), usage_json (per-stage token usage). It's an output node, so the report also previews in the node.

Install

Manager → search Strata-T8 (strata-t8, publisher t8star), or:

cd ComfyUI/custom_nodes
git clone https://github.com/T8mars/Comfyui-Strata-T8.git
python -m pip install -r Comfyui-Strata-T8/requirements.txt

jsonschema is the entire pip list. Video work reuses ComfyUI's PyAV; the engine and weights live in the portable runtime - the video nodes download nothing of their own. Note the version pairing: video nodes are the 1.1.x line, and frames mode needs a runtime of at least 0.1.39-t8.14 with native mode requiring an explicit capability declaration from the service.

Where people get burned

task: prompt will fail on purpose. If the final generation prompt comes back empty or whitespace-only, the node errors and cleans up the lease rather than handing you a report in the prompt slot. Right decision - you'd never notice the wrong thing in the wire otherwise - but it does mean the prompt task is the fiddliest one. Give it room with max_tokens.

Long answers blow the merge budget. Segment summaries are merged in groups of at most four, with a size cap on each merge; push past it and you get an explicit error telling you to shorten per-segment output or run shorter time windows. Setting max_tokens to 4096 with six segments is how you find that limit.

Times are relative to your window, not the film. The manifest keeps the input-to-source mapping, but the events you get back are timed against the sampled window. Don't read a report from a 300–320s window as a precise cut boundary in the original timeline - and the pack says plainly that it doesn't promise frame-exact cuts or continuous motion reconstruction.

Native mode is not a free upgrade. It requires the service to advertise the capability; without it you get an immediate error rather than a silent fallback. Frames mode is the compatible path, and it's the one the pack's own verification numbers were run against.

One frame proves nothing about direction. The evidence model requires multiple frames for motion, and the limitations list in the JSON will say so. If a clip's story is movement, raise sampling_fps in the sampling node - no amount of prompt tuning fixes a gap in the evidence you sent.