Strata 视频采样 / Video Sample

A video model can't watch your clip. Neither can a vision model, really - what they get is a handful of stills and a story about what happened between them. This node is the honest half of that trade: it decodes frames on the CPU, keeps real timestamps, and writes a manifest describing exactly what it did and didn't sample. Its partner node, StrataT8VideoAnalyze, refuses to work without that manifest.

How it works

Give it a core Load Video output and it decodes the file itself, on the CPU, with PyAV - ComfyUI's own av, so there's no ffmpeg subprocess to install or babysit. It preserves the container's presentation timestamps, honours the official VIDEO node's trim and crop fields and display rotation in the documented order, and picks frames across your window at sampling_fps. When that yields more than max_frames, it spreads the samples evenly across the selected window instead of just truncating, and records the resulting interval.

It accepts the reviewed VideoFromFile layouts (both the old file-only object and the current file+trim+crop one), VideoFromComponents and the current VideoFromList. Anything it doesn't recognise, or a file without duration metadata, errors out explicitly rather than guessing - and the error tells you to use the IMAGE input instead.

That IMAGE path is the second mode: feed it an existing frame batch and supply fps yourself. With variable-framerate footage, hand it timestamps_json (strictly increasing, one entry per frame) and it keeps your real times instead of estimating them.

Inputs

• video (optional, VIDEO) or images (optional, IMAGE) - you need one of them.
• fps - 0.001 to 240, default 24. Used for the IMAGE path, and as the fallback time base when a file has no usable rate.
• timestamps_json - strictly increasing seconds, one per input frame. Empty means it derives times from fps.
• start_s / end_s - the window. end_s: 0 means "to the end of the input". Positions go up to 86400s, but a single window is capped at 3600s.
• sampling_fps - default 1 (one frame per second), 0.01–30. Raise it for fast action; the model can only reason about what you actually sent.
• max_frames - 1 to 64, default 64. This is the hard budget everything downstream assumes.
• max_pixels - default 589824 (~768×768), up to 1048576. Per frame.

Outputs

frames (IMAGE) and manifest_json (STRING). Wire both into StrataT8VideoAnalyze - the analysis node cross-checks the tensor against the manifest and errors if the dimensions or frame count disagree. The manifest carries separate books for input position, clip-relative time and source PTS, plus a coverage block with the sample count and window, and the real tensor size where frames were letterboxed.

Install

Manager → search Strata-T8 (strata-t8, publisher t8star), or:

cd ComfyUI/custom_nodes
git clone https://github.com/T8mars/Comfyui-Strata-T8.git
python -m pip install -r Comfyui-Strata-T8/requirements.txt

Just jsonschema. Video decoding rides on ComfyUI's existing PyAV, and the sampling core is a fixed copy of the runtime's strata_video.py with its provenance and SHA-256 recorded in the repo - it does not need the runtime source tree on disk, and it starts no ffmpeg. Restart after installing, and keep one install directory so you don't double-register the node IDs.

Where people get burned

HDR and alpha. PQ/HLG files need converting to SDR first, and non-sRGB Components need converting to sRGB - the node composites transparent frames and separated alpha onto white, which is right for analysis and wrong if you were expecting an HDR pass-through.

Big source frames are refused, not resized forever. Individual source frames over 16Mi pixels, input files over 256MiB, and the total decode/sampling time budgets all error out. For a long 4K clip, export a lighter file first or feed it an IMAGE batch you already control.

Mixed-size frames get letterboxed onto the first frame's canvas, with the original size and content box recorded in the manifest. Reasonable, but if your analysis output mentions white bars, that's why.

A nonzero-PTS MKV is handled - the pack corrects the duration, tail trim and stitched timeline - but a file with no duration metadata at all is a hard stop. Re-mux it or use IMAGE.

Sampling is sampling. At 1fps, a 2-second gesture between samples doesn't exist as far as the pipeline is concerned. The manifest records the interval so the report can say so; your job is to raise sampling_fps or narrow the window when the action you care about is short.