OmniShotCut Detector

OmniShotCutDetector is the brain of the ComfyUI_OmniShotCut pack. You feed it a video, it hands back a list of every shot boundary in that video - as frame numbers you can act on - plus a visual preview you can actually eyeball. If you've ever sat through a 10-minute Wan or LTX render just to find the one cut that ruined the take, this is the node that finds it for you.

The name is a lie, kind of: it isn't "detecting" shots with a pixel-difference threshold the way the old ffmpeg select='gt(scene,0.3)' trick does. It runs a real transformer model (OmniShotCut, from UVA's Computer Vision Lab, itself a DETR-style detector with a ResNet18 backbone). The video gets decoded by ffmpeg to raw frames at a small 128×96 working resolution, then scanned in overlapping ~100-frame windows. Each window emits candidate shot ranges plus a transition type, and the node stitches and merges them into one global list of [start_frame, end_frame] pairs. That's the part that's actually useful: it classifies how each cut happens, not just where.

Inputs and outputs that matter

• model - the OMNI_MODEL output of OmniShotCutLoader. Wire that up first; without it you get a "Model input is empty" error string out.
• video - VIDEO. This is the nice part: the code defends against every way a video lands in ComfyUI. It takes the native ComfyUI VIDEO object, a plain path string, a VHS-style dict with a path/filename key, or a tuple. So you can feed it straight from ComfyUI's own video input or from Video Helper Suite.
• mode - clean_shot (default) vs default. This is the one that changes your output meaningfully. default treats every transition - dissolves, wipes, fades, zooms - as its own shot, which is what you want for studying edit style. clean_shot filters down to general hard cuts only, which is what you want before you start cutting for an edit.
• num_context_frames - overlap between analysis windows, default 5. Leave it unless you're chasing a missed boundary right at a window edge.

Outputs: shots_json (STRING) - a JSON blob with pred_ranges (list of [start, end] frame pairs), pred_intra_labels (shot type: General, Dissolve, Wipes, Push, Slide, Zoom, Fade, Doorway…), and pred_inter_labels (Hard_Cut, Transition, Sudden_Jump…). Feed this into OmniShotCutShortCount to count shots or OmniShotCutVideoSplitter to cut one out. preview (IMAGE) - a contact-sheet of frames with colored boundaries and mm:ss markers, capped by preview_max_pages (default 4) × preview_frames_per_page (default 264) frames per page. Wire it to a Preview node and you'll see where the cuts land.

Install

This pack needs ffmpeg on the system before anything else - the detector shells out to it to decode video.

# in ComfyUI/custom_nodes
git clone https://github.com/GG-A9/ComfyUI_OmniShotCut
cd ComfyUI_OmniShotCut
pip install -r requirements.txt

You also need the checkpoint dropped into checkpoints/OmniShotCut_ckpt.pth (see the loader's article for the exact command - a missing model shows up as a Chinese "no .pth file" message in the Loader dropdown). ComfyUI Manager works too: search for "ComfyUI_OmniShotCut" and install.

Common issues

• shots_json comes back as "Path Error: ..." or a "Detection Failed" string. The node couldn't resolve or open your video path, or ffmpeg choked on it. Check ffmpeg is on your PATH (ffmpeg -version), and that the video actually exists where the node thinks it is.
• The preview looks wrong but the JSON is fine. The preview labels frames at a fixed 24 fps regardless of your source framerate - cosmetic, harmless, don't panic.
• It's slow on long videos. Small model, but it decodes the whole video frame by frame. Short source clips (a few seconds to a couple minutes) are the sweet spot.