Nodes/ComfyUI-BFSNodes/BFS Shot Planner
ComfyUI Node

BFS Shot Planner

Stop expecting a video model to follow 30 seconds of guide

By alisson-anjos·Created 7 months ago·Updated about 10 hours ago· 129
BFS Shot Planner
  • mask
  • vlm
  • shots
  • count
  • fps
  • width
  • height
  • audio
  • summary
  • timeline
  • refs
  • ref_count
  • second_pass
  • upscale_width
  • upscale_height
◄plan{"video": "", "fps": 24.0, "grid": "H3 (17n+5)", "mode": "shots", "max_s": 4.5, "min_s": 1.0, "sensitivity": 0.5, "max_parts": 0, "max_total_s": 0.0, "start_s": 0.0, "end_s": 0.0, "bounds": [], "segs": [], "global_refs": [], "global_ref": "", "global_ref2": "", "global_prompt": "", "global_color_guide_ref1": "#00ff00", "global_color_guide_ref2": "#9b30ff", "picture_colors": [], "megapixels": 0.5, "multiple": 32, "generation_size_mode": "megapixels", "custom_width": 1024, "custom_height": 1024, "detector": "adaptive", "run": "recursive", "filters": {}, "skip_fill": "original", "cast": {}, "cast_split": false, "cast_only": false, "mask_cfg": {}, "vlm_cfg": {}, "audio_mode": "auto", "ref_details": {}, "picture_details": {}, "picture_cfg": {}, "auto_prompt": {}, "auto_prompts": {}, "between_shots": "keep", "shot_cache": "output", "shot_storage": "lossless", "unload_before_join": true}►
◄prompt—►

The problem it solves, which is not the one you think

You feed a video model a 40-second guide clip and it follows it beautifully for the first five seconds and then starts summarising: the room changes, the framing drifts, the person's clothes reinvent themselves at the two-thirds mark. That isn't a bug in your workflow. Models follow guides reliably only inside the clip length they were trained on - an H3 body-swap LoRA trained on 107-frame clips holds the scene and motion at 4.5 seconds and loses them at 10. The fix is boring and effective: cut the source into model-sized shots and render each one properly.

That's this node. Pick a video in its timeline panel, let it split at camera cuts (or by fixed length, or by hand), give shots their own reference images and prompts, and the rest of your workflow runs once per shot.

How the loop actually works

The shots output is a ComfyUI list, and ComfyUI runs any node that receives a list once per item, pairing lists by index. That's the whole mechanism. Shot i gets guide i, reference i, prompt i; single values like the model and VAE are shared across every run. Downstream you feed that list into BFS Shot H3 Conditioning or BFS Shot Unpack, sample, decode, and give the decoded list to BFS Shot Join, which is a list-input node and puts the video back together with the original timing and soundtrack. You write the workflow once.

Two run modes, and picking the right one is the practical decision here. Auto loop renders every shot in a single queue run - simple, but all decoded shots sit in memory. Queue loop renders one shot per run, stores it on disk, and re-queues itself; the join blocks everything after it until the last shot, so a Save Video node only fires once. Long video, limited VRAM: queue loop.

What you set, and what the panel does for you

The one required input is plan, a JSON string the panel writes - video path, split settings, boundaries, per-shot references and prompts. You're not meant to hand-edit it, though nothing stops you.

Optional inputs override the panel: ref_image and ref_image_2 become the default references for any shot without its own (and beat the panel's global reference), prompt does the same for text, and vlm takes a Qwen3-VL (via a normal CLIPLoader, qwen3vl_4b or qwen3vl_8b) that looks at every shot and can suggest what to segment, a description of the shot, and whether it should run at all.

The panel itself is the selling point, and it's unusually loaded:

  • Splitting - PySceneDetect camera cuts (adaptive or content, from the scenedetect dependency; there's a built-in detector when it's missing), fixed equal-length parts, or by hand by dragging boundaries. Minimum and maximum seconds per shot stop you generating 0.3 s fragments or 20 s monsters, and length converts to frames snapped down to the frame grid (H3 17n+5, LTX/Wan 8n+1).
  • Guide frames - a shot shorter than the grid-valid length gets its extra frames from the video that follows it, so the guide never repeats or stretches. Those frames double as a real overlap, which the join uses to cross-fade cuts that aren't camera cuts.
  • Filters - skip shots with no person, too-small people, too many people, no face, near-black frames or no motion. YOLO from models/ultralytics for people and faces, with OpenCV fallbacks.
  • Cast - Find people tracks faces through the video with InsightFace (buffalo_l in models/insightface, on the CPU) and groups them into people; link a reference to a person and every shot picks it up from whoever has the most screen time. Good for a two-hander where the references swap.
  • Mask & crop - segment something per shot with SAM 3 text or points, crop to it, and the join pastes the result back feathered. SAM 3 weights download themselves on first use.
  • Continuity - a shot can carry the previous shot's generated frame as an extra reference, or anchor it as frame 0. It needs the previous result to exist, so it lives in queue loop or with the duet renderer.

Outputs: shots (the list), count, fps, width, height, audio (the whole soundtrack, trimmed to the planned duration), summary (human-readable plan), timeline, and ref_image / ref_image_2 - the references used, deduplicated, so a plan where everyone shares one reference gives you a single image.

Install

Manager → ComfyUI-BFSNodes, or:

cd ComfyUI/custom_nodes
git clone https://github.com/alisson-anjos/ComfyUI-BFSNodes
cd ComfyUI-BFSNodes && pip install -r requirements.txt

The planner's UI is frontend JavaScript in the pack, so reload the browser tab, not just the ComfyUI backend - a stale tab is why people think the pack installed wrong.

Where it bites

Skipped shots only get filled with the original video (or dropped) if you connect timeline to BFS Shot Join - without that wire, the filters silently shorten your video. insightface==0.7.3 is pinned, so cast detection is the feature most likely to conflict with another pack's dependencies. And changing the plan starts a new run, so don't tweak a boundary mid-queue-loop expecting the shots already on disk to be reused.

CategoryBFS/shot loop

Inputs (4)

NameTypeDefaultDescription
planSTRING{"video": "", "fps": 24.0, "grid": "H3 (17n+5)", "mode": "shots", "max_s": 4.5, "min_s": 1.0, "sensitivity": 0.5, "max_parts": 0, "max_total_s": 0.0, "start_s": 0.0, "end_s": 0.0, "bounds": [], "segs": [], "global_refs": [], "global_ref": "", "global_ref2": "", "global_prompt": "", "global_color_guide_ref1": "#00ff00", "global_color_guide_ref2": "#9b30ff", "picture_colors": [], "megapixels": 0.5, "multiple": 32, "generation_size_mode": "megapixels", "custom_width": 1024, "custom_height": 1024, "detector": "adaptive", "run": "recursive", "filters": {}, "skip_fill": "original", "cast": {}, "cast_split": false, "cast_only": false, "mask_cfg": {}, "vlm_cfg": {}, "audio_mode": "auto", "ref_details": {}, "picture_details": {}, "picture_cfg": {}, "auto_prompt": {}, "auto_prompts": {}, "between_shots": "keep", "shot_cache": "output", "shot_storage": "lossless", "unload_before_join": true}The panel writes this. JSON with the video, split settings, boundaries and the per-shot reference and prompt.
promptoptSTRINGDefault prompt for shots without their own (overrides the panel's global prompt).
maskoptMASK,IMAGEOptional mask of the WHOLE video (white = the subject), e.g. rotoscoped in another tool and loaded with Load Video (+ Convert Image to Mask). Replaces SAM 3 for every shot without its own mask video / text / points. One mask per source frame; a different count is spread over the video.
vlmoptCLIPOptional vision-language model (CLIPLoader with qwen3vl_4b / qwen3vl_8b). It looks at every shot and suggests what to segment, a description of the shot (fills {shot} in the prompt) and whether to run it. Settings in the panel's VLM card.

Outputs (13)

NameTypeDescription
shotsBFS_SHOTOne item per shot. Every node that receives this list runs once per shot; connect it to BFS Shot Unpack or BFS Shot H3 Conditioning, sample, decode, then BFS Shot Join.
countINTNumber of shots that will run.
fpsFLOATTimeline frame rate.
widthINTGeneration width.
heightINTGeneration height.
audioAUDIOThe whole soundtrack, trimmed to the planned duration.
summarySTRINGHuman-readable plan.
timelineBFS_SHOT_TIMELINEEvery shot in order, including the ones that do not run (disabled or filtered out). Connect it to BFS Shot Join so skipped shots are filled with the original video (or dropped).
refsIMAGEEvery reference picture the shots use, without repeats, <Picture 1> first (a grey image when there is none).
ref_countINTHow many different reference pictures the shots use.
second_passBOOLEANRun tab > Second pass: on or off. Use it to switch the upscale branch (latent upscale + second sampler).
upscale_widthINTSecond pass width (the generation width upscaled as set in the Run tab, on the latent grid).
upscale_heightINTSecond pass height.