CoachBate H3 Subject Track + Crop
Track the subject, give it its own canvas
- images
- crops
- transform
- preview
- report
- canvas_w
- canvas_h
- frame_count
Start here, because this is the node the whole pack exists for
Every video model has the same embarrassing failure: a face that occupies 90 pixels of a 1080p frame comes back as a smear. Not because the model is bad, but because the latent has no budget to spend on that region. The answer has been solved in images for years - crop the region, render it big, paste it back - and ComfyUI Impact Pack's detectors and detailers made it an industry standard. Video is harder: the box has to move, and a box that jitters makes the whole thing worse than doing nothing.
CoachBate H3 Subject Track + Crop is the front half of that loop for MiniMax H3: detect the
subject on every frame, follow one subject through the clip, crop it so it fills a 768 canvas,
and emit both a constant-size crop batch for H3 and the transform the stitch needs to put the
result back. It's a fork of ComfyUI-H3-FaceRefine (MIT, credited in source) with the face-only machinery
stripped out, so "subject" can be a face, a hand, or anything a YOLO model finds.
How the tracking actually works
Pick a rule with select (largest, left_most, centre_most, closest_to_xy, …) and the
subject is chosen once per shot - on the first frame that holds it - after which continuity
follows that same subject, never re-ranked per frame. That matters: a subject who walks across frame while someone else crosses the other way is still tracked correctly.
Two things smooth the box, and they want different amounts:
smooth_window(default 21 frames, ~0.9s at 24fps) smooths the crop centre. Too much and the box lags fast movement; too little and it shivers.size_smooth_window(default 51) smooths the crop size, and genuinely wants more. Size jitter makes the crop breathe, which changes the resample factor every frame, which reads as shimmer.
smooth_method is gaussian by default (best jitter rejection), savgol if you want a push-in to
keep its shape at large windows. There's also cut_detection, off by default and correct that way
for H3, since H3 generates single-shot clips - leave it off unless you're refining an edited video
with real cuts, because with it on each shot picks its own subject again by select, which is
exactly what you don't want mid-take.
The inputs that matter, and the crop geometry
Required: images, detector, confidence, crop_factor, canvas_width, canvas_height,
canvas_mode, smooth_window, size_smooth_window, smooth_method, size_mode.
Three of those decide the result:
crop_factor(default 2.5) - crop side as a multiple of the detected box. 2.5 puts the subject at roughly 40% of the crop. Bigger means more context so the seam lands in hair or background, at the cost of magnification. 2.0–3.0 is the useful band; 1.5 is available and more aggressive.canvas_mode/canvas_width/canvas_height- the resolution H3 generates the crops at. 768 is H3's native short edge and the default.manualuses your numbers exactly;auto_no_downscalesizes the canvas from the largest crop so nothing is ever downscaled (expensive on clips with close-ups);auto_capped_768clamps that to 768. Cost scales with area.size_modedecides whether the crop keeps a constant subject fraction per frame (per_frame, right for push-ins) or one size for the whole clip (max_of_clip, static shots only).
Optional but worth knowing: select_index (which subject in the ranking to follow - feed it from
CoachBate H3 Loop Subject), select_override (a string name that overrides select), and
crop_size_from (height or longest_side, the latter so a subject lying sideways isn't cut off).
Outputs: crops (the IMAGE batch), transform (the geometry the inject/denoise/stitch nodes all
need - don't try to reconstruct it), preview (the tracked box drawn on each frame; green means
detected, red means interpolated), report, canvas_w,
canvas_h, frame_count.
Install
ComfyUI Manager → search "CoachBate", or:
cd ComfyUI/custom_nodes
git clone https://github.com/CoachBate/ComfyUI-CoachBate.git
Restart ComfyUI. You also need ultralytics installed - the node imports it to run the detector
- and detector weights in
models/ultralytics/segm/ormodels/ultralytics/bbox/, registered by ComfyUI-Impact-Subpack (the pack looks there directly if it isn't installed). A bbox model tracks fine but pastes as an ellipse; use a-segmodel if the seam matters.
Where people get burned
- The detector isn't listed - it's not in
models/ultralytics/segmorbbox, or ultralytics isn't installed. The error names the folders it searched. - The box jumps to the wrong person - your
selectrule picks the wrong subject at the lock-on frame. Setselect_overrideorselect_indexexplicitly rather than tuning windows. - A hard cut lands on someone else -
cut_detection = nonedoing its job: everyone is renumbered at a cut, so the tracker picks the nearest subject to the last position, whoever that is. Turn cut detection on if the clip is an edit rather than a take.
Inputs (20)
| Name | Type | Default | Description |
|---|---|---|---|
| images | IMAGE | — | |
| detector | COMBO | 1 options: face_yolov8m.pt | |
| confidence | FLOAT | 0.350.05–0.95 | — |
| crop_factor | FLOAT | 2.51.2–8 | Crop side as a multiple of the detected box (see crop_size_from). 2.5 puts the subject at ~40% of the crop, comfortably inside H3's good regime. Bigger = more context so the seam lands in hair/background, but less magnification. 2.0-3.0 is the useful range. |
| canvas_width | INT | 768128–1344 | Resolution H3 generates at. 768 is H3's native short edge and the default here; 512 is cheaper. In manual mode this is used exactly as typed, high or low. Ignored when canvas_mode is not 'manual', where the canvas comes from the crop instead and never falls below 512x512. Cost scales with area: 768 is 2.25x the latent tokens of 512. |
| canvas_height | INT | 768128–1344 | — |
| canvas_mode | COMBO | manual | manual: use canvas_width/height as given. auto_no_downscale: size the canvas from the LARGEST crop in the video so no frame is ever downscaled (magnification never drops below 1.0x). Can get expensive on videos that include close-ups. auto_capped_768: same, but clamped to 768 - H3's native short edge and a sane VRAM ceiling. Both auto modes clamp UP to a minimum of 512x512, whatever the crop, so a small subject in a low-resolution clip is still magnified. manual is not clamped - it uses canvas_width and canvas_height exactly as typed. |
| smooth_window | INT | 211–201 | Frames of smoothing on the crop CENTRE. 21 at 24fps is ~0.9s. Raise if the box still shivers; lower if it lags behind fast movement. |
| size_smooth_window | INT | 511–201 | Frames of smoothing on the crop SIZE. Wants MORE than the centre: size jitter makes the crop breathe, which changes the resample factor every frame and reads as shimmer. Real zoom moves are slow, so heavy smoothing here costs nothing. |
| smooth_method | COMBO | gaussian | gaussian: best jitter rejection. savgol: preserves the shape of a push-in better at large windows. moving_average: the old boxcar, leaves residual jitter. |
| size_mode | COMBO | per_frame | per_frame: constant subject-fraction in every crop (correct for push-ins). max_of_clip: one size for the whole video, only useful when the shot is genuinely static. |
| selectopt | COMBO | largest | How the subject is chosen. It is chosen ONCE per shot, on the first frame that holds it, and continuity follows that same subject from there - it is NOT re-ranked each frame. A subject who walks from the left of frame to the right while someone else crosses the other way is still tracked correctly. largest / smallest: biggest or smallest subject by height. left_most / right_most / top_most / bottom_most: by the CENTRE of the subject box. centre_most: nearest the centre of the frame. closest_to_xy: nearest the X, Y you give, measured on frame_index. detector_score: the detection the detector is most confident about. AT A HARD CUT a rank means nothing across the join - everyone is renumbered. With cut_detection ON, each shot chooses again by this rule, so a cut can land on a different person. With it OFF the video counts as one shot and continuity runs straight through a real cut onto whichever subject is nearest the last position, which may be any of them. |
| cut_detectionopt | COMBO | none | Hard-cut detection, so the crop is not smoothed ACROSS a cut. none: off. Correct for single-shot videos, which is what H3 generates. auto: split at the cuts and treat each shot as its own window for the smoothing, interpolation and composite fade. The canvas is still sized once for the whole video. |
| cut_thresholdopt | FLOAT | 3.000.5–20 | How far a frame has to stand out from its NEIGHBOURS to count as a cut. 3.0 is PySceneDetect's adaptive default. Lower catches more and risks splitting a continuous shot, which costs smoothing on both sides. Higher takes only unambiguous cuts. Only used when cut detection is on. The report says how many cuts were found and where, so tune it from that. |
| select_indexopt | INT | 00–63 | Which subject in that ranking to track. 0 is the first, 1 the second, and so on. The subject is locked on the first frame that actually contains this index, so a video that opens on one subject and only later shows several still finds subject 1. Frames before that lock-on are interpolated from it and faded out of the composite, the same as a detection dropout; the report counts them. Clamped, with a warning in the report, if the video never shows that many subjects at once. |
| crop_size_fromopt | COMBO | longest_side | What the crop is sized from. 'height' suits a roughly square subject (a face). 'longest_side' uses the larger of the box's smoothed height and width, so a subject lying sideways is not cut off at the sides of the crop. |
| select_overrideopt | STRING | When not empty, replaces `select`. Lets a loop drive the subject choice per round (CoachBate H3 Loop Subject). | |
| Xopt | INT | 00–16384 | Only used by select=closest_to_xy. Horizontal position in PIXELS of the source video frame, measured from the TOP-LEFT corner, increasing to the right. On a 960x544 clip, 0 is the left edge, 960 the right, 480 the middle. The point does not have to sit on a subject: the nearest subject CENTRE wins, however far away it is. |
| Yopt | INT | 00–16384 | Only used by select=closest_to_xy. Vertical position in PIXELS of the source video frame, measured from the TOP-LEFT corner, increasing DOWNWARD. On a 960x544 clip, 0 is the top edge, 544 the bottom, 272 the middle. |
| frame_indexopt | INT | 00–999999 | Only used by select=closest_to_xy. The frame the X, Y measurement is taken on, counting from 0 for the FIRST frame. It counts the frames this node loaded, so with skip_first_frames or select_every_nth set it counts from the first frame kept, not the first frame of the file. The subject found there is followed forwards and backwards through its shot, so pick a frame where the subject is clearly visible rather than one the detector may miss them on. |
Outputs (7)
| Name | Type | Description |
|---|---|---|
| crops | IMAGE | — |
| transform | H3FACEXFORM | — |
| preview | IMAGE | — |
| report | STRING | — |
| canvas_w | INT | — |
| canvas_h | INT | — |
| frame_count | INT | — |