OpenPose BoundingBox Tracker

If you've ever tried to crop "just that one person" out of every frame of a video, you know the pain: people move, swap positions, and a fixed crop is useless. OpenPose BoundingBox Tracker (class OpenPoseTrackerNode) is the free lunch version of that problem. It takes pose keypoints you've almost certainly already computed and converts them into per-person bounding boxes that stay consistently labeled frame to frame - no object detector, no extra model download.

It ships in logtd's small ComfyUI-TrackingNodes pack, alongside a YOLO-based sibling. That sibling makes you download a YOLOv8 model into models/yolov8; this node doesn't. It's the one you reach for when your workflow already has a pose pass in it and you want tracking as a side effect rather than a whole extra pipeline.

How it works

You feed it two things: the video frames as an IMAGE batch, and a POSE_KEYPOINT stream - the per-frame skeleton data from a pose preprocessor (the DWPose/OpenPose preprocessors in ComfyUI ControlNet Auxiliary Preprocessors, which this KB calls functionally required for any serious ControlNet workflow anyway). Both inputs are required; there are no optional inputs and no parameters at all. Zero knobs.

Under the hood it's a simple but clever trick. For each person in each frame it takes the four keypoint groups OpenPose produces - body, face, left hand, right hand - and computes a bounding box by min/maxing the keypoint coordinates. Then it pads the box a bit, with sensible defaults per part: face and hands get a small buffer (20px, or 30px vertically on the face), body gets none. Those boxes become detections with class IDs body (0), face (1), hand_left (2), hand_right (3), and get fed into supervision's ByteTrack for consistent IDs across frames.

The output is two things: an IMAGE of the frames with boxes and labels drawn on (you'll see things like #0.body, #1.face), and a TRACKING structure - a dict keyed by class and tracker ID, holding a per-frame box [x0, y0, x1, y1, W, H]. That dict is the useful bit. Wire it into downstream nodes that want "this instance, in these frames" - cropping a specific person for a detail pass, or feeding instance-level data into an AnimateDiff pipeline.

Installing it

The standard path: ComfyUI Manager, search "Tracking Nodes", install, restart. Or by hand:

cd ComfyUI/custom_nodes
git clone https://github.com/logtd/ComfyUI-TrackingNodes
cd ComfyUI-TrackingNodes
pip install -r requirements.txt

Then restart ComfyUI. The requirements install supervision, ultralytics, timm, scikit-image, and numpy. The heavy hitters (ultralytics, timm) are really for the YOLO sibling node - this one just needs supervision (and torch). No model download required here, which is a genuinely nice change from most tracking tooling.

Where people get burned

The POSE_KEYPOINT input is picky: it only connects to nodes that output that exact type, i.e. the DWPose/OpenPose preprocessors from ControlNet Aux. If you're feeding it a plain pose image, it won't plug in. And those keypoints have to be frame-aligned with the images batch - same length, same order - or your boxes will drift off the people they belong to.

Remember the boxes are derived from keypoints, not a detector. If the pose estimator drops a keypoint group for a frame (and any of them falls under the hard-coded confidence threshold of 0.1), the box silently disappears for that frame - the TRACKING dict keeps None gaps where a tracked instance wasn't seen, so anything downstream should handle missing frames. In practice that means slightly flickery boxes on fast motion or heavy occlusion. It's a tracking assistant, not magic.

Finally, if ComfyUI fails to start after installing, it's almost always supervision missing - run the pip install and retry. It's a small pack, and for "track a person across a video without dragging in a detector" it does exactly one thing, well.