Upper Body Tracking From PoseKps (InstanceDiffusion)

This is a bridge node for InstanceDiffusion, and if you're not doing InstanceDiffusion work you can safely ignore it. What it does: take detected pose keypoints and convert each body part - head, neck, shoulders, torso, arms, forearms - into a tracking box of a size you specify. InstanceDiffusion uses per-instance region tracking to control where individual things appear across a generation (or across animation frames), and this node produces the tracking data for a person's upper body from an OpenPose-style pose. It's the plumbing that turns "here's a detected pose" into "here are the tracked regions InstanceDiffusion should place each body part in."

How it works

Pose preprocessors output POSE_KEYPOINT data - the joint positions of detected people. This node reads those keypoints and, for each named upper-body part, draws a box centered on the relevant joints at the width and height you configure. The collection of boxes becomes a TRACKING output - the region-tracking structure InstanceDiffusion consumes - plus a prompt string that carries the per-region labeling. Feed both into the InstanceDiffusion nodes and the model conditions each region on its corresponding part, giving you spatial control over the body's layout beyond what a plain pose skeleton provides.

The "upper body" scope is deliberate: the configurable parts are head, neck, shoulder, torso, and left/right arm and forearm. It's built for portrait and upper-body framing, not full-figure poses.

The inputs and outputs

The two connections that matter are pose_kps (POSE_KEYPOINT, from a pose preprocessor) and the outputs: tracking (TRACKING, into InstanceDiffusion) and prompt (STRING). The rest are text-field size settings:

• id_include - which detected person IDs to include (leave empty to take them all). Useful when a frame has multiple people and you only want to track one.
• The per-part ..._width_height strings (Head_width_height default 256, 256, Torso_width_height default 350, 450, the arms at 128, 256, and so on) - each sets the box size for that part as "width, height". The defaults are reasonable for a standard upper-body framing; adjust a part's numbers to grow or shrink its tracked region. Leaving one blank skips it.

Installing it

ComfyUI Manager: search ComfyUI's ControlNet Auxiliary Preprocessors, install, restart. Or manually:

cd ComfyUI/custom_nodes
git clone https://github.com/Fannovel16/comfyui_controlnet_aux
pip install -r requirements.txt

Restart ComfyUI. No model download - it's a keypoint-to-boxes converter. You'll need a pose preprocessor upstream and the InstanceDiffusion nodes downstream for it to be useful.

Where people get burned

The honest headline: this node is only meaningful inside an InstanceDiffusion workflow. Its TRACKING output isn't a ControlNet condition - it doesn't plug into a normal ControlNet Apply node - so if you're not running InstanceDiffusion, there's nothing here for you. Second, the format of those size fields matters: they're strings like "256, 256", width then height, so keep the comma format. Third, it needs real upper-body keypoints, so make sure your pose preprocessor is actually detecting the person cleanly; a bad detection makes bad boxes. It's a specialized tool for a specialized pipeline - powerful there, irrelevant everywhere else.