CV Rapid Track (Sequence)

Tracks a known 3D mesh through an IMAGE batch with cv2.rapid (Harris & Stennett 1990): for each frame it samples control points along the model's projected silhouette, searches perpendicular to the contour for the strongest image gradient, and refines the pose by PnP. Every frame starts from the previous frame's result, so it is a TRACKER, not a detector - it needs an initial pose that is already close (tens of pixels) and small motion between frames. Feed it a mesh from 'CV Mesh From 3D Model' via 'CV Mesh Split Long Edges': a coarse mesh makes cv2.rapid's control points wrong by tens of pixels and the track diverges (see that node's description). Failure tolerant: if the model leaves the frame or its silhouette collapses, cv2 raises internally; the node stops advancing, reports tracked=false and repeats the last good pose for the remaining frames instead of halting the workflow.

By bmad4ever·Created 3 months ago·Updated 2 days ago· 0

frames
pts3d
tris
K
rvec
tvec

rvecs
tvecs
ratios
rmsds
rvec
tvec
tracked

◄num_control_points128►

◄search_length12►

◄iterations2►

◄scale1.00►

Categoryimage/CV/contrib

Inputs (10)

Name	Type	Default	Description
frames	IMAGE		The clip to track through, as one IMAGE batch (a video loader, or core 'Batch Images'). Frames are processed in order and the pose carries over, so the batch must BE the sequence.
pts3d	NPARRAY		Nx3 model vertices, in the same units as tvec.
tris	NPARRAY		Mx3 int triangle indices. Used for backface culling and the silhouette, so the winding matters.
K	NPARRAY		3x3 camera matrix of the FRAMES as given ('CV Camera Matrix' or a calibration). Lens distortion is not modelled by cv2.rapid - undistort the frames first if the lens is wide.
rvec	NPARRAY		Initial rotation (Rodrigues 3x1) for the FIRST frame. Typically from solvePnP on a marker, or the pose the object was placed at.
tvec	NPARRAY		Initial translation (3x1) for the first frame, in the mesh's units.
num_control_points	INT	1288–2048	How many points to sample along the silhouette per iteration. More points average out bad matches but cost linearly. 128 is a good default; below ~32 a few wrong correspondences can swing the pose.
search_length	INT	121–256	Half-length of the search line, in pixels of the SCALED frame - the tracker looks this far either side of the predicted contour. It sets the capture range, but NOT for free: cv2.rapid takes the strongest gradient anywhere on the line, so a long line lets texture and background edges outvote the silhouette. Measured on a textured vehicle over clutter, raising it from 12 to 96 pinned the error at ~46 px no matter how good the start was. Prefer raising capture range with 'scale' instead.
iterations	INT	21–20	Refinement passes per frame. 2 is usually the sweet spot; 1 lags behind the motion, and many passes on a static frame drift rather than settle (cv2.rapid's control points are re-extracted each pass, so it is not a converging solver).
scaleopt	FLOAT	1.000.05–1	Downscale the frames before tracking (the camera matrix is scaled to match, so the pose stays in the original units). This is the CAPTURE-RANGE knob, and the one to reach for before search_length: the range it buys is search_length/scale in original pixels, it costs LESS than the equivalent search_length (the mask rasterization shrinks quadratically), and the INTER_AREA decimation suppresses the texture edges that mislead a long search line rather than adding more of them. It is NOT free accuracy: on a clip that starts on the object and only needs precision, dropping to 0.5 took the measured error from 2.8 px to 18 px. Leave it at 1.0 unless the pose you start from, or the motion between frames, is genuinely outside search_length.

Outputs (7)

Name	Type	Description
rvecs	NPARRAY	Bx3x1 float64 stack: the rotation for each frame of the batch, in order. Slice one out with 'CV Index Batch' to project or render at that frame's pose.
tvecs	NPARRAY	Bx3x1 float64 stack of translations, aligned with rvecs.
ratios	NPARRAY	Bx1 float32: per frame, the fraction of search lines that produced a usable correspondence (cv2.rapid's return value). A RELATIVE health signal - a sudden drop means the model came unstuck. Not an accuracy measure.
rmsds	NPARRAY	Bx1 float32: per frame, cv2.rapid's own 2D reprojection difference. Low rmsd with a wrong pose is possible (the tracker can fit a self-consistent but shifted silhouette), so read it together with ratios rather than alone.
rvec	NPARRAY	Rotation after the LAST tracked frame - the starting pose for a following clip.
tvec	NPARRAY	Translation after the last tracked frame.
tracked	BOOLEAN	False if cv2.rapid failed on any frame (the model left the view, or its silhouette produced no usable vertices). The pose outputs are still valid up to that frame and repeat afterwards. Gate downstream compositing on it with an 'If/Else Switch'.