Nodes/ComfyUI-AniDoc/LoadCoTracker
ComfyUI Node

LoadCoTracker

The optional node that keeps your AniDoc line art from re-inking itself every frame

By LucipherDev·Created 2 years ago·Updated about a year ago· 56
LoadCoTracker
    • cotracker
    trackingtrue
    cotracker_model
    tracker_shift_grid0
    tracker_grid_size8
    tracker_grid_query_frame0
    tracker_backward_trackingfalse
    max_points50

    If you're setting up a ComfyUI-AniDoc workflow, LoadCoTracker is the node you can skip on your first pass - and the one you'll come back to when the color drifts. It loads CoTracker, the Meta research model, so that AniDoc's sampler can track matching points across all your frames instead of only matching the reference to the first one. No tracking means your reference colors can wander between frames. With it, the same shade of red stays stuck to the same character's hat from frame 1 to frame 60.

    What it actually does

    AniDoc colorizes line art by matching a colored reference image to your lineart frames - the mechanism is basically a per-pixel correspondence problem wrapped around Stable Video Diffusion. Match the reference to one frame and you get a decent still; match it once and let every later frame guess, and identity slowly drifts. CoTracker2 tracks keypoints frame-to-frame so the matching stays anchored. It's the difference between "paint a reference once" and "re-paint the reference every frame and pray."

    The node itself has zero image inputs. It loads the tracker model, exposes a handful of tracking knobs, and hands the whole thing to AniDocSampler through the cotracker socket.

    The inputs that matter

    • tracking (default on) - the master switch. Flip it off and the node just emits a config that tells the sampler to use single-frame matching. Good for a first sanity run before you pay the VRAM cost.
    • cotracker_model - a dropdown of .pt checkpoints in models/cotracker. It's empty until you drop cotracker2.pth in there (more below).
    • max_points (10–100, default 50) - how many tracked trajectories get fed to the matcher. More points = tighter adherence, slower, and more VRAM. 50 is a sensible default; only nudge it if matching is missing fine detail.
    • tracker_grid_size (2–32, default 8) - how densely the tracker seeds its initial point grid. Smaller values track more stuff; larger values are cheaper.
    • tracker_shift_grid (0 or 1) and tracker_backward_tracking - grid offset and direction switches. If tracks look sparse or one-sided, toggling these is the first thing to try; they're the model's native tuning knobs, not ComfyUI-isms.
    • tracker_grid_query_frame (default 0) - which frame the tracking queries are anchored to.

    Output is a single ANIDOC_COTRACKER object. Wire it into the optional cotracker input on AniDocSampler. That's the only place it goes.

    Installing the checkpoint

    The python install.py step installs the cotracker Python package and the AniDoc/SVD models, but it does not fetch the model checkpoint. You need cotracker2.pth from the facebook/cotracker HuggingFace repo:

    mkdir -p ComfyUI/models/cotracker
    # put cotracker2.pth in there
    

    The node registers models/cotracker as a folder, so restart ComfyUI after adding the file and it'll show up in the dropdown.

    Gotchas

    • "Couldn't import cotracker module" - that's the pack telling you install.py never finished (or you installed via ComfyUI Manager without running it). The tracker is compiled from source inside the pack, so just run python install.py from the custom node folder and let it do its thing.
    • Empty cotracker_model dropdown = missing checkpoint, not a broken node.
    • VRAM is real. You're loading CoTracker on top of an SVD-sized diffusion pipeline, all in the same graph. On a mid-range GPU, consider skipping tracking entirely for your first successful run, then adding it back.

    One honest caveat: this whole pack is a research-model wrapper with a small community, and tracking is the finickiest part of it. Treat LoadCoTracker as an optional quality knob - get a clean 14-frame colorization working without it first, then switch it on.

    CategoryAniDoc

    Inputs (7)

    NameTypeDefaultDescription
    trackingBOOLEANtrue
    cotracker_modelCOMBO0 options:
    tracker_shift_gridCOMBO02 options: 0, 1
    tracker_grid_sizeINT82–32
    tracker_grid_query_frameINT0
    tracker_backward_trackingBOOLEANfalse
    max_pointsINT5010–100

    Outputs (1)

    NameTypeDescription
    cotrackerANIDOC_COTRACKERCoTracker