TF Tokens From Mask

TF Level Canvas renders a trajectory level as something you can paint on, core Painter gives you the brush, and this node is what happens when the stroke comes back: TF Tokens From Mask downsamples the painted mask to the 16×16 token grid and hands you a tokens selection the edit nodes actually understand.

The mechanism is the part worth respecting. A token counts as selected only once a threshold fraction of its pixels is painted - coverage, default 0.35 - rather than being selected by any touch. That's a deliberate design decision, and it saves you from a nasty silent failure: with whole-footprint selection, a stroke that clips the corner of a token would overwrite that token's entire feature vector. The partial-coverage rule means a sloppy stroke selects whole tokens or nothing, and the boundary between what you meant and what you grazed stays honest. Wire regions in from a TF Region Map and there's a second threshold, region_overlap (default 0.3): a region joins the selection once that fraction of its tokens is painted, so a rough stroke snaps to whole semantic regions instead of being taken literally. That's the "brush the whole cluster" behavior of the paper's editing environment, exposed as one wire.

Inputs levels (optional) and regions (optional) have specific roles: levels is only read for its token-grid size and defaults to 16×16, which every released checkpoint uses, and regions is what switches snapping on. Both outputs matter - tokens for the edit, info to see what the mask resolved to. If the selection looks wrong, the info string and the what-was-selected preview (TF Level Canvas with a highlight wire, or TF Tokens Preview) will show you before you waste a resume on a mis-aimed edit.

One behavior surprises people and it's deliberate: with nothing painted yet, the node stops the graph rather than failing. No error, no red - it just blocks execution downstream. That's because the first run of a painting workflow exists to produce the canvas for you to paint on; failing would spam errors on a run that's working exactly as intended. Queue it, look at the canvas, paint, re-run.

The choice between this node and TF Tokens From Coords is a workflow preference, not a capability difference. Mask-painting traces organic region boundaries; coordinates give you a selection that's text - quotable in a writeup and reproducible by typing. For a serious experiment you often want coordinates (there's a reason they're the paper-friendly form); for grabbing "that lump that looks like the background" by eye, painting wins. TF Tokens Combine can union or subtract the two if you want both, e.g. a region minus a sub-region.

Install

Pack-standard: Manager → search Trajectory Forcing, or git clone https://github.com/korayulusan/ComfyUI-TrajectoryForcing into custom_nodes/. The whole pack runs JAX inside ComfyUI and wants its own Python 3.11 environment on CUDA 12 (torch 2.8+); requirements.txt is empty on purpose so Manager can't rewrite your torch, and install.py declines with a reason when your setup can't host it. Declining is normal - use the README's env/setup.sh. And remember the Painter you paint with needs ComfyUI Node 2.0 rendering enabled (Settings → search "Node 2.0") or it refuses to run:

cd ComfyUI/custom_nodes/ComfyUI-TrajectoryForcing
python -m tf_nodes.doctor