AP Get RAFT Optical Flow

AP Get RAFT Optical Flow is the engine room of this whole pack. Everything else - the apply nodes, the occlusion mask, the compositor - is downstream of the flow_data it produces. You hand it frame A and frame B, it figures out where every pixel moved between them, and it hands you back a field of motion vectors. That's the entire trick of temporal consistency work: warp the previous frame forward along the motion field instead of letting the diffusion model redraw it from scratch and wander off.

It's a real answer to a real problem. Optical flow nodes in ComfyUI have a history of being outdated, breaking against newer ComfyUI, or only working in toy cases - the author of this pack built it specifically because the existing ones "usage really sucks" and didn't support proper batch or loop graphs. If you've bounced off other flow packs, this is the "somebody actually uses this in production" rewrite.

What actually happens inside

The flow is computed by RAFT - the Recurrent All-Pairs Field Transforms model - pulled straight from torchvision.models.optical_flow. That's worth knowing for two reasons. First, there's no separate model file to download and place: the raft_large or raft_small weights auto-download into your torch cache on the first run. Second, it means the pack's one real dependency is a compatible torchvision (torchvision>=0.15 in requirements.txt). If your torch and torchvision builds don't match, nothing works and the error message tells you exactly that.

The node computes flow in both directions by default (compute_backward=true). The forward ab field is what you usually warp with; the backward ba field is what the occlusion-mask node uses to figure out which regions are covered and which are newly revealed. You can disable it to halve the compute, but then APFlowOcclusionMask has to fake the backward pass by negating the forward one, which is less accurate.

The inputs that actually matter

Most of the widget soup on this node is about managing VRAM, not quality. The ones you'll touch:

• model_size - large for real quality, small for speed. On faces/eyes the author recommends large, and I agree: RAFT-small smears fine motion.
• max_side - the flow is computed at a capped resolution (default 1024, up to 4096) then resized back to your frame. This is your main speed/VRAM lever. Bigger cap, better subpixel accuracy, more memory.
• model_residency - unload_after_use (default) keeps the RAFT model from squatting on VRAM between calls; cache_on_gpu keeps it hot for repeated runs. For loop-heavy pipelines, caching on GPU is a real speedup at the cost of a permanent chunk of VRAM.
• flow_offload - cpu_ram holds the flow tensors in system RAM (the default); disk_storage writes them to a .pt file and hands you a lightweight handle. Flows are big - two dense float fields per frame pair - and this is how you keep them from eating your VRAM and RAM on long sequences.
• compute_backward and use_fp16 - quality dials. fp16 on is faster; off is more precise. If you see ghosting on high-contrast edges, try use_fp16=false before you blame the warp.

compute_mode (sequential vs batched) is the other VRAM lever: sequential processes a batch one pair at a time, batched pushes everything through at once and is faster but greedier.

Wiring it up

Outputs are flow_visualization (a color image of the motion, handy to eyeball) and flow_data - an AP_FLOW object you feed straight into APApplyRAFTOpticalFlow, APFlowOcclusionMask, or APFlowComposite. The custom AP_FLOW type is the pack's contract: it carries both directions plus height/width, and it's what keeps every node in the pack batch-safe.

One tip from the README: if motion looks reversed in the final warp, you've got the direction backwards - flip flow_direction to ba on the apply node rather than re-computing.

Installing it

ComfyUI Manager is the easy path (search "AP_OpticalFlow"). Manually:

cd ComfyUI/custom_nodes
git clone https://github.com/adampolczynski/ComfyUI_AP_OpticalFlow
python -m pip install -r custom_nodes/ComfyUI_AP_OpticalFlow/requirements.txt

Then restart ComfyUI. First run downloads the RAFT weights, so it'll take a few extra seconds.

Troubleshooting

• "torchvision RAFT import failed" - your torch/torchvision pair is broken or lacks the optical flow submodule. Fix torchvision to match torch (a venv reinstall with the right CUDA build usually does it). This is the most common install failure and it's an environment problem, not this pack.
• CUDNN_STATUS_NOT_SUPPORTED - some CUDA/cuDNN combos choke on RAFT's internals. The pack retries in float32 with cuDNN disabled automatically, so usually it just works; if you keep hitting it, use_fp16=false gives it fewer chances to trip.
• VRAM stuck high after runs - run once with model_residency=unload_after_use and clear_cached_models_first=true to flush old cached models.

Skip the flow-computation rabbit hole if you don't need it: APSaveOpticalFlow/APLoadOpticalFlow let you compute flow once, save it, and reuse it on later runs so you're not paying RAFT on every iteration.