ComfyUI Node

AP Flow Composite

The node that decides who wins — blending warped and original frames without ghosting

By adampolczynski·Created 6 months ago·Updated 6 months ago· 1
AP Flow Composite
  • original_images
  • warped_images
  • valid_mask
  • occlusion_mask
  • effect_mask
  • composited_images
  • composite_alpha
alpha_modeflow_confidence
mask_threshold0.000
use_difference_gatefalse
difference_threshold0.010
difference_feather0
blend_strength1.00
feather0
invert_occlusionfalse

You've warped a frame with flow and computed an occlusion mask. Now you have two versions of every pixel - the original and the warped - and the composite node is the referee that decides, per pixel, which one shows. It's the last step of the pack's core temporal chain (APGetRAFTOpticalFlowAPFlowOcclusionMaskAPApplyRAFTOpticalFlowMasked → here), and it's where a flow workflow either looks like clean video or falls apart into ghosting.

The idea is simple: warped pixels are what you want where the warp is trustworthy; original pixels are what you keep where it isn't. The art is deciding "trustworthy" - and this node gives you three levers to decide it.

The mechanism

APFlowComposite takes original_images, warped_images, valid_mask (from the apply node), and occlusion_mask (from the occlusion node), and builds an alpha per pixel. The base alpha is valid_mask × (1 − occlusion_mask) - trust the warp where the flow was consistent and not occluded. Then three dials refine that alpha:

  • alpha_mode - flow_confidence (default), flow_confidence_x_mask, or mask_only. The first uses pure flow validity; the second multiplies by an optional effect_mask; mask_only makes the effect mask the whole story. For the README's recommended anti-blur setup you want flow_confidence_x_mask, with the apply node's warped_mask wired into effect_mask.
  • use_difference_gate - the smartest feature here. When enabled, the node computes the per-pixel difference between warped and original, and only lets the alpha through where the difference exceeds difference_threshold (default 0.01). The point: if the warp didn't actually change a pixel, the composite shouldn't touch it either. This is what keeps background that shouldn't have moved from being repainted. The README recommends a low threshold like 0.005–0.02 for the anti-ghost setup.
  • mask_threshold - binarizes the effect_mask above a value, for when you want a hard mask rather than a soft one.
  • blend_strength, feather, invert_occlusion - blend amount (1.0 = full alpha, under 1.0 = partial), alpha edge feathering, and a polarity flip for the occlusion mask if yours is inverted relative to expectations.

Outputs

composited_images is your finished blend. composite_alpha is the alpha that produced it - a MASK you can preview or reuse, which is handy for debugging "why is my background getting touched" (answer: the alpha is too high there; check your difference gate).

The wiring that actually gets results

The README's "professional anti-blur compositing" is worth quoting because it's the difference between a soft blurry result and a crisp one:

  1. Connect warped_mask from APApplyRAFTOpticalFlowMasked to this node's effect_mask.
  2. Set alpha_mode = flow_confidence_x_mask.
  3. Enable use_difference_gate = true with a low difference_threshold (0.005–0.02).

That combination confines the composite to genuinely-warped regions and leaves untouched background alone. Skip it and you'll get the classic symptom: background areas subtly blurring every frame because the composite is blending warped-but-identical pixels with the original at some alpha.

Installing it

Standard pack install: ComfyUI Manager (search "AP_OpticalFlow"), or manually:

cd ComfyUI/custom_nodes
git clone https://github.com/adampolczynski/ComfyUI_AP_OpticalFlow
python -m pip install -r custom_nodes/ComfyUI_AP_OpticalFlow/requirements.txt

Restart. Dependency is torchvision>=0.15 only.

The one thing to check first

If your composite output looks like the warped frame is leaking in where it shouldn't, your occlusion mask polarity is probably flipped - try invert_occlusion. If it looks like nothing is being composited, your effect_mask is too tight or use_difference_gate is swallowing everything at a too-high difference_threshold. Both are quick dial-turns, not node-bugs.

CategoryAP_OpticalFlow

Inputs (13)

NameTypeDefaultDescription
original_imagesIMAGE
warped_imagesIMAGE
valid_maskMASK
occlusion_maskMASK
alpha_modeCOMBOflow_confidence3 options: flow_confidence, flow_confidence_x_mask, mask_only
mask_thresholdFLOAT0.0000–1
use_difference_gateBOOLEANfalse
difference_thresholdFLOAT0.0100–1
difference_featherINT00–64
blend_strengthFLOAT1.000–2
featherINT00–64
invert_occlusionBOOLEANfalse
effect_maskoptMASK

Outputs (2)

NameTypeDescription
composited_imagesIMAGE
composite_alphaMASK