Nodes/comfyui-tensorop/FalDifferentialDiffusion
ComfyUI Node

FalDifferentialDiffusion

Soft inpainting with a volume knob, hosted on fal.ai

By un-seen·Created 2 years ago·Updated 2 years ago· 46
FalDifferentialDiffusion
  • foreground_image
  • depth_image
  • IMAGE
foreground_prompt
background_prompt
strength1.00
steps14

Regular inpainting gives you an on/off switch: masked pixels get redrawn, everything else is frozen. Differential Diffusion replaces that switch with a volume knob - a grayscale "change map" where white means change this a lot and black means leave it alone, with every shade of gray in between. FalDifferentialDiffusion runs that on fal.ai's hosted FLUX differential-diffusion model and hands you the edited image back. It's the gentlest way to composite something into a scene without the seam that hard masks leave behind.

How it works

Give it a foreground_image and a depth_image - though the second name is a bit of a lie, in the best way. In the source, that input gets wired straight into the API's change_map_image_url. So it's not literal depth geometry you're feeding it; it's the per-pixel change-strength map. Feed it a proper depth map and you get changes that respect spatial layout. Feed it a soft mask and you get soft inpainting. Feed it a flat gray image and the whole thing changes uniformly. That one line of the code explains half the confusion people have with this node.

The node uploads both images to fal, then submits with a prompt assembled as foreground_prompt, background_prompt, 8k, unreal engine 5, hightly detailed, intricate detailed. That trailing tail is hardcoded in the source - "hightly" typo included - so skip adding your own "8k, highly detailed" to the fields. You get two real dials:

  • strength - 0.01 to 3, default 1, how far the edits go.
  • steps - 1 to 32, default 14, the denoising budget.

One IMAGE comes back. Wire it into whatever's next.

Where it fits

This is a compositing tool more than a generation tool. The classic move: you've got a subject, a background you want to keep, and you want the two to actually belong together. The model re-renders the surroundings to match, and the change map decides how much of the old scene survives. It's the kind of thing you'd otherwise build with a ControlNet inpainting stack, minus all the local plumbing.

The practical bits

Cloud node, so: a fal.ai key is required, you pay per run, and your images leave your machine. The change map is everything - feed it a bad one and you'll wonder why nothing moved, or why everything did. And the pack's config trap applies here too: config.py overwrites FAL_KEY with an empty string on load, so paste your key into config.py rather than trusting a shell export.

Install

In ComfyUI Manager, search comfyui-tensorops. Or:

cd ComfyUI/custom_nodes
git clone https://github.com/un-seen/comfyui-tensorops

Restart and you're set. No model files, no heavy dependencies beyond fal-client - the heavy lifting is all happening in fal's datacenter, and the only thing standing between you and that is a working key.

Categoryexternal_tooling

Inputs (6)

NameTypeDefaultDescription
foreground_imageIMAGE
depth_imageIMAGE
foreground_promptSTRING
background_promptSTRING
strengthFLOAT1.000.01–3
stepsINT141–32

Outputs (1)

NameTypeDescription
IMAGEIMAGE