Depth-FM

If you've ever wanted a depth map without waiting a Marigold-style eternity for it, this is the node. Depth-FM is Kijai's ComfyUI wrapper around the flow-matching depth estimator from the CompVis group at LMU Munich - the same lab that gave us Stable Diffusion. The pitch is right in the name and it's not marketing fluff: the model is designed to produce a usable depth map in one inference step, and it genuinely holds up.

Depth maps are the workhorse behind depth ControlNet conditioning, 3D parallax "living photo" effects, and stereo/VR splits. The classic tension is speed vs. quality: discriminative models like Depth Anything v2 are fast but edge-limited, while generative ones like Marigold look great and take forever. Depth-FM sits in the middle and leans hard toward fast.

How it works

DepthFM is a generative model, but it doesn't denoise from random noise the way Marigold does. Instead it's built on an SD 2.1 UNet and frames depth estimation as flow matching: it directly maps the input image's latent representation to a depth latent by solving a short ODE. The input image is the starting point; the solver just walks it to the depth side of latent space. That's why the node asks you to plug in a VAE - it uses yours to encode the image and decode the depth latent back to pixels. Any SD1.5/2.1 VAE works, which is nice because you almost certainly already have one loaded.

Inputs and outputs that matter

• images - what you want depth for. Wire in anything producing an IMAGE.
• vae - any SD1.5/2.1 VAE.
• depthfm_model - a dropdown of everything in your models/checkpoints folder. Yes, checkpoints - that's where the pruned weights go, not a dedicated depth folder.
• steps (default 4) - the ODE solver steps. The paper's own claim is that 1–2 steps already look good; more steps buy smoother maps, not fundamentally better ones. Start at 4 and go down to 2 if you're in a hurry.
• ensemble_size (default 1) - runs the solve multiple times and averages. It genuinely helps quality, but the model asserts the input batch size is 1 when you use it, so don't feed it a batch of frames and crank this up.
• invert (default true) - keeps the white-is-near / black-is-far convention that ControlNet expects. If your generated image looks depth-inverted, flip this.
• per_batch (default 16) - how many images go through the model at once; lower it if you're hitting VRAM limits on video frames.
• dtype (default fp16) - fp32 if you ever see artifacts, bf16 if you're on newer hardware and want it leaner.

The single images output is a normalized 3-channel depth map (values clamped 0–1). It's relative depth, not metric - which is exactly what you want for ControlNet, and why it feeds straight into a depth ControlNet or a parallax/3D node.

Installing

ComfyUI Manager, search "ComfyUI-depth-fm", done. Manual route:

cd ComfyUI/custom_nodes
git clone https://github.com/kijai/ComfyUI-depth-fm
cd ComfyUI-depth-fm
pip install -r requirements.txt

That requirements file is small - numpy, einops, omegaconf, torchdiffeq>=0.2.3 - so no heavy native build surprises. The model itself is the real download: grab the pruned safetensors from Kijai's Kijai/depth-fm-pruned on HuggingFace (Manager can fetch them too) and drop them in ComfyUI/models/checkpoints. The original depthfm-v1.ckpt from ommer-lab works as well.

Gotchas

The biggest trap is the model location: it reads from models/checkpoints, so if you habitually file depth models elsewhere, the dropdown will look empty. Also remember the ensemble/batch-1 rule, and that internally the node resizes your image to a multiple of 64 and back - fine for typical inputs, just don't expect pixel-perfect preservation of an odd-sized frame. In the big 8-model r/StableDiffusion comparison on 24GB VRAM, the community leaned toward Lotus and Depth Anything for ControlNet preprocessing, with DepthFM holding its own at FP32/10 steps/ensemble 9. Where DepthFM wins is the workflow where you need depth now - one or two steps is genuinely hard to beat.