Metric3D Depth Map

Metric3D is the depth preprocessor that actually cares about real-world scale. Most depth estimators - MiDaS, Depth Anything - give you relative depth: this is closer than that, but no idea by how much. Metric3D estimates metric depth, using the camera's focal length to ground the numbers in something closer to actual distance. For plain ControlNet conditioning that precision is honestly overkill, and often not even what you want. But it's a genuinely strong depth model, it showed up in the big community depth shootout (the ViT-Giant2 variant), and if you're doing anything geometry-aware it earns its place.

How it works and the honest caveat

Metric3D predicts depth conditioned on camera intrinsics, which is why it can output metric values instead of just an ordering. You hand it an image and, optionally, the camera's focal lengths, and it returns a depth map where brightness encodes distance. Feed that into a depth ControlNet (control_v11f1p_sd15_depth and friends) and the model generates a new image respecting that spatial layout.

Here's the thing worth knowing before you commit to it: for ControlNet work specifically, the community's long-standing lesson is that metric-accurate depth isn't automatically better. Sharp edges and clear local contrast are what a depth ControlNet follows well, and models optimized for real-world metric accuracy can come out softer. The canonical "use MiDaS over Zoe" result was exactly this - metric precision at the cost of edge crispness. So Metric3D is excellent, but if your only goal is a control map, Depth Anything v2 is usually the easier daily driver. Reach for Metric3D when you actually want the geometry.

The inputs that matter

The node takes an image and returns an IMAGE depth map for your ControlNet Apply node. The options:

• backbone (default vit-small; also vit-large, vit-giant2) - model size. vit-small is fast and fine for most conditioning; step up to vit-large or vit-giant2 for sharper, more detailed depth at real VRAM and time cost. Giant2 is the quality ceiling and the one from the shootout.
• fx and fy (default 1000 each) - the camera's horizontal and vertical focal lengths in pixels. This is what makes the "metric" part work. The defaults are a reasonable stand-in when you don't know the real camera params, and for ControlNet you can usually just leave them. If you have the actual EXIF focal length and you care about accurate scale, set them.

resolution (default 512) sets the working size.

Installing it

ComfyUI Manager: search ComfyUI's ControlNet Auxiliary Preprocessors, install, restart. Or manually:

cd ComfyUI/custom_nodes
git clone https://github.com/Fannovel16/comfyui_controlnet_aux
pip install -r requirements.txt

Restart ComfyUI. The Metric3D backbone weights download on first run; choosing vit-giant2 pulls a large checkpoint, so expect the first run with that setting to be slow.

Where people get burned

Two traps. First, choosing vit-giant2 and then being surprised it's slow and VRAM-heavy - that's the biggest model, so start with vit-small and only size up if the depth isn't detailed enough. Second, overthinking fx/fy: for ControlNet conditioning the defaults are fine, and fiddling with focal lengths you don't actually know just adds noise. If you're chasing crisp control maps rather than true geometry and Metric3D's output feels soft, that's your cue to try Depth Anything v2 instead. And the pack-wide constant: the depth map only matters if a depth ControlNet is loaded downstream and set to a strength that bites.