3D_GenerateDepthImage

Before anything gets painted, the pipeline has to know what the unpainted model looks like from the cameras it's about to use. 3D_GenerateDepthImage renders the mesh from two view angles and hands you a depth map you feed to a depth ControlNet - which is what keeps the front/back SD1.5 samples from coming back looking like a completely different object. It's the spatial anchor for the first sampling stage.

What it does

It renders the mesh from two camera positions (cam1 and cam2), grabs the depth buffer from each render, and stitches the two depth images side by side into a single 2×1 image. Then it does a small amount of OpenCV post-processing - a repeated dilate/erode outline pass (5 iterations) that fattens thin edges and fills gaps in the silhouette. Why the cleanup? A bare depth render of a low-poly mesh has paper-thin edges that a ControlNet reads poorly, so the dilation gives the condition map a cleaner, chunkier outline.

The output is a condition image, not a pretty picture. You wire it straight into a ControlNetApplyAdvanced node along with the depth ControlNet (control_v11f1p_sd15_depth_fp16.safetensors from comfyanonymous/ControlNet-v1-1_fp16_safetensors). The workflow applies that condition to the positive and negative conditioning, then runs it into KSampler for the front/back sampling pass. Depth ControlNet mechanics here are the standard SD1.5 depth conditioning - the condition steers where the geometry goes while the prompt decides what the surface looks like.

Inputs that matter

• mesh_model - the MESHMODEL from 3D_LoadMeshModel.
• cam1 / cam2 - the two camera indices, 0–26, default 0 and 23. These must match the cameras used in 3D_Projection and the sampler's latent size. The latent coming out of 3D_TrainConfigPipe is sized for a 2-view grid (1024×512), and the sample is generated at that same 2×1 shape - mismatch the cameras and the depth says "front" while your sample is a back view.

Output

One output: an IMAGE - the stitched, outline-fattened depth map for both views. It feeds ControlNetApplyAdvanced's image input.

Installing

The whole pack installs the same way: Manager search "ComfyUI-Paint3D-Nodes", or

cd ComfyUI/custom_nodes
git clone https://github.com/N3rd00d/ComfyUI-Paint3D-Nodes
pip install -r requirements.txt
pip install kaolin==0.15.0 -f https://nvidia-kaolin.s3.us-east-2.amazonaws.com/torch-2.1.0_cu121.html

Kaolin is the fragile dependency - if the torch version on your box doesn't match the wheel, the README says to pin torch==2.1.0 + cu121 and reinstall. The depth ControlNet itself comes from comfyanonymous/ControlNet-v1-1_fp16_safetensors and lives in ComfyUI/models/controlnet.

Gotchas

The classic failure here is a depth image that looks "wrong" - usually because the cameras you rendered don't match the camera indices used downstream, or because your mesh has no proper UV/geometry for the renderer to work with. There's a tiny preview node wired after this in the sample workflow, so you can sanity-check the depth map before you burn a sampling run. This pack has almost no community writeups, so if it renders something odd, your first move is checking the camera numbers, not searching for a known fix.