RGB -> X (Intrinsic Decomposition)

Here's the promise: drop in a photo of, say, a rusty pipe, and this node hands you the same scene split into its material ingredients - albedo (base color with no shading), normals, roughness, metallic, and irradiance (the diffuse lighting). That's called intrinsic decomposition, and it's the hard half of relighting. Once you have albedo separate from lighting, you can change the light without repainting the texture, or hand the maps to a 3D tool and pretend you did a PBR scan. This is the "RGB -> X" direction of the ComfyUI-rgbx pack, which wraps the RGB-X research project.

Fair warning up front: this is research code wearing a ComfyUI jacket, and "decomposition" is a generous word. RGB-X runs an actual diffusion pass to invent each channel, so you're getting an estimate, not ground truth. In the community, this is the project people point to when someone asks "can I generate specular/PBR maps from a photo?" - and the honest answer is "yes, roughly." For relighting and compositing experiments it's great. For shipping game-ready PBR, keep looking.

How it works

The node loads the zheng95z/rgb-to-x pipeline, converts your photo from sRGB to linear space (^2.2 - the single most important detail for these models), resizes it down so it fits within max_side, and runs a diffusion denoise pass conditioned on a text prompt that names the channel you asked for ("Albedo (diffuse basecolor)", "Camera-space Normal", "Roughness", and so on). Then it resizes the result back up to your original resolution.

Two things worth knowing about that: the downscale-then-upscale round trip keeps VRAM and time in check, but it also means output is a little softer than your input - don't expect the noise-free crispness of a real render. And because it's a full sampling run, seed and steps are there for a reason: bump the seed to try a different "guess" at the same channel.

The inputs that matter

• image - your RGB photo. One image in, one image out, at the same resolution.
• aov - the channel to extract. The dropdown: albedo, normal, roughness, metallic, irradiance. That's the whole job in five words.
• seed / steps - reproducibility and quality. Default 50 steps; you can usually drop toward 20-30 and still get a usable map.
• max_side (optional) - max edge length for the inference pass, default 1000. Lower it (512) to go faster and save VRAM; raise it if detail matters more than speed.

Output: one IMAGE, the extracted channel. Albedo and normal outputs wire straight into X2RGB or X2RGBInpainting (same pack) for relighting; a normal map also plays nicely with the kinds of normal-driven compositing covered in our depth estimation docs.

Install and models

Install the pack like any custom node - ComfyUI Manager (search "RGB-X"), or manually:

cd ComfyUI/custom_nodes
git clone https://github.com/leob03/ComfyUI-rgbx.git

Dependencies are diffusers>=0.20.0, transformers, and torchvision - almost always already in a working ComfyUI environment. The actual weights go in ComfyUI/models/rgbx/rgb-to-x/. The configs ship inside the repo, so you only need the safetensors:

huggingface-cli download zheng95z/rgb-to-x --include "*.safetensors" --local-dir models/rgbx/rgb-to-x

If the directory doesn't exist at all, the node silently falls back to downloading from HuggingFace on first run - handy, but it needs internet and it'll stall your first execution. Download the weights first. You'll also want the ~12GB VRAM the README asks for; this is a full SD-size pipeline running in fp16.

Where people get burned

The model-missing trap above is the big one - the auto-download fallback means the first run can look frozen. Also: one run per channel. RGB2X extracts a single AOV per execution, so to get all five maps you run the node five times (or copy the node). And remember those maps are estimates, so if you're chaining straight into a renderer, sanity-check the normal map before trusting it - camera-space vs. tangent-space conventions bite hard downstream.