X -> RGB (Material to Photo)

This is the "X -> RGB" direction of the ComfyUI-rgbx pack, and it's the node that makes the whole thing worth having. Feed it a set of material maps - albedo, normal, roughness, metallic, irradiance, any combination of them - and it generates a photorealistic image from those channels, like a renderer that runs on diffusion instead of ray tracing.

The classic workflow is a relight: run RGB2X on a photo to extract albedo and irradiance, then edit the irradiance map (repaint the shadows, add a sunset glow) and run this node. Because albedo holds the texture and irradiance holds the light, you can change the lighting without the texture going with it - the thing that makes "prompt: relight this" so unreliable in plain text-to-image. Pair it with the pack's X2RGBInpainting when you want that regeneration constrained to a region of a reference photo instead of a free render.

Community take, for what it's worth: this family of "extract then regenerate" PBR workflows tends to get albedo and normals right, roughness roughly right, and metallic/irradiance hand-wavy. RGB-X is no exception. It's genuinely useful for relighting experiments and concept work, not for asset production.

How it works

The node loads the zheng95z/x-to-rgb pipeline and preps each map the way the model expects - this is where the details live:

• albedo is converted from sRGB to linear (^2.2)
• normal is shifted to [-1, 1] and L2-normalized
• roughness and metallic are clamped to [0, 1]
• irradiance gets the linear + clamp treatment

Then it samples a DDIM pass with two guidance signals - a classic text guidance_scale (default 7.5) and an image_guidance_scale (default 1.5) that controls how hard the material maps are enforced. That second knob is the one to turn when the output ignores your maps or, conversely, looks like a copy-paste of them.

The inputs that matter

• seed / steps - defaults 0 / 50. Standard fare; steps can come down.
• guidance_scale - prompt adherence. Leave near 7.5 unless you're fighting the prompt.
• image_guidance_scale - this is the node's personality. Higher = the maps win. Lower = the model freewheels more.
• albedo / normal / roughness / metallic / irradiance - all optional, all IMAGE. Feed any subset; resolution is taken from the first non-None map you provide (falls back to 768x768 if you feed none).
• prompt - optional text conditioning. With no maps at all, this is basically an SD-family text-to-image run; the maps are what make it material-aware.

Output: one IMAGE.

Install and models

Install via ComfyUI Manager (search "RGB-X") or:

cd ComfyUI/custom_nodes
git clone https://github.com/leob03/ComfyUI-rgbx.git

Dependencies are diffusers>=0.20.0, transformers, torchvision - normally already present. The weights for this pipeline go in ComfyUI/models/rgbx/x-to-rgb/; configs are bundled in the repo, so grab just the safetensors:

huggingface-cli download zheng95z/x-to-rgb --include "*.safetensors" --local-dir models/rgbx/x-to-rgb

No local folder means the node falls back to a HuggingFace download on first run (slow, needs internet - pre-download). Budget ~12GB VRAM; it loads the pipeline in fp16.

Where people get burned

The maps and the generated image are the same resolution as your first fed map - so a tiny RGB2X output (low max_side) gives you a tiny photo out. Feed it matching-resolution maps; mixing a 512 albedo with a 1024 normal makes the node guess which one to trust. And manage expectations on "photorealism": it's conditioned on maps, not on a photoreal checkpoint's aesthetics, so the result can look flatter than a good SDXL render. That's the trade for actually respecting your channels.