RenderFormer From JSON

Every scene you've built so far was assembled visually, node by node. RenderFormerFromJSON is the escape hatch: instead of a graph of mesh/camera/light nodes, you paste a JSON description of the entire scene - objects with transforms and materials, lights, cameras - and it becomes a SCENE directly. It's the node for people who want RenderFormer's original, file-based scene format, or who are scripting scenes programmatically. One of the 15 nodes in paulh4x/ComfyUI_PHRenderFormerWrapper, the "100% vibecoded" wrapper around Microsoft's RenderFormer (SIGGRAPH 2025).

How it works

The node takes your JSON, parses it into RenderFormer's SceneConfig structure, and runs the same in-memory conversion the official examples use (patching trimesh and h5py so nothing touches disk). The default template - which you'll see pre-filled in the multiline editor - is a faithful copy of the classic RenderFormer background + light setup: four background meshes (plane.obj plus three walls), one emissive light triangle, and a camera at (0, -2, 0) with a 37.5° FOV.

The inputs that matter

• scene_json - a multiline string holding the whole scene. The structure is {"objects": {...}, "cameras": [...]} where each object has mesh_path, transform (translation/rotation/scale/normalize), and material (diffuse, specular, roughness, emissive, smooth_shading, rand_tri_diffuse_seed). Study the default template - it's the fastest way to learn the schema.
• add_default_background - adds plane + walls if your JSON doesn't include them.
• additional_mesh_file - a dropdown of .obj files in ComfyUI/input/3d (with a "None" option). Picking one injects it into the scene as a new object.
• additional_mesh_path - same idea via an absolute path to an .obj (per the tooltip). If both are set, the path wins.
• mesh_root_dir - optional root directory so relative mesh_path values in the JSON resolve somewhere sane.

One output: SCENE, ready for the Sampler (no Scene Builder needed - you've already built the scene in text).

When to reach for it

Honestly, for most people: rarely. The node-based path is easier to tweak live. But it shines for reproducible scenes (a JSON blob in a workflow is version-controllable), for scenes with many objects where wiring gets tedious, and for porting scenes from the official RenderFormer repo's .json example files - the schema is the same. If you're sharing a "scene recipe" with someone, this is a cleaner artifact than a screenshot of a graph.

Installing it

# ComfyUI Manager: search "ComfyUI_PHRenderFormerWrapper"
cd ComfyUI/custom_nodes/
git clone https://github.com/paulh4x/ComfyUI_PHRenderFormerWrapper.git
cd ComfyUI_PHRenderFormerWrapper
git clone https://github.com/microsoft/renderformer.git renderformer
pip install -r requirements.txt
python -c "import imageio; imageio.plugins.freeimage.download()"

Restart ComfyUI.

Common issues

• JSON parse errors - trailing commas and single quotes are invalid JSON; the node calls json.loads directly, so it's strict. The dacite.exceptions errors from the changelog are usually schema mismatches (wrong key names, wrong types).
• "Mesh file not found" for a relative path - set mesh_root_dir to where the .obj files live, or use absolute paths in the JSON.
• Only one camera - the schema supports a cameras list, but the current node passes the scene through the standard converter, which is built around the static single-camera case. Animations via JSON are on the advanced/experimental side.

It's the power-user node - less friendly day-to-day, but the one that makes this pack feel like it can talk to the rest of the 3D ecosystem.