ComfyUI-CXN

Custom ComfyUI nodes for loading and rendering FBX 3D models.

Nodes

Load FBX Model

Loads an FBX file from ComfyUI/input/3d/, reorients it into the pack's canonical shoe frame (+X = lateral, +Y = up, +Z = toe), and optionally applies up to five image textures to the materials in scene order. Outputs a MODEL_3D object.

The canonical orientation is frontal: at the default Render FBX camera (0, 0, 1) with no rotation, the shoe is seen head-on with the toe pointing toward the camera (+Z), the top of the shoe up (+Y), and the lateral axis right (+X).

Inputs:

| Param | Type | Default | Notes | | ---------- | ------ | ------- | ----- | | model_file | combo | — | FBX file under ComfyUI/input/3d/. | | toe_axis | combo | +Y | Which FBX-local axis points toward the toe in the source file. Defaults match PS5I-style models; pick another value if your FBX is authored differently. | | up_axis | combo | +X | Which FBX-local axis points to the top of the shoe (opposite of the sole). Must be on a different axis from toe_axis. | | texture_1…5| IMAGE | — | Optional. Applied to materials in scene order. |

How to find the right axes: load the FBX with the defaults, run Render FBX at x=0, y=0, z=1, all angles 0 — you should see the shoe front-on with the toe pointing at the camera, the laces facing up, and the lateral side to the right. If the toe end isn't facing the camera, try a different toe_axis. If you see the sole instead of the laces, try a different up_axis. Once the default view is the canonical frontal view, Match Foot Pose and Render FBX downstream will compose correctly.

Render FBX

Renders a MODEL_3D to an IMAGE using an offscreen OpenGL (EGL) renderer with a built-in 3-point + ambient lighting setup so the object is always visible.

Required inputs:

| Param | Type | Default | Notes | | ------ | ----- | ------- | ---------------------------------------------- | | x,y,z | FLOAT | 0,0,3 | Camera world position | | pitch | FLOAT | 0 | Camera rotation around local X axis (degrees) | | yaw | FLOAT | 0 | Camera rotation around local Y axis (degrees) | | roll | FLOAT | 0 | Camera rotation around local Z axis (degrees) | | width | INT | 512 | Output image width | | height | INT | 512 | Output image height | | fov | FLOAT | 45 | Vertical field of view in degrees |

Optional inputs (model pose; identity by default):

| Param | Type | Default | Notes | | -------------- | ----- | ------- | ------------------------------------------- | | model_x/y/z | FLOAT | 0 | Translate the model in world space | | model_pitch | FLOAT | 0 | Model rotation around X axis (degrees) | | model_yaw | FLOAT | 0 | Model rotation around Y axis (degrees) | | model_roll | FLOAT | 0 | Model rotation around Z axis (degrees) |

Convention: right-handed, Y-up. Camera looks down its local −Z axis. Rotation order is YXZ (yaw → pitch → roll). The model pose is applied first, then the camera transform.

Match Foot Pose (SAM3D Body)

Takes a shoe MODEL_3D plus the skeleton output of ComfyUI-SAM3DBody's SAM 3D Body: Process Image node (custom type SKELETON), detects a foot in the joint cloud, and emits the shoe's model_pose (x,y,z, pitch,yaw,roll) plus the photo's vertical FOV — wired into Render FBX they place the shoe exactly where the foot would project in the original photo.

Inputs:

| Param | Type | Default | Notes | | ---------------- | --------- | ------- | -------------------------------------------------------------------- | | model_3d | MODEL_3D | — | Right-foot shoe in canonical orientation (+X = lateral, +Y = up, +Z = toe, frontal view at default camera). Use Load FBX's toe_axis / up_axis to bring any FBX to this convention. | | skeleton | SKELETON | — | The skeleton output of SAM3D Body's Process Image node. Reads joint_positions, camera (=pred_cam_t), focal_length. | | left_or_right | BOOLEAN | right | Detects the requested foot on its own side of the body. | | foot_y_fraction | FLOAT | 0.20 | Fraction of the side's own Y-range (bottom-up) used as the foot region. Decrease (~0.10) to focus on the foot proper and exclude shin joints — important for tilted poses. | | image_height | INT | 0 | Height in pixels of the original photo (e.g. 1200). Used with focal_length to compute vertical FOV. Leave 0 to disable FOV calculation. | | image_width | INT | 0 | Width in pixels of the original photo. Use it to set Render FBX's width so the rendered aspect matches the photo. |

Outputs:

| Output | Type | Wire to (on Render FBX) | | ----------- | -------- | ----------------------- | | model_3d | MODEL_3D | model_3d | | cam_x | FLOAT | x | | cam_y | FLOAT | y | | cam_z | FLOAT | z | | pitch | FLOAT | model_pitch | | yaw | FLOAT | model_yaw | | roll | FLOAT | model_roll | | fov | FLOAT | fov |

The new contract: camera moves to the foot, the shoe stays at world origin and rotates in place. cam_x, cam_y, cam_z go straight to Render FBX's camera inputs; pitch, yaw, roll go to model_pitch / model_yaw / model_roll. Leave Render FBX's model_x, model_y, model_z and camera pitch, yaw, roll at their zero defaults. The rendered image is mathematically equivalent to the previous "model-at-foot, camera-at-origin" wiring (same pixel positions, same pose), but the inputs are easier to override manually — for instance, you can nudge a single rotation channel without disturbing the position, or move the camera to inspect the shoe from a different angle.

How the foot is found

1. Read joint_positions, add pred_cam_t (body moves into camera space), negate Y and Z (CV → graphics: Y up, Z back, camera at origin looking -Z).
2. Drop any joint that sits more than 25 cm below the next-lowest joint in Y. The MHR 127-joint output has a global root anchor at joint 0 that sits ~60 cm below the rest of the skeleton and would otherwise dominate the foot region.
3. Split the remaining joints by the body's lateral centerline (median X) into LEFT (X > median) and RIGHT (X ≤ median) — cam_t shifts the body off X=0, so X=0 is not a reliable split.
4. Within the requested side, take the bottom-foot_y_fraction of joints by Y. Their centroid is the foot position; their XZ-PCA gives the toe-forward axis.
5. If left_or_right is left, mirror the shoe model along its medial-lateral axis so the right-foot canonical mesh becomes a left-foot one. The pose itself comes from the actual LEFT-side joints — no positional mirroring.

Caveat: SAM3D's right-side joints can sit higher than the left's

For some bodies/poses SAM3D's MHR regressor places lower-leg keypoints on the two sides at different heights — i.e. the lowest joint on the right side might be a calf point while the lowest on the left side is a true foot point. When that happens the two shoes will render at different Y levels even though the real person is standing on level ground. If you see this, options:

• Increase foot_y_fraction until both sides include actual ankle-area joints.
• Render each foot in a separate Render FBX, then manually equalize Y in the lower one with model_y before compositing.
• Fall back to driving both feet from the mesh-data variant of this idea (one detected, the other mirrored) if precise photo-overlay isn't required.

Workflow: overlay the shoe on the original photo

[Load Image] ──┬─> [SAM 3D Body: Process Image] ──(mesh_data)──┐
               │                                                │
               └─> width × height                               │
                                                                ▼
[Load FBX Model] ──(model_3d)─────────────────────> [Match Foot Pose] ──(x,y,z,pitch,yaw,roll,fov)──> [Render FBX]
                                                                                                          │
                                                                                       (camera at 0,0,0,  ┘
                                                                                        all angles 0,
                                                                                        width/height = photo size)

Critical wiring rules:

1. Send x, y, z, pitch, yaw, roll from Match Foot Pose into model_x, model_y, model_z, model_pitch, model_yaw, model_roll on Render FBX — NOT into the camera inputs. The camera inputs control the viewpoint; the model inputs place the object in the scene.
2. Set the Render FBX camera to x=0, y=0, z=0, pitch=0, yaw=0, roll=0 — SAM3D's camera lives at the world origin looking -Z.
3. Send fov into Render FBX's fov. (Default 45° will be the wrong scale.)
4. Set Render FBX's width and height to the original photo's pixel dimensions (or any multiple of its aspect ratio).
5. Pass image_height into Match Foot Pose so it can compute FOV.

Common wiring mistakes:

• Wiring the 6 pose floats into the camera inputs instead of the model inputs → shoe appears close-up filling the frame.
• Leaving the camera at the default z=3 → shoe appears small and at the wrong projected position.
• Leaving FOV at the default 45° → shoe scale won't match the photo's perspective.

Install

# 1. System deps (Linux)
sudo apt install libassimp-dev libegl1 libegl-mesa0

# 2. Python deps
pip install -r requirements.txt

Drop FBX files into ComfyUI/input/3d/.

Running ComfyUI in Docker

If you run ComfyUI inside a Docker container, add the system packages to your Dockerfile:

RUN apt-get update && apt-get install -y \
    libassimp-dev libegl1 libegl-mesa0

The custom node's requirements.txt (auto-installed by typical ComfyUI entrypoint scripts) takes care of the Python side. Note: pyrender pins PyOpenGL 3.1.0 (from 2014), which is incompatible with NumPy 2.x — our requirements.txt forces PyOpenGL>=3.1.7 to fix this.

Headless rendering

pyrender is forced to the EGL platform (PYOPENGL_PLATFORM=egl). NVIDIA GPUs provide a hardware EGL backend once libegl1 is installed. If you have no EGL, install libosmesa6 instead and set PYOPENGL_PLATFORM=osmesa before launching ComfyUI.

License

MIT