Triptych Register

Video models do not return what you gave them. In this pack's own documented run, a 1904×816 canvas came back as 1536×672: a 2.1% rescale, and not uniform, because 1904/1536 is 1.2397 and 816/672 is 1.2143. Stretch that frame back to the original canvas and you've sheared the character by about 2% - enough to eat the feet when you cut the panels out on the original pixel columns.

Triptych Register is the small, unglamorous node that fixes this, and it's the difference between a pipeline you can point at any provider and one that only works on the provider it was tuned on.

How it works

The insight is that frame 0 of an image-to-video clip is very nearly the image you submitted. So you don't need to inspect the model, you just compare two pictures.

Each image is reduced to the bounding box of everything that isn't the flat background colour - the same trick the compose step uses - and the scale and offset are read straight off those two boxes: sx = ref_w / frame_w, sy = ref_h / frame_h, and the offsets fall out of aligning the box corners. The flat background is what makes the box exact, which is why the whole pipeline composites onto one.

It then writes that transform into the layout JSON as a transform field and hands the layout on. Triptych Split applies it per frame: resample by sx/sy, paste at the recovered offset onto a canvas of the layout's size, then crop.

The report output is four numbers and a warning:

scale 1.2397 x 1.2143   offset -41.2, -18.6   skew 2.05%
WARNING: over 2% non-uniform scale; check the first split.

Any skew over 2% gets that warning appended. It's the cheapest early warning you'll get that something reshaped your canvas.

Be honest about what it buys you

The README publishes the measurement, which is rare and worth reading rather than skimming. Mean absolute error of the split panels against the source, per 255:

| What the model returned | Split naively | Registered first | |---|---|---| | Rescaled (what MiniMax H3 actually did) | 0.28 | 0.85 | | Letterboxed into 16:9 | 18.35 | 0.89 | | Centre-cropped by 6% | 20.19 | 0.54 |

For the clip this repo ran, a naive stretch was already fine - registration cost about half a point of resampling error and fixed nothing. It earns its place on providers that letterbox or crop, where it's twenty times better, and it turns a silent 2% shear into a printed warning either way. Keep it in the graph: it's free, deterministic, and it catches a failure you would not have noticed.

Inputs and outputs

Three required inputs, no widgets at all:

triptych - the canvas you actually submitted (from Triptych Compose, or a Load Image of the PNG you saved). first_frame - frame 0 of the clip: ImageFromBatch at index 0, or the first image out of Get Video Components. layout - the layout from Compose.

Two outputs. layout - the corrected one, and this is the one that goes into Triptych Split. report - the string above. The node isn't an output node, so if you want to see the report you have to wire it to a text/preview node or let it land somewhere you'll read.

Install

cd ComfyUI/custom_nodes
git clone https://github.com/meryboth/from-mesh-to-motion
# restart ComfyUI

Or ComfyUI Manager → from-mesh-to-motion, or comfy node install from-mesh-to-motion. Pillow and numpy, no downloads, no GPU. Granting a Comfy account/API key is only about the paid video call, not this node - the local half of the pipeline works without any key if you bring your own clip.

Where people get burned

The background has to stay flat. Registration works by finding what isn't the background colour. If the model repainted the canvas, added a gradient, or your video node graded the output, the content box comes back wrong and the transform is wrong with it - silently, apart from a weird skew number. This is also why the prompt node insists on describing the background accurately; a repainted background is the documented way to break this step.

Frame 0 has to be frame 0. Index 1, or a black first frame from a fade, gives you a garbage mapping. The tell is a skew number well above 2% or an offset measured in hundreds.

A frame that is entirely background raises Frame is entirely background; cannot register it. That's better than guessing, but it usually means the clip didn't return anything recognisable.

Wiring Compose's layout into Split and skipping this node is the most common mistake in the whole pack. It only looks fine when the provider preserved the aspect ratio exactly - which is precisely the case you can't rely on, and precisely the case where the error is small enough to blame on your mesh instead.