MangaNinjiaSampler

This is the node that actually runs MangaNinja, the Alibaba paper (arXiv 2501.08332) that colors line art while following a reference image. Drop in a colored reference, drop in your line art, and instead of the model inventing colors and hoping, it pulls them from the reference - that blue shirt on the reference is a blue shirt in the output. The catch, and the whole point of this pack, is that plain colorization is easy; precise reference following needs you to tell it which pixels correspond, and that's where the two point-marker nodes come in.

How it works

Under the hood this is SD 1.5, plus a ControlNet, plus a couple of extra networks. The lineart ControlNet (a control_v11p_sd15_lineart model, the same family the KB's controlnet essay calls out for sketch-guided generation) keeps the output glued to your line structure. A reference UNet works like an IP-Adapter-ish attention injection: it reads the reference image through a CLIP vision encoder and steers color and texture. And a PointNet takes the sparse correspondence points you clicked and maps "this spot on the reference = this spot on the lineart," which is what makes it "precise" rather than vibe-based.

The optional xy_data_ref and xy_data_lineart inputs (type MINJIA_DATA) are where the point pairs arrive from the MarkImageNodes. The sampler turns those into two 512×512 index matrices - each matching point pair gets the same number stamped on both maps - and feeds them to the PointNet. No point data, no precision; you just get reference-conditioned colorization. The whole thing runs on a DDIM scheduler for steps (default 25), then VAE-decodes back to pixels.

The inputs that matter

• model - the MODEL_MangaNinjia bundle from MangaNinjiaLoader. Non-negotiable.
• image - your colored reference. lineart_image - the line art. Note only the first image in a batch is used for the reference embeddings, so keep it simple.
• is_lineart - set to True when lineart_image already is clean line art. Set False and it runs the bundled sk_model.pth preprocessor to extract lineart from a normal image first. That's the "feed it any sketch" escape hatch.
• width / height - 128–2048 in steps of 64. Everything gets upscaled or downscaled to this before sampling, and 512 is the processing resolution internally.
• guidance_scale_ref (9) and guidance_scale_point (15) - reference adherence vs. point-adherence. Point is the precision dial; crank it if colors drift from your markers.
• seed, steps - ordinary sampler fare.

Outputs

image is the colorized result - wire it to SaveImage. lineart is the extracted/used lineart, which is handy when is_lineart is False and you want to see what the preprocessor actually produced.

Install

Same drill as the rest of the pack: ComfyUI Manager → search "MangaNinjia" → install, or

cd ComfyUI/custom_nodes
git clone https://github.com/smthemex/ComfyUI_MangaNinjia

then restart. The README claims you don't need pip install -r requirements.txt ("it's based on sd1.5 and controlnet"), which is mostly true if you already run a full ComfyUI, but if the loader or annotator throws about basicsr or kornia, that's the file you want. You'll also need the model files - an SD 1.5 checkpoint with VAE, a lineart ControlNet, clip_l.safetensors, and the four .pth MangaNinja weights - see the loader article for the exact layout.

Where people get burned

The pack's loader calls enable_xformers_memory_efficient_attention() on the pipeline, so a missing/older xformers is the classic first-run crash. Memory is the second: you're holding SD 1.5 plus a ControlNet plus two UNets plus a PointNet plus CLIP encoders. On 8 GB VRAM expect to stay near 512×512. And if you wire up only one of xy_data_ref / xy_data_lineart, the sampler silently skips points entirely - pair them, or leave both unconnected. Images get forced to multiples of 64, so don't fight the slider with odd dimensions.