Nodes/ComfyUI-Lumina-DiMOO/ImageTokensInpainting
ComfyUI Node

ImageTokensInpainting

Inpainting and outpainting by ratio, not by brush

By L-Hugh·Created 10 months ago·Updated 10 months ago· 2
ImageTokensInpainting
  • vqvae
  • image
  • image_tokens
modeinpainting
mask_h_ratio0.50
mask_w_ratio0.50

Inpainting in ComfyUI usually means a mask you painted by hand. Not here. ImageTokens_Inpainting has no mask input at all - instead it masks a centered rectangle whose size is set by two ratios, and whether that rectangle is what gets filled or what gets kept depends on a two-way mode switch. It's crude next to a hand-painted mask workflow, but for the token-based architecture it fits, and for the two jobs this pack targets it's genuinely simple.

The trick it uses is worth understanding, because it's different from how diffusion inpainting preserves content. The node encodes your image through the VQ-VAE to get real tokens, then sets every token inside the masked region to the mask placeholder while leaving the rest as actual image tokens. So the model doesn't re-draw the unmasked area - it's literally already in the sequence, in the model's own language, and the sampler only has to fill the holes.

The two modes and the ratios

Inputs:

  • vqvae (LUMINA_VQ) - from LoadModel.
  • image (IMAGE) - the source image.
  • mode (enum, default inpainting) - inpainting masks the inside of the centered rectangle and regenerates it, keeping the border. outpainting is the inverse: it keeps the center rectangle and regenerates everything around it, so you can extrapolate a wider scene from a subject in the middle.
  • mask_h_ratio / mask_w_ratio (FLOAT, default 0.5) - the fraction of the image's height and width that the centered rectangle covers. With both at 0.5 you're regenerating the middle half of the image in inpainting mode.

Output:

  • image_tokens (LUMINA_IMAGETOKENS_T2I) → Generator_T2I. Note the type: it's the T2I token type, not I2I, because inpainting reuses the plain text-to-image generator - the masked image tokens are just part of the prompt sequence.

How to actually use it

The pack's inpainting example wires LoadModelImageTokens_Inpainting + PromptBuilder (task inpainting, with a plain-language instruction like "make there be a Porsche logo on the back wall") → Generator_T2IVQDecode. The text prompt guides what fills the hole.

The honest limitations:

  • No arbitrary shapes. It's always a centered rectangle. If you need to fix a face in a corner or a specific object, this isn't the node - grab a regular inpainting workflow with a real mask.
  • It keeps your input resolution. The token grid comes from the input image's size, and the generator fills within it, so what you get out is the same canvas, patched.
  • Ratios are relative, not absolute. mask_h_ratio 0.5 always means half the height, whatever that is. Want a thin horizontal band filled? Set a small height ratio and a wide width ratio.

One more gotcha that's not specific to this node but will absolutely bite you here: the whole pipeline needs the ~17 GB Lumina-DiMOO weight snapshot (including the vqvae/ subfolder) downloaded into models/lumina_dimoo, and the pack's requirements.txt pins old torch/transformers versions that can fight your existing ComfyUI environment. Install through ComfyUI Manager (search ComfyUI-Lumina-DiMOO), start from the shipped examples, and budget time for setup before the creative part.

CategoryLumina-DiMOO

Inputs (5)

NameTypeDefaultDescription
vqvaeLUMINA_VQ
imageIMAGE
modeCOMBOinpainting2 options: inpainting, outpainting
mask_h_ratioFLOAT0.50
mask_w_ratioFLOAT0.50

Outputs (1)

NameTypeDescription
image_tokensLUMINA_IMAGETOKENS_T2I