ComfyUI Node

ImageTokensI2I

Feeding a reference image into a diffusion LLM

By L-Hugh·Created 10 months ago·Updated 10 months ago· 2
ImageTokensI2I
  • vqvae
  • image
  • ref_image
  • image_tokens

Image-to-image in the Lumina-DiMOO world isn't what you're used to. In Stable Diffusion, img2img starts from your image's noisy latent and denoises it into a variation. Here, ImageTokens_I2I encodes your image into VQ tokens and hands them to the model as context - the model then writes a brand-new token grid conditioned on what it saw. Think of it less as "edit this picture" and more as "here's a reference, now draw a matching image from scratch."

That distinction matters, because it changes what you should feed it and what you'll get back.

How it works

The node takes your image and pushes it through the VQ-VAE (the same codec that decodes at the end) to get a sequence of discrete tokens, wrapped in begin/end image markers. There are two important details in how it does that:

  • It doesn't keep your image's full resolution. It randomly center-crops to roughly a 512²-equivalent token canvas before encoding. So fine detail at the edges gets thrown away - you're giving the model the gist, not a pixel-perfect copy.
  • The crop is random each run, which means this node isn't seed-deterministic. Run the same workflow twice and you'll get different crops (and therefore somewhat different outputs), even with a fixed seed everywhere else.

The optional ref_image input is the interesting part. It encodes a second image into tokens as well, and when present the generator builds its conditioning from both images plus the text. That's how the pack's subject-driven, style-transfer, and multi-image editing workflows work - one image is the subject, the other sets the style or context. Leave it empty and you get single-image conditioning, which pairs with the simpler editing tasks.

Inputs:

  • vqvae (LUMINA_VQ) - from LoadModel.
  • image (IMAGE) - your source image.
  • ref_image (IMAGE, optional) - a second conditioning image for subject/style tasks.

Output:

  • image_tokens (LUMINA_IMAGETOKENS_I2I) → Generator_I2I (note: it feeds the I2I generator, not the T2I one - the types enforce this for you).

To make it do anything useful you also need PromptBuilder set to a matching task - the controllable-generation example uses depth_control, and the editing examples use the edit_* tasks. The text tokens and image tokens get combined inside Generator_I2I.

Where people get tripped up

Don't expect an SD-style denoise slider or a "strength" input. There isn't one. The output is a full regeneration at roughly the input's canvas size, and the amount of change is controlled by the prompt and by how much the input image conditions it. People who come here looking for a 0.4-strength img2img leave confused; people who frame it as "make something new inspired by this" do fine.

Also remember: the input image gets cropped down to ~512-equivalent, so the output resolution is tied to that crop, not your original. If you feed a 4K photo, you're still working from a ~512 canvas worth of tokens. This is a model in its early-adopter phase - impressive mechanism, finicky limits, and a ~17 GB weight download plus pinned old dependencies before you even see your first result. Install the pack via ComfyUI Manager or git clone from the ComfyUI-Lumina-DiMOO repo, and start from the shipped example workflows rather than building the graph blind.

CategoryLumina-DiMOO

Inputs (3)

NameTypeDefaultDescription
vqvaeLUMINA_VQ
imageIMAGE
ref_imageoptIMAGE

Outputs (1)

NameTypeDescription
image_tokensLUMINA_IMAGETOKENS_I2I