OmniGen

Here's the whole OmniGen pitch: instead of chaining a ControlNet, an inpaint model and an IP-Adapter together, you type "image_1 wears image_2" or "Find the lamp in image_1 and color it blue" and one unified diffusion transformer does it. That's the OmniGen node - the generation half of AIFSH's OmniGen pack, sitting downstream of Load OmniGen Model. It genuinely feels like cheating the first few times you use it. It also eats VRAM like a cartoon character, so let's set expectations up front: community consensus for comfortable 1024px work is around 24 GB, 48 GB if you want to relax. It's the tradeoff the unified-model idea bought.

How the prompt trick works

The neat part is that you never have to remember OmniGen's raw token syntax. The model itself reads image references as <img><|image_1|></img>, but this node rewrites your text for you: write plain image_1 (or image_2, image_3) in the prompt, connect the matching image port, and the node replaces the short form with the full token before it runs. The prompt field's own tooltip mentions the token form because that's what's underneath, but the short form is the one to actually type.

Mechanically, each connected image gets saved to a temp PNG - the underlying OmniGen pipeline works from image paths, not tensors - and the substituted prompt plus those paths go into a single pipe() call with your settings. One gotcha hiding in there: if you hand-paste the full <img><|image_1|></img> token and wire the image port, the auto-replacement can double-tokenize it into garbage. Just type image_1 and let the node expand it.

The trap nobody warns you about: latent

There's a required latent input, and it's not cosmetic - it sets your output size. The node reads height and width from the latent tensor and multiplies by 8, so you must wire an EmptyLatentImage (or any latent source) before the node will run. There are no width/height widgets on this node; that's the single most common "why won't it generate" question about it. EmptyLatentImage at 1024×1024 is the sane starting point.

Inputs that actually matter

• prompt_text - the sentence. image_1, image_2, image_3 become image references.
• latent - from EmptyLatentImage; sets output resolution.
• num_inference_steps - default 50. OmniGen is not a distilled turbo model; don't crank this down expecting it to behave like one.
• guidance_scale (2.5) and img_guidance_scale (1.6) - text and image conditioning strength respectively. The defaults are sensible; the img one is the dial you nudge for editing jobs.
• max_input_image_size (1024) - input images get cropped to at most this; lower it to save memory.
• use_input_image_size_as_output - flip on for single-image editing so the output matches the input's dimensions instead of the latent's.
• image_1 / image_2 / image_3 - the optional conditioning images, wired in from Load Image nodes.

Output is one standard IMAGE, so it feeds Preview Image or Save Image like anything else. Don't forget the model port on top comes from the pack's loader - the OMNIGEN_MODEL type is pack-specific, so this node won't accept a random checkpoint.

Install and VRAM strategy

Same pack as the loader: ComfyUI Manager → search "OmniGen-ComfyUI" by AIFSH (clones exist, pick the right one), or git clone https://github.com/AIFSH/OmniGen-ComfyUI into custom_nodes, restart, then drop the Shitao/OmniGen-v1 files (~15.5 GB) into ComfyUI/models/OmniGen/<subfolder>/. The pack's requirements - diffusers, peft, timm, accelerate, datasets - install with it.

If it OOMs, work through this order: load with fp8_e4m3fn on the loader, keep separate_cfg_infer on (it already is - it splits the guidance passes to save memory), drop max_input_image_size, and only then reach for offload_model, which buys VRAM at a serious speed cost. The KB's VRAM rule of thumb applies: this is a big bf16 transformer, and on a 12–16 GB card you're in "fp8 and crossed fingers" territory, not "set and forget."

Two more things worth knowing. First, the pack's bundled example workflows reference a PromptTextNode that's gone from the current code - if you load one and see a missing node, just swap in any normal text/string node; the input is a plain string now. Second, this is a v1-era pack (last touched in late 2024) and it's still the straightforward way to run OmniGen-v1 in ComfyUI - the underlying project has since moved on, but the model this pack targets remains a fun, weird one-of-a-kind thing to have around for a rainy editing session.