Load OmniGen Model

OmniGen is the "one model does everything" diffusion transformer - text-to-image, image editing, pose transfer, segmentation, try-on, all from a plain sentence instead of a ControlNet stack. The Load OmniGen Model node is where that pipeline starts, and it's also where most people trip, because there is no auto-download. The README is blunt about it: automatic model downloading was deliberately removed so this loader could handle finetunes, merges, and fp8 dtypes. You bring the checkpoint, this node decides how it lives in your VRAM.

What it actually loads

The loader reads from ComfyUI/models/OmniGen/<subfolder>/. It scans that directory at build time and gives you a model_name dropdown of whatever folders it finds. Point it at a folder containing the Shitao/OmniGen-v1 files (the model.safetensors is a chunky ~15.5 GB in bf16, plus a ~0.3 GB VAE) and it runs OmniGenPipeline.from_pretrained() on it, then casts the transformer to your chosen weight_dtype.

That dtype choice is the main performance lever, and it maps straight from the code:

• default → bf16. Full quality, biggest footprint.
• fp8_e4m3fn and fp8_e4m3fn_fast → float8. Roughly half the weight memory, tiny quality hit.
• fp8_e5m2 → the other fp8 flavor; lower precision, occasionally useful when you're desperate.

If you've got 24 GB+ this is a "set it to default and forget it" node. On a 16 GB card, fp8_e4m3fn plus the memory toggles below is how you make the thing run at all. The knowledge-base framing applies here: check the model file plus what the pipeline needs at inference, not just the on-disk number - a 15.5 GB bf16 transformer isn't going to infer in 12 GB no matter what the file size implies.

The inputs that matter

Three of them do the real work:

• model_name - dropdown of folders in models/OmniGen/. If it says "none", you haven't put the model in a subfolder yet.
• weight_dtype - bf16 or one of the fp8 variants, as above.
• store_in_vram - keep the loaded pipeline in VRAM between generations so the next run skips the slow reload. Faster, but the model stays resident. Leave it off on a small card.

Then two memory tricks, both worth leaving on their defaults: separate_cfg_infer (default on) runs the different guidance scales in separate passes to save memory at the cost of speed, and offload_model (default off) shoves the model to CPU to cut VRAM hard while making everything slower. If you're OOMing, flip offload on before you give up.

Output and wiring

The output is a single OMNIGEN_MODEL port - an opaque pipe object plus its memory config. Nothing else in ComfyUI understands it; it plugs into the OmniGen node (OmniGenNode), and that's it. The node caches the loaded pipeline internally when store_in_vram is set, so switching model, dtype, or a memory toggle tears it down and reloads from scratch - that's the normal behavior, not a crash.

Installing it

Install the pack via ComfyUI Manager by searching "OmniGen-ComfyUI" - and the README explicitly warns you to pick the one by AIFSH, because clones with less functionality have appeared. Or go manual:

cd ComfyUI/custom_nodes
git clone https://github.com/AIFSH/OmniGen-ComfyUI

Then restart ComfyUI. The pack pulls diffusers, peft, timm, accelerate, and datasets from its requirements.txt - a real dependency stack, so let Manager or pip install it before judging it broken.

Then the model, which is the part people actually get stuck on:

mkdir -p ComfyUI/models/OmniGen/OmniGen-v1
# download everything from https://huggingface.co/Shitao/OmniGen-v1
# into ComfyUI/models/OmniGen/OmniGen-v1/

Note the path: it's models/OmniGen, moved from the pack's older folder location, and the model has to sit in its own subfolder - that's why the dropdown lists folder names. git lfs + git clone of the HF repo also works but can look frozen for a while on the 15.5 GB file.

Where people get burned

The classic error is "No model folder found in models/OmniGen/" - the model folder doesn't exist or the files are loose in the parent dir. The second one is treating first load as a hang: a 15.5 GB bf16 checkpoint being cast to fp8 takes a minute, and the console spits a VRAM report while it works. If it ever feels like it's just sitting there, it's probably loading, not stuck.