DiffSynth: Ideogram 4 Loader

Ideogram 4 was the best text-in-image model in the world, and in June 2026 Ideogram shipped the weights: a 9.3B, 34-layer single-stream DiT under a non-commercial agreement, with an nf4 variant that fits a single 24GB card. It set the corpus on fire - 762 of 781 threads in a single month - and then LoRA production stalled within about four weeks, because a non-commercial license puts a ceiling on what anyone will build around it. If all you want is legible text and deep character recall in an image you're not monetizing, that ceiling doesn't apply to you.

This Loader is where you declare four things, not three, and the fourth is the one people skip.

Inputs

Required: model_configs (merged list from DiffSynth: Merge ModelConfigs), torch_dtype (bfloat16 / float16 / float32), device (cuda / cpu). Optional: tokenizer_config - a ModelConfig for the tokenizer directory - and vram_limit from a VRAM Limit node.

Output: pipe → DiffSynth: Ideogram 4 Inference.

The four ModelConfigs

From the shipped fp8 template, all on ideogram-ai/ideogram-4-fp8:

• transformer/diffusion_pytorch_model.safetensors - the model.
• unconditional_transformer/diffusion_pytorch_model.safetensors - the second network. Ideogram 4 runs real classifier-free guidance, and this is the branch that provides the unconditional pass. Leave it out and you'll find out why the node's CFG default is 7 rather than 1.
• text_encoder/model.safetensors - the text encoder weights.
• vae/diffusion_pytorch_model.safetensors - the VAE.

Merge all four, plus a fifth ModelConfig on the same repo with pattern tokenizer/ into tokenizer_config. Yes, five nodes for one model. That's the cost of a pipeline that ships its own unconditional branch.

Install

# ComfyUI Manager: search "DiffSynth-ComfyUI" → Install → restart

pip install git+https://github.com/modelscope/DiffSynth-Studio.git
cd ComfyUI/custom_nodes
git clone https://github.com/modelscope/DiffSynth-ComfyUI.git
cd .. && python main.py

# nf4 is the small-card path here:
pip install "diffsynth[quant] @ git+https://github.com/modelscope/DiffSynth-Studio.git"

The fp8 repo is what the template uses and it's the friendlier of the two; nf4 exists for cards that can't hold fp8. Downloads happen on first execute, from ModelScope by default - Ideogram's repos are on HuggingFace, so if modelscope.cn is slow set download_source to huggingface on each of those five nodes.

Where people get burned

Skipping the unconditional transformer is the top one, and the failure is confusing rather than clean: you get an image, it's just much worse than the samples, because you're effectively running the model with no guidance to steer it.

The second is licensing, and it's worth being blunt about it. The Ideogram Non-Commercial Model Agreement covers the weights. Commercial use needs Ideogram's permission. That's also why there's so little LoRA/fine-tune tooling around it - trainers walked away.

Third is the standard pack-level trap: if the DiffSynth category in your node menu is empty, DiffSynth-Studio didn't install. The plugin imports it at startup, so a missing dependency means no nodes at all, not a partial install.