DiffSynth: Ideogram 4 Inference

Ideogram 4 does two things better than anything open: text in images that actually spells what you asked for, and character and IP recall deep enough to be slightly unsettling. It's also the one model in this pack where the prompt field wants a structured document rather than a sentence - which is the entire reason comparison threads about it look like forum posts full of code.

Required sockets: pipe (from the Ideogram 4 Loader), prompt, seed, num_inference_steps (default 50), height, width (1024). One output: image → Save Image.

Prompting in JSON

The model was trained on structured prompts: objects with bounding boxes and short descriptions, composed into a layout. A community comparison thread against Z-Image Base and Klein 9B consists mostly of people pasting prompts shaped like this:

[
  {"type": "obj", "bbox": [0, 0, 420, 1000], "desc": "Cool-toned concrete facade in the upper half, hard shadow line across the middle"},
  {"type": "obj", "bbox": [0, 1000, 1000, 1480], "desc": "Orange-lit urban grid in the lower half, dense rooftops and traffic streams"}
]

That's the shape. Bounding boxes roughly at the scale the model expects, one line of natural language per region. Prose prompts will produce images, but you're throwing away the interface the model was built around. Resolution wants multiples of 16 between 256 and 2048.

The settings

• cfg_scale - default 7. High by 2026 standards and it's not an SD-era default leaking through: because the loader is expected to declare a separate unconditional transformer, this pipeline runs classic two-pass CFG. Raise it and you're steering harder with a real negative branch; lower it toward 1 and you're back to the faster, more distilled feel of a modern model.
• negative_prompt - live at CFG 7. This is one of the rare 2026 models where a negative actually does work, so use it.
• num_inference_steps - 50 is the node default and the template runs 48. Community settings that were circulating at release were nearer 28 steps at around 1.5MP, which is a good place to start if 50 feels slow.
• input_image + denoising_strength - img2img. Strength defaults to 1.0, so nothing happens to your input until you lower it.

One gotcha that isn't a bug

Ideogram 4 has a refusal image trained into the weights. Certain prompts return a canned, oddly consistent-looking picture instead of an error or a blank. When you hit it, it's the model declining, not your sampler failing - and no CFG value will argue it around. Worth knowing before you spend an hour tuning around it.

Install

# Manager: search "DiffSynth-ComfyUI" → Install → restart
pip install git+https://github.com/modelscope/DiffSynth-Studio.git
cd ComfyUI/custom_nodes && git clone https://github.com/modelscope/DiffSynth-ComfyUI.git

Template tab → the DiffSynth Ideogram 4 fp8 workflow is the working reference, including all four model files plus the tokenizer. Downloaded weights land on first execute; the fp8 repo is several GB, so don't sit there watching the progress bar.

Troubleshooting

If output quality is far below the samples, check the unconditional transformer ModelConfig is present and merged - without it there's no real guidance pass. If your text renders as gibberish, that's usually a coupling problem rather than a settings one: keep each string in its own desc, and quote it exactly. If a specific prompt keeps returning something oddly generic and consistent, you've found the refusal image. And memory-wise, nf4 is the variant that fits a single 24GB GPU; fp8 needs a bit more headroom, so the VRAM Limit buffer and CPU offload are your levers before you resize the canvas.