DiffSynth: ERNIE Image Loader

ERNIE-Image is Baidu's Apache 2.0 8B: a 36-layer single-stream DiT with a Mistral3 text encoder, built for layout and text-in-image rather than photorealism. It had a two-week spike in April 2026, lost the attention race to whatever else shipped that month, and never grew an ecosystem - no LoRA flood, no fine-tune wave. If you're here, it's probably because you want a layout-precise render or because Apache 2.0 matters to you. Both are good reasons.

This Loader is your file declaration and VRAM strategy for it. Required: model_configs (the merged list from DiffSynth: Merge ModelConfigs), torch_dtype (bfloat16 / float16 / float32), device (cuda / cpu). Optional: tokenizer_config - a single ModelConfig for the tokenizer directory, easy to forget and fatal to skip - and vram_limit from a VRAM Limit node. Output: pipe → DiffSynth: ERNIE Image Inference.

The Turbo wiring is deliberately mixed

Copy this from the template rather than improvising, because it's the part people get wrong. The SFT template declares three files from PaddlePaddle/ERNIE-Image: transformer/diffusion_pytorch_model*.safetensors, text_encoder/model.safetensors, vae/diffusion_pytorch_model.safetensors, plus PaddlePaddle/ERNIE-Image → tokenizer/ into tokenizer_config.

The Turbo template changes exactly one of those four lines: the transformer comes from PaddlePaddle/ERNIE-Image-Turbo with the same transformer/diffusion_pytorch_model*.safetensors pattern, while the text encoder, VAE and tokenizer still come from the base repo. That's not a copy-paste slip - the Turbo release is the distilled transformer and you keep the base model's text stack. Swap all four and you'll spend twenty minutes wondering why the loader can't find a text encoder.

Install

# ComfyUI Manager: search "DiffSynth-ComfyUI" → Install → restart

pip install git+https://github.com/modelscope/DiffSynth-Studio.git
cd ComfyUI/custom_nodes
git clone https://github.com/modelscope/DiffSynth-ComfyUI.git
cd .. && python main.py

# the base checkpoint's own advice is 24GB. If that's not you:
pip install "diffsynth[quant] @ git+https://github.com/modelscope/DiffSynth-Studio.git"

That last line matters here more than for most nodes in this pack. Baidu's stated requirement is 24GB for the base model; the quantization extras plus a CPU-offload VRAM Config are how you get in under it, and the shipped templates already run a VRAM Config with offload on both the offload and onload stages. Downloads happen on first execute, from ModelScope unless you flip download_source.

Where people get burned

Grid artifacts. It's the defect that sank ERNIE's reputation, it shows up in most generations, and no amount of prompt work removes it - if it's a dealbreaker, find out before you build a workflow around it. The second trap is resolution: 1024×1024 is native and quality degrades above it, so asking for 2048 on the Inference node is not a free upgrade. Third, the model skews hard toward Asian faces out of the box; the community answer is a LoRA, which is awkward for a model with no ecosystem. And the usual pack-level one: an empty DiffSynth category in the node menu means DiffSynth-Studio didn't install, because the plugin imports it at startup and registers nothing without it.