Diffusers Generate Prior Tokens

GLM-Image isn't a normal diffusion model. It's two models in a trench coat: an autoregressive language model that reads your prompt and writes out a set of image "prior tokens" - a semantic blueprint of what to draw, roughly 256–4096 tokens of meaning and layout - and then a separate diffusion transformer that turns those tokens into actual pixels. Think of it as the LLM drafting the plan and the diffusion model building the house. DiffusersGenPriorTokens is the drafting stage, pulled out into a node so the two halves can run in different environments and at different times.

Why does that matter to you? Because it's the whole reason GLM-Image can render readable text, structured infographics, and dense layouts where a plain diffusion model mangles things. The LLM actually reasons about "three menu items, three prices" before a single pixel is denoised. This node gives you that reasoning stage on a ComfyUI graph.

What you feed it

• model_path - a full path to the GLM-Image model directory on disk (the default is F:/HF_Models/GLM/GLM-Image, a Windows path from the author's machine - you'll change it).
• diffusers_cond - required, and this is the sneaky bit. You don't need a pipeline here; you need the conditioning output of DiffusersTextEncode. This node augments that bundle rather than starting from scratch.
• prompt - multiline, defaults to the author's quality boilerplate.
• image (optional) - wire in an image for image-to-image.
• width / height - the resolution the autoregressive stage plans for, 256–4096 in steps of 64.

How it works under the hood

Here's the interesting design choice: the node doesn't run the AR stage in ComfyUI's process. It shells out to a separate Python subprocess (generate_prior_tokens_isolated.py) using the same interpreter. That subprocess loads a minimal GLM pipeline - text encoder, tokenizer, and the vision-language encoder only, no VAE and no transformer - runs generate_prior_tokens(), writes the result to a temporary .pt file, and the main process loads it back in and stuffs it into your diffusers_cond dict along with the width and height. Skipping the VAE and transformer is the memory trick: the AR stage alone is a fraction of the full model's footprint.

A few implementation facts worth knowing. The subprocess hardcodes bfloat16 and seed 42 - so the "randomness" in this stage is fixed and your seed widget on the sampling node is doing the actual varying. At startup it also sweeps the pack directory for stale temp_prior_tokens_*.pt files from interrupted runs and deletes them, which is a nice touch.

Output

A diffusers_cond bundle with prior_tokens added, wired straight into DiffusersSampling, which recognizes the key and passes the prior token IDs into the pipeline call. So the GLM chain is: DiffusersPipeline (with GlmImagePipeline) → DiffusersTextEncode → this node → DiffusersSampling.

Install

ComfyUI Manager, search "Comfyui-DiffusersUtils". Or manually:

cd ComfyUI/custom_nodes
git clone https://github.com/lrzjason/Comfyui-DiffusersUtils

Restart ComfyUI. For GLM support you also need the bleeding-edge HF stacks - the pack relies on classes that haven't shipped in stable releases:

pip install git+https://github.com/huggingface/transformers.git
pip install git+https://github.com/huggingface/diffusers.git
pip install git+https://github.com/huggingface/peft.git

One gotcha: the README's pip install -r requirements.txt will fail - there is no requirements.txt in the repo. The pack runs on ComfyUI's bundled diffusers/transformers/safetensors plus the git installs above.

Troubleshooting

If this node errors with an import failure in the subprocess, the likely culprit is an environment mismatch - the subprocess uses sys.executable, and if that interpreter can't see the freshly-installed diffusers, you get a confusing stack trace. That's exactly why the pack also ships DiffusersGenPriorTokensDebug, which runs the same logic in-process. If Debug works and this one doesn't, the subprocess can't find your deps; make sure you installed into the same Python that launches ComfyUI.