Scale-RAE Loader

If you've loaded a thousand checkpoints in ComfyUI, this node will still feel slightly off, because there's no single file to pick. Scale-RAE is two separate model folders - a generator and a decoder - that only work as a pair, and the Scale-RAE Loader is where you join them.

Scale-RAE is a research release from NYU (January 2026, "Scaling Text-to-Image Diffusion Transformers with Representation Autoencoders"). Instead of a VAE squeezing your image into a latent, the diffusion model works in the feature space of a frozen vision encoder - Meta's WebSSL DINO - and a separate decoder turns those features back into pixels. Different latent space entirely, which is why the model can be small and still coherent. Whether it's good is another question; you're here to run it.

What the loader is actually assembling

The generator folder is a Qwen2.5-1.5B chat language model with a 2.4B rectified-flow DiT bolted on as its "image head". The decoder folder is the RAE decoder that maps 256 encoder-space tokens back to pixels. Both halves have to live in the same encoder space - for this pack, that means the WebSSL generator plus webssl300m_decoder, and nothing else.

So the loader lists what's in models/scale_rae, builds the pair in the dtype and on the device you asked for, and hands you a handle. Only one model is resident at a time, so loading a different generator frees the previous one. Nothing is downloaded at run time.

The inputs you'll actually touch

• generator - a folder in models/scale_rae. The dropdown only lists folders that look like a real Scale-RAE generator: config.json with "model_type": "cambrian_qwen", the safetensors shards, and the tokenizer files.
• decoder - the matching *_decoder folder, recognised by "model_type": "vit_mae" plus model.pt.
• dtype - float32 is the official setting and burns about 17.4 GiB of VRAM peak. float16 drops that to roughly 9.5 GiB and stayed closest to float32 in the author's tests; bfloat16 uses the same memory but visibly changed images. The RAE decoder always runs in float32 regardless, and CPU always runs float32.
• device - auto, cuda, or cpu. CPU works and is slow (about 52 s per image on a good desktop chip).
• keep_loaded - on by default. Turn it off if you share the GPU with something else and don't want a ~10 GB tenant.

Output is a single model handle (type SCALE_RAE_MODEL) that goes into Scale-RAE Text to Image. No VAE to load, no CLIP, no separate text encoder - the language model is the encoder, baked into the checkpoint. If you're used to wiring a VAE by hand it feels like a step is missing. It isn't.

Install

Use ComfyUI Manager and search for Scale-RAE, or do it manually:

cd ComfyUI/custom_nodes
git clone https://github.com/hiroki-abe-58/ComfyUI-ScaleRAE
python -m pip install -r ComfyUI-ScaleRAE/requirements.txt

The requirements file is genuinely light - transformers>=4.50.3, timm, einops, safetensors, most of which ComfyUI already has. The weight here is the models: about 17.6 GB for the pair.

python custom_nodes/ComfyUI-ScaleRAE/tools/download_models.py --models-dir ComfyUI/models/scale_rae

That script pins the revisions the pack was verified against and SHA-256s every file. --verify-only re-checks folders you already have, which is the fastest way to prove a half-finished download isn't why your dropdown is empty.

You also need a ComfyUI new enough for the V3 node API - the pack is written against comfy_api.latest with no NODE_CLASS_MAPPINGS, so older builds load the folder and show nothing. v0.38.0 was the tested floor. Restart, then open a workflow from workflows/gui/.

Where people get burned

The dropdown is empty. Nine times out of ten the folder is in the wrong place (models/scale_rae/<folder>, not models/scale_rae itself) or the download stopped partway and the tokenizer files or model.pt never landed. The loader is strict on purpose - a mismatched pair produces noise, not an error message, so it filters instead.

"Not enough VRAM" on a card that seemed fine. At float32 you need about 18 GB free. On Windows there's a second bill: the README measured commit charge rising roughly 22 GiB during loading, so a machine with a small pagefile can fail while the GPU looks idle. Try float16 first; it halves the footprint with the smallest quality cost.

Everything runs but the images are 224×224. That's the WebSSL decoder, not a setting you missed. Upscale afterwards like you would any small generation - see the upscaling notes in the KB for what to reach for.

One non-technical caveat worth knowing before you build a product on it: the pack and the weights are MIT, but the frozen WebSSL encoder they were trained in the feature space of is CC-BY-NC-4.0, and the model cards don't resolve whether that reaches models trained on its features. Hobby use, no worries. Commercial use, go read it yourself.