Load Qwen-Scope SAE

Load Qwen-Scope SAE is where the magic of this pack actually lives. A Qwen LLM's residual stream is a soup of 2048–5120 numbers per token - unreadable. A sparse autoencoder (SAE) decomposes that soup into a sparse dictionary of features, each one a direction the model "uses" in a fairly interpretable way. This node loads one layer of Qwen's own pre-trained SAE packs so the other QwenScope nodes can translate model innards into things you can look at and steer.

This is the piece that makes the pack worth installing. Qwen released TopK SAEs for every layer of seven base models (Qwen3 and Qwen3.5, dense and MoE), and this node is the door into all of them.

What the dropdown actually means

The sae_pack combo lists all 14 released packs, named like Qwen3.5-2B · W=32K · L0=50. Two numbers matter:

• W is the dictionary width - how many features the SAE has to work with. 32K is standard; the MoE variants go to 128K.
• L0 is the TopK sparsity - how many non-zero features each token keeps. L0=50 means each token is described by its 50 strongest features; L0=100 is denser and finer-grained, at the cost of more compute.

Same base model, two sparsities, two different granularities of microscope. Start with L0=50; it's the default and the least noisy.

The other inputs are simple: layer (which transformer layer's residual you're reading - check your model's layer count; the small Qwen3 bases have a couple of dozen, the 8B sits in the mid-30s), and device (auto does the right thing).

How it works

Packs are stored as layer{n}.sae.pt files, one per layer, holding encoder/decoder weights. Here's the part people like: this node downloads only the single layer file you request, on first use - no multi-gigabyte pack upfront. It's LRU-cached (up to 8 layer shards), so hopping between layers in one session is cheap. If you see a "fetching layer12.sae.pt from Qwen/..." line in the console, that's normal and it's not downloading the whole model.

Outputs are sae (QSCOPE_SAE) - feed it to Analyze, Analyze CLIP, or Feature → Direction - and an info string confirming d_sae, d_model and L0. The d_model line is the one to sanity-check: it must equal the hidden size of the model or CLIP encoder you pair it with.

Installing it

ComfyUI Manager: search "ComfyUI QwenScope". Or:

cd ComfyUI/custom_nodes
git clone https://github.com/Nynxz/ComfyUI-QwenScope

restart, and you're set - no manual model download. Dependencies (transformers, huggingface_hub, safetensors, Pillow, numpy) are mostly things you already have. If a repo is gated, set HF_TOKEN before launching ComfyUI.

Common issues

The one that bites everyone is the dimension mismatch: pick an SAE pack whose d_model doesn't match the model/encoder you're analyzing and you get a hard error with a "did you mean…?" hint listing packs that fit your hidden size. For image flows, remember FLUX.2 [klein] 9B uses a Qwen3-8B encoder - grab the Qwen/SAE-Res-Qwen3-8B-Base-W64K-L0_50 pack, not a 1.7B or 27B one. And don't set layer higher than your model has layers; you'll get an index error, not a graceful message.