Analyze Prompt (SAE)

Analyze Prompt (SAE) is the node you run when you want to see what a Qwen LLM "fires" on a given sentence - which of its ~32,000+ learned features light up, and how hard, for each token. It's the workhorse capture node of the QwenScope pack: everything downstream in the Inspect category (Feature Heatmap, Token Activation Strip, Top Features) is designed to sit right after it.

Think of it as a very specific kind of prompt analysis. Not "which words does the model pay attention to" - that's attention visualization. This is feature analysis: after running your prompt through the model, it grabs the residual stream at the SAE's layer and encodes it through the SAE, producing a sparse list of active features per token. If SAEs are the theory, this is the practical "read the model's mind" step.

How it works

Three steps, all visible in the console if you're curious: the prompt is tokenized and run through the model, a forward hook captures the residual hidden state after the SAE's transformer layer, then encode_features projects that through the SAE's encoder matrix and keeps the top activations. The top_k input (default 64) controls how many features survive per token; set it to 0 and you get plain ReLU - everything above zero, which is noisier and slower but shows you the full picture. Higher top_k = more detail, more clutter.

The inputs that matter

• model - the QSCOPE_MODEL from Load Qwen Model. The base model and the SAE must come from the same family (matching d_model), or the math silently makes no sense.
• sae - the QSCOPE_SAE from Load Qwen-Scope SAE. The node uses the layer you chose there, so to analyze a different layer you change the loader, not this node.
• prompt - the text. Anything. Single sentences are the sweet spot; very long prompts just give you more tokens to wade through.
• template - raw by default. think / no_think wrap the prompt in Qwen's ChatML think-mode tags, which matters if you're analyzing the same input format you'd actually generate with.
• skip_first_token - drops the BOS/instruction token from the output. Usually leave it off; useful when the first token is just boilerplate.

Outputs: features (QSCOPE_FEATURES) carries the token list, per-token feature activations, layer, and SAE label - wire it straight into the three inspect nodes. info tells you nonzero/tok, a handy one-line summary of how sparse the firing was (low single digits to ~50 at L0=50).

Installing it

Part of the same pack, one install: ComfyUI Manager, search "ComfyUI QwenScope" - or:

cd ComfyUI/custom_nodes
git clone https://github.com/Nynxz/ComfyUI-QwenScope

then restart. Needs transformers, huggingface_hub, safetensors, Pillow, numpy; you almost certainly have them. The SAE layer file downloads lazily on first run.

Common issues

If nothing shows up in the visualizers, check the wire: this node outputs QSCOPE_FEATURES, and only the QwenScope inspect nodes accept that socket type - a generic image or tensor node won't take it. If you're analyzing a sentence and get a d_model mismatch error, the model and SAE are from different families (2B vs 8B etc.) - that's the mismatch message doing its job. And if you want to analyze image model text encoders (FLUX.2, Qwen-Image), don't use this node - that's what Analyze CLIP (SAE) is for.