Train Lens (SAE)

The contrastive trainer gives you a direction that separates concepts, but it's opaque - a black box vector you have to trust. This node is the "show me your work" version. It trains a Sparse Autoencoder on the text encoder's residual stream, decomposes the concept into specific sparse features, and saves a lens whose metadata records exactly which feature indices define "cinematic" and by how much. It's slower - roughly five minutes instead of thirty seconds - but it's the pack's bridge into real mechanistic interpretability, the same SAE approach from Bricken et al. (Anthropic, 2023) that's been the backbone of LLM interpretability research.

How it works

Full pipeline, straight from the pack's HOW_IT_WORKS.md:

1. Hook into the text encoder's residual stream at layer 22 (~60% depth - the point where syntactic processing gives way to semantic representation), and collect token-level activations from ~500 diverse prompts (~15k vectors).
2. Train an SAE on those activations: 8× expansion on Qwen's 2560d gives 20,480 sparse features, with unit-norm decoder columns and L1 sparsity.
3. Run your positive and negative concept texts through the model + SAE, compute mean(pos_features) - mean(neg_features), and take the top-K features by absolute difference.
4. Reconstruct a direction from those features via bias-free decoding (decode_sparse).
5. Optionally blend with a contrastive direction (on by default) for better separation.

The payoff is that the lens carries sae_features in its metadata - indices like [4821, 12033, 7944, ...] - plus sanity metrics like how well the SAE direction matches the raw centroid difference. That's the thing contrastive lenses just don't give you.

The inputs that matter

• positive_texts / negative_texts - same discipline as contrastive: one per line, similar scenes without the concept for the negatives, 10+ pairs recommended.
• layer - 22 by default. Earlier layers = lower-level features, later = more abstract. 22 is the documented sweet spot for style/aesthetics.
• sae_expansion - 8 (20,480 features). Higher is more fine-grained but slower and a bigger file.
• sae_features - how many top features go into the direction. 30 default; more = richer but noisier.
• refine_contrastive - on by default; blends in a contrastive direction so you get interpretability and robust separation.
• sae_save_path / sae_load_path - this is the good stuff. Train the SAE once, save it, and reuse it across concepts to skip the ~5-minute training step for every subsequent lens.
• transcoder_repo - a HuggingFace repo for pretrained 64× transcoders (e.g. mwhanna/qwen3-4b-transcoders, 163,840 features trained on ~1B tokens). This is the shortcut: skip training a small SAE entirely and use a much larger, better-quality one. It downloads on first use.

Output is lens_path - the absolute path to the saved lens, which also lands in lenses/ for the dropdown. The exact same training is available from the CLI with python tools/lens_factory.py sae <preset> (add --sae-save ./my_sae.pt to cache the SAE).

Installing

Standard for this pack - ComfyUI Manager search "Concept Steer", or clone https://github.com/Nynxz/ComfyUI-ConceptSteer into custom_nodes/ and restart. Training needs transformers and safetensors (pip install transformers safetensors) and ~10 GB VRAM for Qwen. No pretrained lenses ship with the pack - everything is generated locally.

Common gotchas

• Five minutes feels like forever on the first run because the SAE activation collection + training is the slow part. Save with sae_save_path and reuse it; subsequent lenses are ~30s.
• The feature indices aren't human-readable yet. The README is direct about this: without a feature labeling pipeline, [4821, 12033, ...] is just numbers. That's what the Feature Dictionary / Feature Map / Feature Gate / Feature Probe nodes in the pack are for - this lens is one stop on a longer interpretability workflow.
• Refine-contrastive is on by default for a reason. Turning it off gives you a purer SAE direction but weaker separation. Turn it off only if you specifically want the raw SAE vector.