SLM: Generator

SLMGenerator ("SLM: Generator") is where the pilcothink pack's text side actually produces text. You feed it a model object from the pack's model selector plus a prompt, and it runs the small language model locally and returns the answer as a STRING. SLM here means small language model - the whole selling point is that this stuff runs on your box in the middle of a ComfyUI graph, no API key, no cloud.

It's wired for the current Qwen3-0.6B only. Qwen3 models are trained to emit a <think> block before the answer, and this node handles that for you: it decodes the output and strips everything up to the </think> token, so the SLM_TEXT you get is the actual answer, not the model's internal reasoning. That's a real convenience - a raw Qwen3 generation dumps reasoning all over your downstream nodes if you don't parse it.

How it works

The node calls into the pack's Qwen3 loader, which uses AutoModelForCausalLM + AutoTokenizer from transformers, applies the Qwen chat template, and generates with the sampling parameters you set. Then it finds the </think> token ID (151668 in Qwen3's vocab) and cuts everything before it, returning only the response.

The pack also runs a single-resident model cache. With use_model_cache on, the first run loads the model and keeps it in VRAM so subsequent runs are fast; loading a different model unloads the old one automatically - only one model lives in memory at a time. Flip use_model_cache off and it unloads after every run, which is slow but frees VRAM between executions.

The inputs that matter

• slm_model - the SLM_MODEL object from "SLM: Model Selector". Required, force-input.
• user_prompt - your text. Feed it from a String Text node or the pack's RAG node.
• max_new_tokens - 1024 by default, up to 8192.
• temperature (0.85), top_p (0.9), top_k (0), repetition_penalty (0.0) - standard sampling knobs; zero means "not applied."
• use_model_cache - default on.

The single output is SLM_TEXT, a STRING.

Install

Ships in the comfyui_pilcothink_VisionSLM pack:

cd ComfyUI/custom_nodes
git clone https://github.com/gpdev-Pilcothink/comfyui_pilcothink_VisionSLM

Restart, or install via ComfyUI Manager (search comfyui_pilcothink_VisionSLM). You'll want transformers ≥ 4.56.2 for the Qwen3 chat template.

Gotchas

First run downloads the model (Qwen3-0.6B is only a bit over 1 GB, so this isn't the painful one - the vision models are) into the pack's own Models/SLM/ folder. If you choose cpu as the device in the selector, the README says float16 only; other dtypes will error. And when text comes back oddly short or empty, check your max_new_tokens - 0.6B models fill their budget fast and the node trims thinking before it returns anything, so a too-small budget can leave you with an answer that's mostly gone.

Honest positioning: a 0.6B model is a toy by 2026 standards. It's great for quick in-graph labeling, routing, or prompt transformation where latency matters more than brilliance, and for that it's genuinely handy. If you need real reasoning quality, the pack's Ollama and vLLM paths exist precisely for that.