Qwen Thinking Loader (Merged or Sharded)

The name is a mouthful and the display name - "Qwen Thinking Loader (Merged or Sharded)" - is worse, but the promise is simple: this node drops a real, local LLM into ComfyUI so you can refine prompts inside the graph instead of tabbing out to a chatbot. It calls no API, needs no key, and after the one-time model download it works completely offline. The "Merged or Sharded" bit just tells you it accepts both a single safetensors file and a sharded multi-file set.

This is the heavy half of the two-node Qwen-llm-loader pack. The loader is the ~8GB part you download once; its sibling, the Qwen Thinking Prompt node, is the brains that turns your raw prompt into something usable. If you're new to the whole "let a local LLM write my prompt" idea: it's not a gimmick anymore. Prompt enhancers went from a browser-tab habit to an in-graph node workflow, and on the 2026 generation of LLM-encoded image models (Z-Image, Anima, the whole class that reads your prompt as a message rather than a CLIP vector) an LLM writing your instructions is a translation between two things that speak the same language.

How it works

Under the hood it's a thin wrapper around Hugging Face transformers. Point it at a folder and it runs AutoModelForCausalLM.from_pretrained with local_files_only=True (offline), trust_remote_code=True, use_safetensors=True, and SDPA attention. Despite the Qwen branding, it's a generic causal-LLM loader - the author says it can handle almost any model that ships the right config.json, so Qwen is the default, not the ceiling.

The inputs that matter

• model_folder - pick the subfolder under ComfyUI/models/qwen/. The list is built when the node is created, so if you see "No model folders found", drop your model in first and refresh.
• keep_loaded - true keeps the model resident so repeated runs are fast. false tells ComfyUI to drop it when the run finishes, giving the VRAM back to your image model. The author keeps it off-loaded; if your checkpoint and a 4B LLM are fighting for one card, so should you.
• use_multi_gpu - defaults to true (device_map="auto"), which is great on 2+ GPUs and pointless on a single card. The README explicitly says to turn it off on single-GPU setups like an RTX 3090.
• dtype - bf16 is the default and the right call. fp32 doubles the memory for no visible gain.
• compile_model - torch.compile for a real speedup, at the cost of a slow first generation while it warms up.

Installing it

ComfyUI Manager → Install Custom Nodes → search Qwen-llm-loader, or:

cd ComfyUI/custom_nodes
git clone https://github.com/capitan01R/Qwen-llm-loader

Then restart ComfyUI. The README promises no extra pip packages, and it's telling the truth - it rides on the transformers and torch ComfyUI already ships. The actual work is the model. Download all files from Qwen/Qwen3-4B-Thinking-2507, or the author's pre-merged qwen-thinking-merged, and put them in:

ComfyUI/models/qwen/your-model-folder/
config.json
tokenizer.*
*.safetensors
model.safetensors.index.json

Restart again and the folder shows up in model_folder.

Where people get burned

The most common failure is the easiest: the loader only sees subfolders of models/qwen/, and only on node creation. Dump the files loose into models/qwen/ and you'll stare at "No model folders found" forever. Add the folder, restart, done.

Single-GPU users routinely leave use_multi_gpu on and wonder why loading feels weird - flip it off. And if you're on an old clone of the pack, update it: v1.0 had a tokenizer NameError bug that v2.0 fixed, along with torch.compile and SDPA support. It's a one-commit repo, so you're not losing anything by staying current. That first compile warmup is also expected, not broken - let it cook once and the next runs are noticeably faster.