qwen_chat_model

qwen_chat_model drops a full, locally-running Qwen2.5 or Qwen3 chat model into your node graph. The name might make you expect an API wrapper, but it's the opposite of that: no key, no account, no cloud round-trip. The weights live in your ComfyUI/models/LLM folder and inference runs on your own GPU.

Why would you want an LLM sitting in the middle of a diffusion workflow? Because "LLM-assisted prompting" stopped being a browser-tab habit and became a node. Have a model translate your half-formed idea into the structured prompt your image model actually responds to, and you get results without writing prompts by hand. Qwen is a natural fit here - half the image models people run these days (Z-Image, Klein, Anima) literally use a Qwen3 LLM as their text encoder, so you're having the same family of model that reads your prompt write it. It's a translation between two things that speak the same language.

How it works

Under the hood this is a stock Hugging Face pipeline: the node loads a checkpoint with AutoModelForCausalLM, device_map="auto", bf16 on CUDA, wraps your system and prompt in a chat template, and runs autoregressive generation. On first use it downloads the model itself (via snapshot_download) into ComfyUI/models/LLM/<model_name>/ - there's no separate "download the model file" step. A shared global cache keeps loaded models in VRAM between runs, so calling the same node twice is fast; see the toggles below.

The inputs that actually matter

• system / prompt - your instructions and request. Multiline strings, so you can paste a whole job description in.
• model - the checkpoint dropdown. Default is Qwen3-4B-Instruct-2507, which is the sweet spot for this: smart enough for prompt work, small enough to run comfortably. The Qwen3 options split into Thinking and Instruct variants - thinking ones reason out loud before answering (more verbose, slower, occasionally sharper).
• quantization - none / 4bit / 8bit, via bitsandbytes. On a normal card you want 4bit for anything above 4B; if bitsandbytes fights you, none is the escape hatch.
• temperature, max_new_tokens (default 512), seed - -1 means "don't set the seed", so you can run it semi-randomly each time.
• model_loaded_permanently and offload_after_used - pin the model in cache, or unload it and free VRAM after each run. Leave both off and the cache evicts least-recently-used models once you have more than 2 loaded (bump that with the QWEN_MAX_LOADED_MODELS env var).

The single output is a STRING - the model's reply - which wires straight into anything that takes text, like a prompt box or a text concat node.

Installing it

The README documents the manual path, and ComfyUI Manager should find it too (registry name "ComfyUI CALL LOCAL Qwen Chat Models"). Manual, from inside your ComfyUI virtualenv:

cd ComfyUI/custom_nodes
git clone https://github.com/ConstantlyGrowup/ComfyUI_Qwen_chat_models.git
cd ComfyUI_Qwen_chat_models
pip install -r requirements.txt

Then restart ComfyUI and look under the Comfyui_Qwen category. Heads up: the dependency list is the full transformers stack - transformers, accelerate, bitsandbytes, qwen-vl-utils, optimum. Even though this is the text-only node, the pack imports qwen-vl-utils at module level, so you install the whole thing no matter which node you use. It's a chunky install, not a one-liner.

Common issues

• First run stalls - that's the model downloading (several GB). Pre-download it yourself into ComfyUI/models/LLM/Qwen3-4B-Instruct-2507/ if you hate surprises.
• Cannot load pinned model - you pinned more models than QWEN_MAX_LOADED_MODELS (default 2) allows. Unpin some or raise the env var.
• 4bit/8bit errors - bitsandbytes doesn't match your CUDA setup. Set quantization to none.
• VRAM won't drop / OOM - turn on offload_after_used, lower max_new_tokens, or pick a smaller model.
• Qwen3-8B-Instruct fails to download - that one's a gated Hugging Face repo: you must accept its terms on the model page and have a HF_TOKEN set. The other six choices download fine.

One honest caveat about the whole genre: local-LLM nodes are one of the known malware vectors in the ComfyUI ecosystem (the LLMVISION incident is the cautionary tale), and this pack is a small, low-profile one. Install from the official repo, glance at what you're running, and don't grab random "LLM vision" packs. Otherwise - free, local, no API key. That part of the name is not a lie.