📋 V2 Local Text Model Loader

Not every LLM job in ComfyUI needs to see an image. Sometimes you just want a small local model to clean up a prompt, generate tags, rewrite a caption, or act as a text utility - and you don't want to run a server to get it. V2 Local Text Model Loader is the pack's text-only loading node: it pulls a plain GGUF language model into llama-cpp-python and hands it out as a TEXT_MODEL for the text generation nodes.

It's the non-vision sibling of V2 Local Vision Model Loader, and the differences matter. No mmproj to worry about, for a start - a text model is one file, full stop. That makes it the friendliest loader in the pack to get working. The flip side: don't feed it images. It's a language model; the analysis nodes are the vision ones.

The dropdown and what's in it

The model enum defaults to three registry picks, all Q8_0 and all, again, abliterated variants:

• ✗ Huihui-Qwen3-8B-abliterated-v2.Q8_0.gguf
• ✗ Huihui-Qwen3-4B-Instruct-2507-abliterated.Q8_0.gguf
• ✗ gemma-3-4b-abliterated.Q8_0.gguf

The ✗ prefix means "not downloaded yet" - pick one and the node pulls it from HuggingFace automatically on load, just like the vision loader. The ✓ prefix means it's already in your models folder. The 8B Qwen3 is the capable default; the 4B models are the "I want this to be snappy" options. A Q8 8B is a chunky download (~8GB), so budget accordingly.

The inputs that matter

• model - the dropdown above.
• n_ctx - context window, default 8192, up to 128,000 here. Bigger context = more input text and longer generations before truncation, at a VRAM and speed cost. For prompt rewriting and tag generation, 8192 is plenty.
• device - Auto, GPU, or CPU. Auto uses the pack's device optimizer to pick GPU-layer offload. On a machine where the GPU is busy diffusing, CPU is a legitimate choice for a small 4B model - slow but doesn't fight the sampler for VRAM.
• system_prompt - optional, multiline. Defaults to empty. This is where you set the model's persona once (e.g., "you are a tag generator") so you don't have to repeat it in every generation.

The output is a single model (TEXT_MODEL) that feeds V2 Text Generation. That's the whole pipeline: loader → generation node.

How it works, briefly

The loader scans your GGUF folders (LLM/GGUF, text_encoders, clip, clip/gguf, VLM_GGUF), filters out vision models so the text list stays clean - a fix the changelog specifically calls out, because earlier versions let Qwen-VL models leak into the text loader - and initializes llama-cpp-python with your context and device settings. The registry also has Gemma3 support baked in, so a Gemma GGUF in your folder gets recognized and listed.

Install

cd ComfyUI/custom_nodes
git clone https://github.com/walke2019/ComfyUI-GGUF-VLM.git
cd ComfyUI-GGUF-VLM
pip install -r requirements.txt

Restart ComfyUI; it's under GGUF-VLM → v2 → Text. The meaningful dependency is llama-cpp-python - if it's CPU-only on your Linux box, an 8B Q8 will be glacial; rebuild with CUDA or pick a 4B.

Troubleshooting

If a model you downloaded doesn't appear, it's in the wrong folder or the list predates it. If loading a ✗ model seems to hang, it's downloading - give it time, and check the console for progress. If generation is unbearably slow on a GPU machine, llama-cpp-python isn't using your GPU. And the changelog's honest note about Windows blue-screens with certain llama-cpp-python versions is worth remembering: if you hit one, update llama-cpp-python, update your GPU driver, or drop to CPU mode - in that order.