lopi999 LLM Node

LLMNode(lopi999) - shown in the menu as "lopi999 LLM Node" - runs real language models inside ComfyUI, on your GPU, with no API and no key. Drop a GGUF file or a full HuggingFace safetensors model into the right models folder, pick it from the dropdown, and you've got a text-generation node you can wire into prompt workflows, image captioning, subtitle rewriting, batch prompt expansion.

That's the role LLM nodes play in this ecosystem: they give the graph a brain. Rewrite a prompt, describe an image so it can feed back into an image model, generate captions for LoRA training. One trust note, because LLM nodes earned a bad name: the ComfyUI_LLMVISION malware shipped through fake API library wheels and exfiltrated browser data. This node is the opposite shape - it runs local models and touches no API libraries at all - but the standing rule still applies: only install a custom node you're willing to run as an executable.

How it works

When you add the node it scans models/clip, models/text_encoders, and models/LLM (plus any extra clip/text-encoder paths from extra_model_paths.yaml) for .gguf and .safetensors files and builds the model dropdown from what it finds. Then it picks a backend:

• .gguf → llama.cpp via llama-cpp-python. It estimates a context window from a rough 4-characters-per-token heuristic (+1024 tokens if you attach an image), streams the completion with a progress bar and a tok/s readout, and tries GPU offload starting at "all layers" and stepping down. If your llama-cpp-python wheel has no CUDA support, default/cpu silently fall back to CPU while cuda raises a clear error.
• .safetensors → HuggingFace transformers, which means the file has to live inside an HF-style folder - one with a config.json somewhere in its parent chain (up to six levels up). No config.json, no load; the node tells you exactly that. When an image is attached it loads via the image-text-to-text auto-class, so vision-language models work.

After generation, the output is post-processed: any <think>...</think> blocks get pulled into the THINKING output and the rest goes to MAIN. That's built for reasoning models like DeepSeek-R1-distill or Qwen3, and it's a nice touch.

Inputs and outputs that matter

• instructions (system prompt) and prompt (user message) - both multiline; this is where your actual text goes.
• model - the dropdown. It shows the model's path relative to your models dir, so you always know what you're running.
• device - default, cpu, cuda. default tries GPU and falls back gracefully.
• tokens and max_tokens - tokens is your context budget, max_tokens is the new text generated. If max_tokens exceeds tokens it gets clamped down to it, so set tokens generously.
• temperature (0.7), top_p (0.95), top_k (40), seed - standard sampling knobs; drop temperature toward 0.2 for deterministic rewriting work.
• image (optional) - an IMAGE input that unlocks vision models. Send the image and the prompt together.
• Outputs: MAIN (STRING) and THINKING (STRING), both wire into any text input in your graph.

Installing it

Manager search for "comfyui-lopi999-llm", or:

cd ComfyUI/custom_nodes
git clone https://github.com/LaVie024/comfyui-lopi999-llm
# restart ComfyUI

The catch: the pack's requirements install the entire NeMo toolkit, sacrebleu, and llama-cpp-python - and the LLM node never touches NeMo. You're paying for the sibling ASR node's heavy dependency just to get this one. The other gotcha is that pip install llama-cpp-python ships without CUDA by default, so GPU-accelerated GGUF needs a CUDA-enabled wheel; otherwise run on cpu/default and accept the slower pace.

Common issues

• (no models found) - nothing matching in the scanned roots. Put a .gguf or a safetensors-containing HF folder under models/LLM, then re-add the node to refresh the dropdown.
• cuda errors with "no GPU offload support" - your llama-cpp-python wheel isn't CUDA-enabled. Reinstall it with GPU support or switch to cpu/default.
• Safetensors fails with "missing config.json" - the error message names the fix: keep the file in models/LLM/<model_name>/ alongside config.json and the tokenizer files.
• THINKING is empty - normal for non-reasoning models. Only models that emit <think> tags fill it.
• Generation stops suspiciously early on GGUF - the context clamp kicked in; raise tokens.

It's one of the friendlier local-LLM options because one node covers GGUF, safetensors, and vision, and everything stays on your hardware. The NeMo tax is annoying if you only want the LLM half, but if you're looking for local text generation inside a graph without standing up a separate server, this does the job with fewer moving parts than most.