🖥️Local LLM Loader

If GGUF is the compressed, VRAM-friendly way to run a local model in this pack, Local LLM Loader is the other one - the standard transformers-format loader, for a model stored as a folder of weights rather than a single quantized file. The pack's own description of it is blunt about that boundary: "the model must not be in GGUF format but instead be an LLM model stored in a folder." If you're loading a .gguf file, this is the wrong node - you want one of the GGUF loaders instead.

Inputs, straight from the node's own tooltips

This node is unusually well documented for a comfyui_LLM_party node - its author actually wrote tooltips, so there's no guessing involved here.

model_name_or_path accepts either an absolute path to a local model folder, or a Hugging Face repo ID typed directly - the tooltip gives lllyasviel/omost-llama-3-8b-4bits as the example. That second option matters more than it looks: point this at a repo ID you've never downloaded and the node will pull it from Hugging Face on first run, no separate download step required.

device controls where the model runs, and the tooltip spells out exactly what auto does: it tries cuda first, falls back to mps (Apple Silicon) if that's not available, and falls back to cpu as the last resort. Leave it on auto unless you have a specific reason to force one.

dtype picks the numeric precision, and again the tooltip is direct: auto picks a sensible default, or you can force float32, float16, bfloat16, int8, or int4. This is your main lever for trading quality against memory when you're not using a GGUF quant - int8/int4 shrink the model dramatically at some cost to output quality, similar in spirit to a GGUF quant tier but implemented through a different quantization path (bitsandbytes-style, not llama.cpp).

is_locked (default on) stops the model from being reloaded on every graph re-run - genuinely important for anything multi-billion-parameter, since reloading from disk or re-downloading is slow. The tooltip explains the escape hatch too: turn it off and pair it with this pack's clear-GPU-memory node when you actually want to force a fresh load, for example after swapping to a different model path.

Outputs

Two, both tooltipped: model, "the loaded model," and tokenizer, "the loaded tokenizer." Both matter - a transformers model on its own can't turn text into tokens or tokens back into text, so anything downstream expecting to run inference against this model needs both wires connected, not just model.

Where it fits

This is the loader behind the pack's "start_with_LLM_local" example workflow, and the one to reach for when you want the model in its original, unquantized (or lightly quantized via dtype) form - useful when a GGUF conversion of the model you want doesn't exist yet, or when you specifically need transformers-level compatibility for something further down the chain (this pack's LoRA loader, for instance, expects a model object from this loader, not from the GGUF path).

Installing it

• ComfyUI Manager: search "comfyui_LLM_party", install, restart ComfyUI.
• Manual: cd ComfyUI/custom_nodes && git clone https://github.com/heshengtao/comfyui_LLM_party, then pip install -r requirements.txt inside your ComfyUI Python environment, restart.

transformers is the dependency this node actually leans on, and it's part of the pack's shared requirements.txt - no extra install step needed for this specific node, but it's worth confirming that install went through cleanly if this node is the one giving you trouble, since it's one of the heavier packages in that file.

Common issues

The Hugging Face repo-ID path is convenient but silent about failure - a typo'd repo ID or a gated model you haven't accepted the license for on huggingface.co will fail the download with an error that doesn't always make the actual cause obvious. If you're loading a gated repo (common for some Llama variants), you need to have accepted its terms on the Hugging Face website and be logged in via the huggingface-cli in the same environment ComfyUI runs in - the node itself won't prompt you through that.

VRAM is the other predictable failure. A full float16 load of anything above roughly 7–8B parameters is a lot to ask of a consumer card; if you're getting out-of-memory errors, drop dtype to int8 or int4 before assuming you need different hardware - that's exactly the lever this node gives you for the tradeoff. And if you're iterating on model_name_or_path and the change doesn't seem to take effect, check is_locked - that's the tooltip's whole point, and it's the most common reason a "reload with a different model" attempt silently keeps using the old one.