LLM Pipe Loader - Ostris

LLM Prompt Upsampling needs a real language model to do its rewriting, and something has to get that model onto your GPU. That's this node: LLM Pipe Loader - Ostris takes a HuggingFace model name, downloads it, loads it in 4-bit quantization, and hands you a ready-to-use LLM_PIPELINE for the upsampler. It's the expensive part of the pack, in every sense.

One input, one output:

• model_name - a HuggingFace model ID, defaulting to HuggingFaceH4/zephyr-7b-beta. Type any causal LM that has a chat template and it'll try to load it.
• llm - the LLM_PIPELINE object that wires into the Prompt Upsampling node.

Mechanically, it's a straightforward transformers pipeline. It builds a text-generation pipeline from AutoModelForCausalLM plus a tokenizer, quantized with BitsAndBytes load_in_4bit (double quantization, fp16 compute), and downloads the weights into ComfyUI/models/llm so you only download once. A few smart details in the source: it falls back to a slow tokenizer if the fast one fails, it only enables trust_remote_code for stabilityai-prefixed models (a decent default-security instinct), and if the model has no chat template at all it injects a generic one so stablelm-style models still work.

The thing to internalize before you click anything: the default model is a 7-billion-parameter LLM. First run downloads several gigabytes from HuggingFace into ComfyUI/models/llm. Then it loads that model into VRAM, and despite the 4-bit quantization the README warns it's still "VRAM hungry." A 4-bit zephyr-7b will comfortably eat 4–6GB of VRAM on top of whatever diffusion model you're already running. On a 12GB card that's the difference between fitting and not. If you only have the memory for one big model at a time, think hard about whether local prompt rewriting is what you want to spend it on.

Install, from the pack README:

cd <your_comfyui_folder>/custom_nodes
git clone https://github.com/ostris/ostris_nodes_comfyui.git
cd ostris_nodes_comfyui
pip install --upgrade -r requirements.txt

or via ComfyUI Manager (search Ostris Nodes) and restart. The requirements file is just transformers and bitsandbytes, and they matter here specifically. Two gotchas worth knowing, both from how the pack actually loads:

• The whole pack depends on these imports. The pack imports its LLM module unconditionally at startup, so if transformers or bitsandbytes isn't installed, the entire pack fails to register any nodes - including the ones that have nothing to do with LLMs. Run the pip install line even if you only want the seed node.
• bitsandbytes is historically the fragile part, especially on Windows, where old versions had real trouble locating the CUDA runtime. Modern versions are much better, but if the loader crashes before any model is even named, bitsandbytes is your first suspect.

If the default model is too big, you can point model_name at something smaller - a 1B or 3B instruct model will still do prompt rewriting, just less eloquently. That's the one real knob this node gives you for tuning the VRAM cost. Everything else about it is "load a big model, wait." It does that job fine, and the only consumer of the pipe is the upsampler, so the whole thing stands or falls on whether you actually want that feature.