Llama.cpp Hardware Runtime Profile

If Model Profile is about which model you're running, Llama.cpp Hardware Runtime Profile is about which hardware you're running it on. It bundles seven settings that are pure machine-tuning - batch sizes, GPU offload, CPU threads, flash attention, memory mapping - into one typed connection for the compact Generate nodes. You create one per machine, connect it, and stop touching these knobs in every workflow. It's the "this is my 3090" node.

The profile combo holds named starting points, with GPU Full Offload as the default - which, conveniently, is also exactly what the compact Generate uses when you don't connect this node at all. So the first thing to know: this input is optional, and skipping it gets you a perfectly sane configuration. You connect it when the default doesn't fit your hardware.

The seven widgets mirror the things you'd tune in llama.cpp directly:

• n_batch (512) - logical prompt batch size. If you override image token ceilings in Generate, this must be at least the effective limit.
• n_ubatch (default 0) - physical batch size. The 0 means "use the llama.cpp backend default"; that's the deliberate escape hatch, so don't panic that it's not 512.
• gpu_layers (all / auto / CPU-only) - how much of the model offloads to the GPU.
• main_gpu (0) - which GPU index gets the main compute.
• n_threads (0 = let llama-cpp-python choose).
• flash_attention (auto / enabled / disabled).
• use_mmap (true) - memory-maps the GGUF while loaded.

The low-VRAM playbook is the obvious use: set gpu_layers to a partial value, or auto, and let llama.cpp decide what fits your card. For a 12GB card running a 14B GGUF alongside diffusion, that's the difference between a working workflow and a crash. The docs are careful to note n_ubatch=0 means "no explicit override is sent to llama.cpp," so you're never accidentally forcing a small physical batch you didn't ask for.

Wiring: output hardware profile goes to the hardware_profile input on either compact Llama.cpp Generate or Sequential Generate. Disconnect it and you silently revert to GPU Full Offload defaults - handy for A/B testing whether your tuning actually helped.

Install is the pack standard:

cd ComfyUI/custom_nodes
git clone https://github.com/craftingmod/ComfyUI-Ollama-ImageList

Restart ComfyUI after cloning, and remember the llama.cpp prerequisite shared across the pack: the JamePeng multimodal llama-cpp-python wheel installed into ComfyUI's own Python environment, plus ComfyUI 0.19.3+ for the V3 API.

A thin node, but a genuinely useful pattern. The trap to avoid: obsessing over n_ubatch and n_threads before you've verified the wheel and the model actually work. Get a generation running with the default GPU Full Offload first, then reach for this node when your card is the bottleneck - most people only ever need to touch gpu_layers and maybe n_batch.