Local AI Model (Advanced)

Local AI Model (Advanced) is the same idea as the simple Local AI Model node, but with every lever exposed. If you're happy with the safe defaults, you don't need this node - the simple one sets them for you. The moment you're running low on VRAM, pinning a model to a second GPU, or keeping a server resident for speed, this is where you go. It outputs a single config socket (GGUF_LLM_CONFIG) that wires into the other "Advanced" nodes - Local AI Generate, Prompt Enhancer (Advanced), the captioners.

What each of the required inputs actually does:

• model - the GGUF dropdown from ComfyUI/models/LLM. Still the entry point; nothing works without a real file here.
• llama_server - "auto" by default. Put a full path here to force a specific llama-server executable. The pack's search order otherwise is PATH, LLAMA_SERVER_PATH, C:\llamacpp\, and the pack's own bin/ folder.
• context_size - 8192 default, up to 262144. The prompt plus image tokens plus output all live inside this, and generation errors if max_tokens doesn't fit. Bigger is not free - KV cache scales with it.
• gpu_layers - -1 means offload every layer the backend supports. Drop it (or set a number) when the model doesn't fit and you want to spill layers to CPU.
• target_free_vram_mb - the pack asks ComfyUI to free this much VRAM before launching the server. Default 24576 (24 GB) matches the README's reference big-Qwen setup. On a 12 GB card, lower it or your diffusion model gets evicted.
• release_after_generate - true by default: the server shuts down after each call and VRAM returns. Set it false to keep the server resident between consecutive calls - the speed option, but read the warning below.

The optional block is where the power lives. mmproj selects the vision projector (required for any image/video work). keep_alive_seconds auto-releases an idle resident server; zero means manual, indefinite. flash_attn (auto/on/off) and cache_type_k/cache_type_v (f16/q8_0) tune the KV cache - Q8 caches save memory but can shift speed or quality slightly, per the README. cuda_visible_devices pins a secondary GPU, and comfy_vram_handoff (auto/always/never) controls whether ComfyUI hands VRAM over at all. extra_server_args appends raw llama-server flags, but the reserved ones (-m, --ctx-size, -ngl, --flash-attn, etc.) are blocked - you configure those through the widgets instead, which is a genuinely good design. startup_timeout_s and request_timeout_s are your tripwires for slow first loads and long generations.

The gotcha that burns people: while a resident server is up (release_after_generate = false and keep-alive active), llama.cpp's VRAM is invisible to ComfyUI. ComfyUI thinks there's free VRAM, happily starts a heavy diffusion or video job, and you OOM. The README's fix is explicit: run Unload Local AI Model before returning to a heavy branch, and create a real STRING dependency edge when ordering matters. If you instead hit CUDA OOM at server startup, the fixes are aggressive_vram_handoff, a smaller context, or fewer GPU layers - in that order.

Install comes with the pack: ComfyUI Manager (search WepeNerd) or clone https://github.com/WepeNerd/ComfyUI-WepeNerd.git into custom_nodes, pip install -r requirements.txt, restart. And once more, the pack will not fetch llama.cpp for you - a recent llama-server build with --jinja, /health, and streaming support is the prerequisite. "llama-server was not found" means the executable, not the .gguf, is missing.