🦙 Llama-Swap Client

The name is a small lie, and the lie is the point: the 🦙 Llama-Swap Client doesn't swap any models, and it loads nothing into ComfyUI's VRAM. It's a thin, OpenAI-compatible chat client that talks to a llama-swap server running on localhost and lets you drive that server from inside your workflow. No API key, no cloud call, no giant LLM node pack squatting in your graph.

Why would you want that? ComfyUI has quietly become a place where people use text, not just generate it - prompt expansion before a KSampler, captioning a render to loop it back in, asking a vision model what's actually in the image you just made. The usual way to do that is to keep a terminal or a browser tab open and copy text back and forth. This node ends the copy-paste. The author built it for exactly that reason: he was tired of juggling tabs, and he wanted the hot-swap model manager he already ran to be reachable from a workflow. It's a small pack (MIT, a couple hundred lines) and it does one thing well.

How it works

The node never touches llama.cpp. When you hit Run, it sends one POST to {server_url}/v1/chat/completions with a standard OpenAI payload - model, a messages array built from your system_prompt and prompt, and stream: false - then waits up to 300 seconds for the answer. The llama-swap server (the mostlygeek project this is named after) does the actual inference and hands back the model's raw text. There's no GGUF loading in ComfyUI; all the quantized weights live on the server side.

Two details are worth the price of admission. First, thinking extraction: reasoning models like DeepSeek-R1, QwQ and Qwen3 wrap their chain-of-thought in <think>/<thinking> tags, and this node regex-strips them so you get a clean response and a separate thinking output holding the raw reasoning trace. That's a genuinely useful feature for debugging why a model answered the way it did. Second, vision: connect any IMAGE and the first frame is converted to a JPEG base64 string and sent as an image_url. It only matters if the model you picked is vision-capable (Qwen-VL and friends) - otherwise it's inert.

The inputs that matter

There are only a few, and three of them are what a beginner actually touches:

• server_url - defaults to http://localhost:8080, which is llama-swap's default. Change it only if your server is elsewhere.
• model - leave it blank and click the 🔄 Fetch Models button on the node. It reads the server's /v1/models and drops a floating picker. Note this lists model ids configured in llama-swap, not filenames from your models/ folder - a mismatch here is the most common "why is nothing working" moment.
• unload_after_generate - calls /unload on the server after every run. The author calls it a lifesaver on 16GB cards, and he's right: it's the difference between a reasoning model staying warm in VRAM and ComfyUI being able to run diffusion at all.

Outputs are two STRINGs, response and thinking. Wire both into Preview Text nodes - thinking will be an empty string when the model produced no chain-of-thought.

Install

Via ComfyUI Manager, search "ComfyUI llama-swap", or the manual route:

cd ComfyUI/custom_nodes
git clone https://github.com/ai-joe-git/comfyui_llama_swap

Restart ComfyUI. Dependencies are requests and pillow, which any standard ComfyUI install already has - the pyproject.toml declares zero extra deps. There are no models to download into ComfyUI; those live on the llama-swap server.

Gotchas

The server has to actually be running and reachable from wherever ComfyUI executes, and it must be llama-swap (or anything else with the same OpenAI-compatible endpoints - the code doesn't care). First request after a cold model load can take a while, which is why the timeout is 300s; if you're hitting the timeout, it's usually a model that's too slow to load, not a hang. And since this is the internet, a word of context: LLM-ish nodes have a bad reputation in this ecosystem for good reason - the notorious LLMVISION node shipped malware and landed its author in federal trouble. This pack is the opposite end of that spectrum: a plain HTTP client to your own local server, no telemetry, MIT licensed. You still shouldn't install random nodes blind, but this one is about as boring-safe as it gets.