ToriiGate Llama.cpp Vision Generate

This is the node the pack's author actually wants you to use. The README marks the llama.cpp path "Recommended" over the local Transformers Captioner, and for good reason: the model runs as a GGUF through a separate llama-server, which means much faster inference, lower VRAM usage, and - the big one for anyone who's fought custom-node dependency hell - none of the transformers/torch loading code running inside ComfyUI. The trade is that you have to run the server yourself, manually, before you start.

ToriiGate-0.5 itself is an anime and digital-art captioning model, and this node is the "send it over the wire" version. It takes an image from your graph, converts it to a base64 PNG, embeds it in an OpenAI-compatible chat-completions request, and POSTs it to your llama-server, which runs the GGUF weights plus the vision projector (--mmproj). Out the other side comes your caption.

The inputs that matter

• server_url - defaults to http://127.0.0.1:8080. This must point at a live llama-server; there's no auto-start.
• model_name - the GGUF quantization dropdown, defaulting to DraconicDragon/ToriiGate-0.5-GGUF:Q4_K_M (3.07 GB). The identifier must match what the llama-server router registered at startup. Q4_K_M is the sane default; the tooltip calls it the "recommended balance of quality vs size," and it is. If you have VRAM to spare, Q8 is closer to the full model, but for captioning text output the difference is rarely worth it. custom_model_name overrides the dropdown for renamed local files or non-GGUF backends.
• image - required; only the first image in the batch is sent.
• prompt - optional; wire the ToriiGate Grounding Builder's prompt output here for grounded, character-aware captions. Blank falls back to "Describe this image in detail."
• max_pixels_mp - the sleeper input. Default 1.0. This is the resolution limit sent to the model, and the tooltip is blunt: lower values drastically reduce time-to-first-token in llama.cpp. If captions feel slow, drop it.
• max_new_tokens, temperature, decoding, seed - standard generation knobs. greedy_fast just forces temperature to 0.0. Set a nonzero seed in sample mode to repeat results.
• timeout - 120s default; raise it for slow hardware or very long long_thoughts captions.

The output, caption, is a STRING - same destinations as the local Captioner: save it, preview it, or feed it back into your workflow.

Setting up the server

Install the pack (Manager search "ToriiGate", or git clone https://github.com/litch230/comfyui_toriigate.git into ComfyUI/custom_nodes), restart ComfyUI, then do the server dance from the README:

1. Grab the latest llama.cpp release and pick the win-cuda zip matching your CUDA (or win-vulkan/win-rocm for AMD).
2. Download the matching cudart zip from the same release page and extract it into the folder with llama-server.exe. Skip this and NVIDIA inference will be painfully slow - this is the most-missed step.
3. Run the server:

llama-server.exe [-m <model.gguf>] [--mmproj <mmproj.gguf>] -b 2048 -ub 1024 -fa on -fit on -fitt 1024 -ngl 999

Leave off -m and --mmproj and the node will instruct the server to download the right GGUF automatically on first use. -ngl 999 offloads everything to the GPU. Once you see the server listening on http://127.0.0.1:8080, you're set.

Common issues

• "Cannot connect to llama-server" - it's not running, or the URL is wrong. The error message says exactly that. Start the server, then re-run.
• Horribly slow on NVIDIA - you skipped the cudart zip. Re-read step 2.
• "Model not found" on the request - the repo:quant identifier doesn't match what the router registered. Match it in the dropdown or override with custom_model_name.
• Timeout on long captions - raise timeout and/or trim max_new_tokens.

If you're captioning batches and have a GPU worth the name, this is the node to reach for over the local Captioner. The one-time server setup buys you speed and VRAM headroom every run after.