ToriiGate Llama.cpp Text Generate

Let's be straight with you: ToriiGate-0.5 is a captioning model, not a chatbot. Its own source notes say it's "designed for single-image captioning, not general chat or multi-turn use." So the ToriiGate Llama.cpp Text Generate node is the odd one out in this pack - a text-only chat endpoint against the same llama-server the vision nodes use, with no image involved at all.

That doesn't make it useless. It's genuinely handy in two situations: as a way to verify your llama-server is up and responding before you burn time on a vision caption, and as a lightweight text side-channel when you're already running the server for the Vision Generate node. Just don't expect a capable assistant. Keep your expectations in line with a model that was fine-tuned to describe anime.

How it works

The node sends an OpenAI-compatible chat-completions request to your llama-server. That's it - no transformers, no torch model loading, no CUDA initialization. The pack's nodes_api.py builds a plain messages payload and POSTs it to {server_url}/v1/chat/completions, then hands back the response text. It only needs the requests library, so it dodges the whole heavy dependency stack the local Captioner requires.

The inputs that matter

• server_url - defaults to http://127.0.0.1:8080. Must point at a running llama-server, same as the vision node.
• model_name - the GGUF identifier dropdown, defaulting to DraconicDragon/ToriiGate-0.5-GGUF:Q4_K_M. It has to match what the llama-server router registered at startup (shown as operator(): ...). If you renamed a local model or run a non-GGUF backend, use custom_model_name to override it with any string.
• prompt - the user message, defaulting to "Describe the following topic in detail:".
• system_prompt - optional; leave blank to skip the system turn. This is where you'd steer the model toward acting like a reasonable captioner rather than a confused chatbot.
• temperature (0.7 default), max_tokens (512), timeout (120s) - standard knobs. Bump the timeout on slow hardware.

The output is a single STRING, text, which you can route anywhere strings go - though realistically you'll mostly just read it in a preview node.

Installing and running it

Install comes with the pack - ComfyUI Manager (search "ToriiGate") or:

cd ComfyUI/custom_nodes
git clone https://github.com/litch230/comfyui_toriigate.git

Then restart ComfyUI. Because this node talks to a remote server, you don't need the transformers/torch requirements at all - the only optional extra is requests if it isn't already in your environment (the node prints a clear pip install requests message if it's missing). What you do need is a running llama-server, per the pack's README:

llama-server.exe [-m <model.gguf>] [--mmproj <mmproj.gguf>] -b 2048 -ub 1024 -fa on -fit on -fitt 1024 -ngl 999

Common issues

• Connection refused / timeout - the server isn't running, or the URL is wrong. The error message literally tells you to check that llama-server is up.
• Model identifier mismatch - if the server preloaded a model with -m, the dropdown's repo:quant format may not match what the router registered. Use custom_model_name to match, or let the node request the GGUF by leaving -m off the server command.
• Sluggish text - ToriiGate is a 4B captioner; it's not fast at prose. That's the model, not your setup.