ToriiGate Captioner

If you've ever captioned anime art with a generic vision model, you know the pain: it describes the composition fine, then fumbles identity - "a girl with blue hair" when it's unmistakably Hoshimi Miyabi - or writes clean prose that your Danbooru-trained checkpoint can't turn back into a prompt. ToriiGate-0.5 exists to fix exactly this, and the Captioner node is the fully-local way to run it inside ComfyUI. No API key, no cloud, no external server. Give it an image, get a caption back.

This is the "heavy" half of the litch230/comfyui_toriigate pack. The same pack ships llama.cpp API nodes that talk to a separately-run llama-server; this node instead loads the model natively through Hugging Face Transformers. The README frames it honestly as the simple-but-slow path. Simple because you drop it in and it just works. Slow because ToriiGate-0.5 is a Qwen3.5-4B-based model and you're running the whole thing in-process on your GPU.

How it works

The node takes the first image in your batch, converts the ComfyUI tensor to a PIL image, forces RGB, downscales it to your max_pixels_mp budget (1.0 MP default), and feeds it to a Qwen3_5ForConditionalGeneration model with ToriiGate's own system prompt. First run downloads the whole snapshot from Hugging Face - several GB - with a progress bar in the console. The model is cached by model path + device + dtype, so repeated runs skip the load.

The inputs that matter

• image - required, and only the first frame of a batch gets captioned.
• prompt - optional. Wire the text output of the ToriiGate Grounding Builder here, or type your own. Blank falls back to "Describe this image in detail."
• max_new_tokens - 512 default. Keep 256–512 for quick captions; if you're using the long_thoughts prompt formats, go 1024+ or the model runs out of room.
• decoding - greedy_fast (default) is deterministic and faster; sample plus temperature buys you variety at the cost of stability.
• device / dtype - leave cuda/bfloat16 alone unless you know better. bfloat16 is the model's native format. cpu "works, but will be very slow" per the tooltip, and it means it.
• keep_model_alive - off by default, which unloads the model from VRAM after every run. For batch-captioning a folder, flip it on or you'll pay the reload tax every image.
• model_path - defaults to Minthy/ToriiGate-0.5; point it at a local folder if you've already downloaded the weights.

The single output, caption, is a plain STRING. Wire it to a Save Text node, a preview, or back into your workflow as a prompt - ComfyUI doesn't care where the string goes.

Installing it

Two options: ComfyUI Manager (search "ToriiGate" or "comfyui_toriigate"), or the manual route:

cd ComfyUI/custom_nodes
git clone https://github.com/litch230/comfyui_toriigate.git

Then install the Python dependencies - this is the part that actually matters, because the Captioner needs the whole list:

pip install -r ComfyUI/custom_nodes/comfyui_toriigate/requirements.txt
# ComfyUI Portable on Windows:
python_embeded\python.exe -m pip install -r ComfyUI\custom_nodes\comfyui_toriigate\requirements.txt

That pulls in transformers>=4.52.0 (required for Qwen3_5ForConditionalGeneration), torch, Pillow, and on Windows flash-linear-attention and triton-windows. Restart ComfyUI after installing.

Common issues

• Transformers too old - the node throws a clear ImportError telling you to install/upgrade requirements.txt. Follow it.
• First run seems hung - it's downloading gigabytes. Watch the console for the HF progress bar, not the ComfyUI canvas.
• Running on CPU by accident - if CUDA isn't available the node warns and falls back to CPU, where a caption can take minutes. Check your device input.
• Caption keeps getting cut off - you picked a long_thoughts format with max_new_tokens still at 512. Bump it.

Honest take: if you have a decent GPU, the llama.cpp Vision node in this same pack is the faster route for repeated captioning. Reach for the Captioner when you want zero server setup and are okay with a slower, more VRAM-hungry run.