llama-api-comfy

🦙 ComfyUI-LlamaCpp-Caption

ComfyUI custom node for captioning images using a local llama.cpp server with vision/multimodal support.

Features

• 🦙 LlamaCpp Caption — Send an image to your llama.cpp server and get back a clean caption
• Auto CoT removal — Strips <think>, <reasoning>, and <thought> blocks from reasoning model outputs
• JoyCaption compatible — Designed to slot into the ImageBatchPath → Caption → CaptionSaver workflow
• Batch support — When connected to JoyCaption's Image Batch Path (which outputs lists), ComfyUI auto-iterates and processes each image individually
• OpenAI-compatible — Works with any llama.cpp server running a vision model (Qwen-VL, Gemma 3, LLaVA, etc.)

Requirements

• A llama.cpp server running with a vision-capable model:

  # Example with a single GGUF that includes the projector:
  ./llama-server -m your-vision-model.gguf
  
  # Or with a separate projector:
  ./llama-server -m your-model.gguf --mmproj your-mmproj.gguf
  
  # Or from Hugging Face:
  ./llama-server -hf ggml-org/gemma-3-4b-it-GGUF
  

Installation

Option 1: Symlink (recommended for development)

# From your ComfyUI directory:
ln -s /path/to/llama_api_comfy custom_nodes/llama_api_comfy

Option 2: Copy

cp -r /path/to/llama_api_comfy custom_nodes/llama_api_comfy

Then install dependencies:

pip install requests Pillow numpy

Note: torch is already provided by ComfyUI's environment.

Node: 🦙 LlamaCpp Caption

| Input | Type | Default | Description | |-------|------|---------|-------------| | image | IMAGE | (required) | The image to caption | | prompt | STRING | "Describe this image in detail." | The prompt to send with the image | | server_url | STRING | http://localhost:8080 | URL of the llama.cpp server | | temperature | FLOAT | 0.7 | Sampling temperature (0.0–2.0) | | max_tokens | INT | 1024 | Maximum tokens in the response | | system_prompt | STRING | (optional) | System prompt for the model | | image_path | STRING | (optional, from ImageBatchPath) | Passthrough for CaptionSaver |

| Output | Type | Description | |--------|------|-------------| | caption | STRING | The generated caption (CoT stripped) | | image_path | STRING | Passthrough of input image_path |

Usage with JoyCaption Nodes

This node is designed to work with ComfyUI-JoyCaption:

┌──────────────────┐       ┌─────────────────────┐       ┌───────────────┐
│ Image Batch Path │       │ 🦙 LlamaCpp Caption │       │ Caption Saver │
│                  │       │                     │       │               │
│        IMAGE ●───┼──────▶┼─● image             │       │               │
│   IMAGE_PATH ●───┼──────▶┼─● image_path        │       │               │
│                  │       │                     │       │               │
│                  │       │     caption ●───────┼──────▶┼─● string      │
│                  │       │  image_path ●───────┼──────▶┼─● image_path  │
└──────────────────┘       └─────────────────────┘       └───────────────┘

1. Image Batch Path loads images from a folder (outputs lists)
2. 🦙 LlamaCpp Caption receives each image + its file path, captions it, and passes both through
3. Caption Saver saves each caption as a .txt file matching the original image filename

ComfyUI auto-iterates over the list outputs from Image Batch Path, so every image in the batch gets captioned automatically.

License

MIT