Image to Prompt (Abacus.AI)

The name says "Abacus.AI", but read the README's fine print before you assume anything: this is a third-party node with no affiliation to Abacus.AI at all. What it actually does is simpler and more useful than the branding suggests - it's a thin wrapper that uploads the image sitting in your graph to Abacus.AI's RouteLLM API gateway and hands you back a prompt. No local vision model, no extra VRAM, no downloads. You rent a frontier-class VLM per request and pay per call.

Why you'd reach for it

Image-to-prompt nodes live between your image source and CLIP Text Encode, and they come in two flavors. There's the local crowd - Florence-2, Joy Caption, WD14 taggers - which run free but eat VRAM and top out at "good enough" captioning. Then there's this: it points at the same class of vision models you can't run locally at all, GPT-4o / Claude / Gemini-level captioning, for the price of an Abacus subscription and a few cents per image. If you already have an Abacus account and want genuinely good captioning in a workflow without hosting a VLM, this is the play. If you don't want another subscription bleeding credits, the local captioners are slower and less eloquent but cost nothing.

How it works

On startup the node calls https://routellm.abacus.ai/v1/models (5-second timeout) and builds its model dropdown from whatever comes back - that's why the list has 140+ entries and can include models that don't exist on your plan. If the fetch fails, it falls back to a small hardcoded list: route-llm, gpt-4o, claude-3-5-sonnet, gemini-1.5-pro. On each run it takes the first image in your batch tensor, scales the float [0,1] values to [0,255] uint8, encodes a PNG, base64s it, and sends an OpenAI-compatible chat completion - your text instructions plus the image as a data URL, capped at 1024 tokens - to the same endpoint. The caption comes back as a plain string.

Inputs and output

Four required inputs, and honestly only two of them matter most of the time:

• instructions - the one you'll actually edit. Defaults to "Describe this image in detail for an image generation prompt."
• model - defaults to route-llm, which auto-routes to the "best" model. Pick a specific one when you want predictable cost or speed.
• image - your IMAGE tensor.
• api_key - your Abacus key. Required; the node returns a polite error string rather than crashing if it's empty.

The single output is text (STRING), which wires straight into CLIP Text Encode.

One piece of advice from the prompt-engineering side of things: the default instruction produces natural-language prose, which suits Flux and the newer LLM-encoded checkpoints. If you're captioning for a tag-based anime model like Illustrious, Pony, or NoobAI, change instructions to ask for comma-separated Danbooru tags - the caption is only as useful as the text encoder it's feeding.

Installing it

Through ComfyUI Manager, search "Image to Prompt (Abacus.AI)". Or do it by hand:

cd ComfyUI/custom_nodes
git clone https://github.com/kplkasteel/ComfyUI-Image-to-Prompt-Abacus.AI-
cd ComfyUI-Image-to-Prompt-Abacus.AI-
pip install -r requirements.txt

Then restart ComfyUI. Dependencies are tiny - openai, requests, pillow, numpy - and there are no model files to fetch. You do need an Abacus.AI account, an API key from the dashboard, and enough credits to actually run it.

Where people get burned

• It costs money every time. Each request burns credits, and there's no local fallback. If you leave this in a batch workflow, watch the dashboard or you'll find out from your bill.
• Your key lives in the workflow JSON in plaintext. Don't share a workflow file that contains this node, and remember every image you run through it leaves your machine for a third-party API. That privacy tradeoff is inherent to paid API nodes, and it's the same class of trust the ecosystem has already been burned on once.
• "Failed to fetch models" at startup just means the API call timed out - the node keeps working on its fallback list, so don't panic.
• "Error: Failed to generate prompt from API." usually means the key is invalid, credits are gone, or the model you picked isn't on your plan.

It's a small, single-commit pack with one node and a thorough test suite - set your expectations accordingly. But as a dead-simple way to put frontier captioning in a workflow without touching your GPU, it does exactly what it says on the tin.