Pixtral Vision Analysis

The hook

You want an actual language model to look at your image and tell you what's there - a caption for a LoRA dataset, a description to feed into a prompt, an answer to "what's wrong with this face?" - but you don't want to babysit a 12B vision model on your own GPU. That's exactly what this node is for. It takes an image, sends it to Mistral's Pixtral API, and hands you back the text answer as a STRING you can route anywhere in your graph. No local model, no VRAM, no GGUF hunting. Just a key.

The one thing it is not is an image generator. Pixtral is a multimodal language model - it reads images and writes about them, it doesn't create new ones (the community spent real time clarifying this when the model dropped). Think of it as the "describe and explain" step in a pipeline, not the "make" step.

How it works

Open the source and the mechanism is plain. The node takes your images tensor, squeezes it to numpy, and converts each frame to a PIL image - handling grayscale, RGB, and RGBA (RGBA gets dropped to RGB). It saves each as a JPEG, base64-encodes it, and builds a standard chat message with a text part plus an image_url part pointing at the data:image/jpeg;base64,... blob. Then it POSTs to https://api.mistral.ai/v1/chat/completions with model: "pixtral-12b-2409" and your temperature and token settings. The model's reply text is what comes out of the output socket.

Two practical consequences. First, your images physically leave your machine and land on Mistral's servers - that's the privacy trade-off of any API vision node, and worth knowing if you're captioning anything sensitive. Second, the API is the thing that decides quality, not the node. Pixtral-12B is a genuinely solid captioner - it placed high on Vision Arena against models three times its size - but captioning veterans generally reach for Qwen2-VL fine-tunes for dataset work, and Pixtral has lost some head-to-head eval comparisons. For quick "describe this" jobs it's hard to beat for convenience.

The inputs that matter

• images - your batch of IMAGE tensors. It accepts multiple, so you can feed a whole batch of frames or dataset images at once.
• prompt - the instruction, defaulting to "Describe the image". This is where you do the actual work: ask for a booru-style tag list, a natural-language caption, or "what's in the background?"
• api_key - paste your Mistral key here (see below). It's a required input, not a global setting.
• temperature - default 0.7, range 0–1.5. Lower it for consistent, literal captions; for dataset work, keep it low.
• maximum_tokens - default 1000, capped at 4096. A caption rarely needs more; bump it only for long multi-image analyses.

The single output is a STRING. Wire it into a text preview, a preview_text node (ships in the same pack), a prompt assembler, or save it with a text-save node. The pack also includes MultiImagesInput, which batches several images into one IMAGE for the input - handy if your workflow produces images on separate branches.

Install and getting a key

ComfyUI Manager can find it as ComfyUI_pixtral_vision; or:

cd ComfyUI/custom_nodes
git clone https://github.com/ShmuelRonen/ComfyUI_pixtral_vision

Restart ComfyUI. Dependencies are requests, Pillow, torch, and numpy - and since ComfyUI already ships torch and numpy, the only real add is requests, which you almost certainly have. No model downloads at all, because there's no local model. Small caveat: the README's own install example contains a placeholder yourusername clone URL - don't copy that; use the real repo above.

For the key, sign up at Mistral AI, generate an API key under your account, and paste it into the api_key field. Mistral gives you some free trial credit; after that, vision calls are metered per token. If your key dies mid-workflow, you'll know immediately - the node returns the API error as its text output rather than crashing.

Where people get burned

• No key / expired key → you get "API Error: 401 ..." as the output string, not a console crash. Easy to mistake for a bad prompt at first.
• Forgetting images leave your machine. It's the same story as any hosted model; don't batch-send private reference images through a third-party API.
• Expecting generation. It describes, it doesn't generate. Pair it with an image model - describe, feed the description into your prompt - and you've got a genuinely useful loop.
• Over-temperature. Near 1.5, captions get flaky and unreliable for dataset tagging.

For a quick "what's actually in this image" answer, or to caption a few hundred images without breaking out a full local VLM stack, this is the one you reach for.