Generate Text with Pixtral

This is the node you'll actually run. Generate Text with Pixtral takes a Pixtral-12B vision model, a prompt, and a list of images, and writes text back - captioning being the thing 90% of people use it for. Pixtral is Mistral's open vision model, and in this pack it's the most flexible of the three: it tokenizes images directly into the context, so you can drop images anywhere in the prompt with the [IMG] token, use any number of them, and mix them at any aspect ratio. Multi-image comparison - "what's different between these two?" - is the one thing in the README that Pixtral genuinely does well and the other two models fumble.

How it works

The prompt you write gets wrapped in the model's instruct format ([INST]...[/INST]), which is why there's no separate system-prompt field on this node - the prompt is effectively a system instruction, and the author notes they may change that later. Every [IMG] in your prompt is a placeholder for an image: if you include fewer tags than images, the node auto-prepends the extras to the start of the prompt; too many tags just prints a warning. Each image costs roughly 1,000–4,000 tokens, so multi-image prompts eat context fast.

Inputs that matter

• pixtral_model - a VISION_MODEL from Load Pixtral Model.
• prompt - default "Caption this image:\n[IMG]". That default is fine for a first run.
• images (optional) - an IMAGE input. Note this is meant to be a list of images for one multi-image generation, not a batch - the nodes don't do batched text generation.
• max_new_tokens - default 256, the cap on how long an answer can be.
• temperature / top_p / top_k - sampling dials. Default 0.3 temperature is low and deliberate; crank it for variety, keep it low for captioning a training set.
• repetition_penalty - default 1.1, and this is the only generation node in the pack that has one. It's the thing that stops long captions from looping on themselves.
• stop_strings - default </s>, comma-separated list of strings that end generation.
• seed - for reproducibility.
• unload_after_generate - frees VRAM after each run, useful when the 12B model is crammed into an 8GB card.

Output is a single STRING - the generated text - which you can wire into a text viewer, a save node, or a dataset-captioning pipeline that writes 1.png/1.txt pairs for LoRA training.

Install

One-time pack setup (Manager search: ComfyUI-PixtralLlamaVision, or clone the repo into custom_nodes), requirements transformers >= 4.45, accelerate, bitsandbytes, torchvision. Model in ComfyUI/models/LLM, and the nf4 quant on HuggingFace if your VRAM is normal.

Troubleshooting

• "[IMG]" warnings - harmless; the node self-corrects missing tags.
• Bad OCR / garbled text in images - the nf4 Pixtral build is considerably worse on text-heavy images, per the README. Llama Vision is the better pick for OCR.
• Large images degrade quality - scale to ~512×512 before sending in.
• OOM - a 12B model is heavy; use the 4-bit quant and/or unload_after_generate.

It's not a speed demon - one generation at a time - but for captioning a folder of images for a dataset, it's the most reliable text the pack produces. People in the wild lean on it precisely for that.