comfyui-mistral-caption
A custom ComfyUI node that allows interaction with Mistral AI language and vision models directly inside a ComfyUI workflow.
ComfyUI Mistral Caption Node
A custom ComfyUI node that allows interaction with Mistral AI language and vision models directly inside a ComfyUI workflow.
The node supports both text-only prompts and image-augmented prompts, enabling multimodal analysis using models such as Pixtral.
It can return responses as plain text or structured JSON, making it useful for automation, dataset generation, prompt analysis, and structured workflows.
Example Workflow
<p align="center"> <img src="images/workflow_example.jpg" width="800"/> </p> <p align="center"> Example ComfyUI workflow showing image caption generation using the <b>Mistral Caption</b> node. </p>Features
- Access Mistral AI models directly from ComfyUI
- Supports text and image inputs
- Optional structured JSON responses
- Automatic fallback model retry
- Configurable temperature
- Supports system instructions
- Works with Mistral API key or environment variable
Supported Models
The node currently supports:
mistral-small-latest
mistral-medium-latest
mistral-large-latest
pixtral-12b-latest
Pixtral models support multimodal image analysis.
Node Overview
Node Name:
Ask Mistral
Category:
Mistral
Output:
text
The node returns the model response as a string.
Inputs
Prompt
Main user prompt sent to the model.
Example:
Describe the objects in this image.
Response Type
Controls the output format.
Options:
text
json
When json is selected, the model is instructed to return valid JSON only.
Model
Select the primary Mistral model used for inference.
Optional Inputs
Images
Optional image input.
Images are automatically converted to base64 data URLs before being sent to the model.
This enables multimodal prompting with Pixtral models.
API Key
Your Mistral API key.
If left empty, the node will use the environment variable:
MISTRAL_API_KEY
System Instruction
Optional system message used to guide the model.
Example:
You are a computer vision assistant that describes images precisely.
Temperature
Controls output randomness.
Typical values:
0.2 → deterministic responses
0.7 → balanced responses
1.0 → more creative responses
Default:
0.7
Retry Model
Fallback model used if the primary model fails.
Example flow:
mistral-small → mistral-medium
Error Fallback Value
Optional value returned if all model attempts fail.
Useful for maintaining workflow stability.
Seed
This parameter does not control model randomness.
Instead, it forces ComfyUI to re-execute the node, allowing users to regenerate responses.
Output
Text
The model response returned as a string.
This can be connected to:
- Text display nodes
- JSON parsers
- logging nodes
- automation pipelines
Example Workflow
Load Image
↓
Ask Mistral
↓
Text Display
Installation
Navigate to your ComfyUI custom nodes directory:
cd ComfyUI/custom_nodes
Clone the repository:
git clone https://github.com/tardigrade1001/comfyui-mistral-caption
Restart ComfyUI.
Requirements
Install dependencies:
pip install -r requirements.txt
Dependencies include:
mistralai
Pillow
API Key Setup
You can provide your API key directly in the node or through an environment variable.
Example:
export MISTRAL_API_KEY="your_api_key"
Use Cases
This node can be used for:
- image captioning
- visual analysis
- structured dataset generation
- prompt engineering
- automated tagging
- JSON-based AI pipelines
Acknowledgements
Development of this project involved the use of AI-assisted coding tools, including ChatGPT and Google Gemini, for code suggestions, debugging, and documentation support.
License
MIT License