Replicate yorickvp/llava-v1.6-mistral-7b

Same LLaVA-1.6 (LLaVA-NeXT) architecture as its 34B sibling in this pack, but built on a Mistral-7B backbone instead - smaller, faster, and considerably cheaper per call on Replicate. yorickvp packaged both; this is the one you reach for when you want quick captions or VQA answers without paying 34B prices, and it's still a real step up from the original LLaVA-1.5 generation thanks to 1.6's higher input resolution and better OCR handling.

Like its bigger sibling, and unlike the plain captioner nodes elsewhere in this pack, image here is optional - prompt is the only required field, so you can run this as a pure text chat model if you want. There's also a history input for multi-turn conversations, which the simpler VLM nodes don't have.

How it works

The vision encoder turns your image into embeddings the language model can reason over alongside your text prompt; drop the image and it just behaves like a regular instruction-following chat model. The node sends everything to Replicate, the Mistral-7B-backed model runs there, and the reply comes back as a STRING.

The inputs and outputs that matter

• prompt (required) - your instruction or question.
• image (optional) - include it for captioning or visual Q&A; leave it out for plain text chat.
• temperature (default 0.2) - low by default, good for grounded answers; raise it for more varied output.
• max_tokens (default 1024) - plenty of headroom; lower it if you want short, tight responses.
• history - carry prior conversation turns forward so the model has context across multiple calls instead of starting cold each time.

top_p (default 1) is the other sampling knob. force_rerun skips Replicate's cache for a fresh generation on unchanged inputs. Output is a single STRING - route it into a text node or a downstream prompt input.

How to install it

One install covers every node in this pack.

cd ComfyUI/custom_nodes
git clone https://github.com/replicate/comfyui-replicate
cd comfyui-replicate
pip install -r requirements.txt

Restart, or via ComfyUI Manager: search ComfyUI-Replicate, install, restart. Set your token before launching:

export REPLICATE_API_TOKEN="r8_************"; python main.py

Token from replicate.com/account/api-tokens. Nothing runs locally - this is entirely Replicate's compute.

Common issues & troubleshooting

Node errors immediately. REPLICATE_API_TOKEN isn't set in the environment ComfyUI actually launched from - by far the most common first-run failure across this pack. On the portable Windows build, a token set as a global Windows environment variable specifically has been reported not to get picked up automatically; set it in the launch .bat if that's your situation.

Answers feel shallower than you expected for a VLM. 7B is the trade-off for speed and cost - it's noticeably faster and cheaper than the 34B node, but gives up some depth on genuinely hard reasoning or dense-text images. Step up to the 34B sibling when that matters.

Multi-turn context isn't sticking. history has to be explicitly passed with each new call - the node doesn't remember anything between separate runs on its own.

Hallucinated detail, wrong counts. Standard VLM limitation, more pronounced on smaller models. Lower temperature and ask a narrower, more specific question.

Same output on identical input. Replicate's cache - flip force_rerun for a fresh run.