Replicate yorickvp/llava-v1.6-34b

LLaVA-1.6 (also known as LLaVA-NeXT) is a real step up from the original LLaVA - higher input resolution, better OCR and reasoning, and support for swapping in different base language models depending on how much quality you want to pay for. This is the 34B version: the biggest, most capable backbone in the 1.6 line, packaged onto Replicate by yorickvp, the same person behind the llava-13b node also in this pack. If you've used that one, this is the same idea with a stronger, slower, pricier brain underneath.

Two things set this node apart from most of the other captioners in the pack, and both come straight from the schema: image is optional here, not required - you can run it as a pure text chat model with no picture at all - and there's a history input for multi-turn conversation, something the plain captioner nodes don't have.

How it works

LLaVA pairs a vision encoder with a language model and trains the combination end-to-end to follow instructions about what it sees (or, with no image, just follow instructions). The 1.6 revision processes images at higher resolution than the original LLaVA, which is where its better text-reading and fine-detail performance comes from. The node sends your prompt, optional image, and optional prior conversation to Replicate; the model reasons over all of it and returns a STRING.

The inputs and outputs that matter

• prompt (required) - your instruction or question.
• image (optional) - attach it for captioning/VQA; skip it to use this purely as a chat LLM.
• temperature (default 0.2) - keep it low for grounded, factual answers about an image; raise it if you want more varied phrasing.
• max_tokens (default 1024) - generous headroom by default; lower it for terse output.
• history - pass in the prior turns of a conversation to keep context across multiple calls, rather than starting fresh every time.

top_p (default 1) rounds out the sampling controls. force_rerun bypasses Replicate's cache for a fresh generation. Output is a single STRING - feed it into a text node or downstream prompt input.

How to install it

One install covers every node in this pack.

cd ComfyUI/custom_nodes
git clone https://github.com/replicate/comfyui-replicate
cd comfyui-replicate
pip install -r requirements.txt

Restart, or via ComfyUI Manager: search ComfyUI-Replicate, install, restart. Set your token before launching ComfyUI:

export REPLICATE_API_TOKEN="r8_************"; python main.py

Token from replicate.com/account/api-tokens. Nothing local - 34B runs entirely on Replicate's hardware.

Common issues & troubleshooting

Node errors immediately. Missing REPLICATE_API_TOKEN in ComfyUI's launch environment - the most common first-run problem across this pack. On the portable Windows build, a token set only as a global Windows environment variable has been reported not to get picked up automatically; set it directly in the launch .bat if that happens.

It's noticeably slower and pricier than the 13B or Mistral-7B sibling. That's the trade for the biggest backbone in the family - reach for this one when quality genuinely matters (fine detail, careful reasoning, dense text), and drop to the smaller nodes for quick iteration.

Multi-turn conversation isn't behaving. Make sure you're actually passing the previous turns into history on each subsequent call - without it, every call starts from zero context regardless of what you asked before.

Description hallucinates or gets small details wrong. Normal VLM behavior, especially on counts and fine text. Lower temperature toward 0 and ask a narrower question if precision matters.

Same output on identical input. Replicate's cache - flip force_rerun.