Replicate meta/llama-2-7b-chat

The small sibling to llama-2-70b-chat in this pack - same model family, a tenth of the size. 7B is faster to run and noticeably cheaper per call on Replicate, at the cost of the reasoning depth and nuance you get from the 70B version. For simple, well-defined text jobs inside a ComfyUI graph - expand a keyword list into a prompt, rewrite a caption, generate a handful of variations - 7B is usually plenty, and it's the one to iterate with before switching to something bigger for a final pass.

Also worth knowing: this pack ships Llama 3 nodes too (8B, 70B), which are a straightforward generational upgrade. This Llama 2 node is worth reaching for on cost and speed, or if you specifically want Llama 2's output style - not because it's the strongest option available.

How it works

Same mechanism as its bigger sibling: send a prompt (and optionally a system prompt), Replicate runs the hosted 7B model, and the completion comes back as text. No local weights, no VRAM cost, per-call billing.

The inputs and outputs that matter

• prompt (required) - your instruction.
• system_prompt (default "You are a helpful, respectful and honest assistant.") - shapes tone and behavior; edit it to get output in a specific format.
• temperature (default 0.7, range 0.01–5) - lower for consistency, higher for variety.
• max_new_tokens (default 128) - the length cap; raise it if answers are getting truncated.

Beyond that: top_p (0.95), top_k (default -1, meaning unrestricted), min_new_tokens (-1), stop_sequences, seed, debug. One thing this node has that the 70B sibling doesn't: repetition_penalty (default 1.15) - bump it up if you're seeing the model loop or repeat phrasing, which shows up more often on smaller models than large ones. There's also replicate_weights for pointing at a custom fine-tune you trained on Replicate for this base model - leave it blank for the stock checkpoint. Output is a single STRING. force_rerun bypasses the cache for a fresh generation on unchanged inputs.

How to install it

One install covers the whole pack.

cd ComfyUI/custom_nodes
git clone https://github.com/replicate/comfyui-replicate
cd comfyui-replicate
pip install -r requirements.txt

Restart. Or through ComfyUI Manager: search ComfyUI-Replicate, install, restart. Set your token before launch:

export REPLICATE_API_TOKEN="r8_************"; python main.py

Get one at replicate.com/account/api-tokens.

Common issues & troubleshooting

Node errors immediately. Missing or unset REPLICATE_API_TOKEN in the environment ComfyUI launched from - the most common issue across every node in this pack. On portable Windows builds, a token set as a global environment variable specifically has been reported not to get picked up; set it in the launch .bat if export/system settings don't work.

Output repeats itself or loops. This is where repetition_penalty earns its keep - nudge it up from 1.15 toward 1.3 or so if you're seeing the model get stuck restating the same phrase.

Answers feel shallow compared to bigger models. That's the size trade-off - 7B is fast and cheap, not deep. Step up to the 70B node, or a Llama 3 node, when quality matters more than speed.

Replies get cut off. Raise max_new_tokens.

Same output on re-run. Replicate's cache on identical inputs - change the prompt or flip force_rerun.