Replicate meta/llama-2-70b-chat

This is Llama 2's flagship chat model - Meta's 70-billion-parameter instruction-tuned release from 2023 - running through Replicate instead of on your GPU, because 70B parameters isn't fitting on anything you're running ComfyUI on. It's a pure text node: no image input anywhere in the schema. Feed it an instruction, get back writing you can drop into a prompt, a caption, or wherever else your workflow wants generated text.

Worth saying plainly: this pack also ships Llama 3 nodes (70B, 8B, and the 405B heavyweight), and Llama 3 is a straightforward upgrade over Llama 2 on basically every axis. If you're choosing fresh, reach for one of those instead - this node earns its place if you specifically need Llama 2's behavior, or you're comparing outputs across generations.

How it works

Send a prompt and (optionally) a system prompt describing the persona/behavior you want, and the node hands it to Replicate's hosted Llama 2 70B, which generates a completion and returns it as text. No local compute, no VRAM - just an API round trip with per-call billing on Replicate's side.

The inputs and outputs that matter

• prompt (required) - your instruction.
• system_prompt (default "You are a helpful assistant.") - sets tone and behavior. Rewrite it if you want terser output, a specific persona, or instructions to only return a bare prompt with no commentary.
• temperature (default 0.75, range 0.01–5) - lower for consistent/factual output, higher for varied, more surprising completions.
• max_new_tokens (default 128) - raise this if replies are cutting off mid-sentence.

Further down: top_p (0.9), top_k (50), min_new_tokens (-1, meaning no floor), stop_sequences (custom strings to end generation early), seed, and debug. There's also replicate_weights - a field for pointing the node at a custom fine-tuned checkpoint you trained on Replicate for this base model, instead of the stock weights; leave it blank to use the default. Output is a single STRING. force_rerun skips Replicate's cache for a fresh generation on identical inputs.

How to install it

One install covers every node in this pack.

cd ComfyUI/custom_nodes
git clone https://github.com/replicate/comfyui-replicate
cd comfyui-replicate
pip install -r requirements.txt

Restart ComfyUI afterward. Or via ComfyUI Manager: search ComfyUI-Replicate, install, restart. Before running anything, set your token:

export REPLICATE_API_TOKEN="r8_************"; python main.py

Get a token at replicate.com/account/api-tokens.

Common issues & troubleshooting

Node errors immediately. REPLICATE_API_TOKEN isn't set in the environment ComfyUI is actually running in. This is by far the most common first-run failure across this whole pack. On the portable Windows build in particular, people report that a token set as a global Windows environment variable doesn't get picked up automatically - set it directly in your launch .bat if that happens to you.

Replies get cut off. Raise max_new_tokens.

Output feels dated or clunky compared to what you've read about modern LLMs. That's expected - this is a 2023-era model. It's fine for straightforward prompt-writing and text tasks, but if quality matters more than familiarity, switch to one of the Llama 3 nodes in this same pack.

Support is slow if something's actually broken (not just a config issue). This node pack is officially maintained by Replicate, but community reports describe GitHub issues and Discord questions going unanswered for extended periods - search first, official channels second.

Cost creeps up on long chains. Every call is billed. 70B is pricier per call than the 7B node next to it - use the smaller model while iterating, switch to 70B for the final pass if quality matters.