Replicate meta/meta-llama-3-70b-instruct

This is the LLM node you'll actually reach for out of the two Llama nodes in this pack. Llama 3 70B is Meta's mid-large model - big enough to write genuinely good prompts, rewrite text, translate, brainstorm variations, or answer questions, but a lot cheaper to run than the 405B monster sitting next to it. For the kind of text work an image workflow needs - turn a scribbled idea into a detailed prompt, spit out ten variations, caption a batch - 70B is the sweet spot.

Like everything in ComfyUI-Replicate, it doesn't run on your machine. 70B still won't fit on a normal GPU, so the node calls Replicate's cloud, runs the model there, and returns the text. Local GPU untouched.

How it works

Queue the graph and ComfyUI sends your prompt plus settings to Replicate; their hardware runs Llama 3 70B and the reply comes back as a STRING. No download, no VRAM, no dependency mess - just an API round-trip. What you trade for that is money per run and a little latency, plus your text leaving the machine.

Inputs and outputs that matter

Output is a single STRING. Feed it into a CLIP text-encode, a prompt combiner, or a show-text node.

• prompt - required here (unlike the 405B node, where it's optional). Your instruction.
• system_prompt - "You are a helpful assistant" by default. Rewrite it to shape the output: tell it to return bare prompts, or to answer in one line.
• max_tokens (512) - length cap; raise it if replies truncate.
• temperature (0.7) - creativity dial, 0 to 5. Drop toward 0 for consistent, factual output; push up for wilder variations.

There's a prompt_template field pre-filled with Llama 3's chat format (the <|begin_of_text|>...<|eot_id|> scaffolding) and a matching stop_sequences default - leave both alone unless you know exactly why you're touching them, because breaking the template is a good way to get garbled output. top_p, top_k, length_penalty, presence_penalty, and seed round it out; defaults are fine. And force_rerun forces a fresh generation when your inputs are identical to a previous run (otherwise you get the cached answer).

How to install it

ComfyUI Manager: search ComfyUI-Replicate, install, restart. Or manually:

cd ComfyUI/custom_nodes
git clone https://github.com/replicate/comfyui-replicate
cd comfyui-replicate
pip install -r requirements.txt

Restart afterward. It's a light install - the only real dependency is the replicate package.

Set your token first. Get one at replicate.com/account/api-tokens and export it before you start ComfyUI:

export REPLICATE_API_TOKEN="r8_************"; python main.py

Common issues

Node errors immediately → the REPLICATE_API_TOKEN isn't set in the environment ComfyUI is running in. That's almost always it.

Getting cut-off replies is a max_tokens problem - raise it. Getting weird, half-formatted output usually means someone edited the prompt_template or stop_sequences and broke Llama 3's expected chat structure; reset them to the defaults. Cost-wise, 70B is far friendlier than 405B, but it's still per-run cloud billing, so don't leave a loop hammering it. And since it's a remote call, expect a second or two of latency plus the odd cold-start pause - normal, not a hang.