Replicate meta/meta-llama-3.1-405b-instruct

Llama 3.1 405B is Meta's largest open model - 405 billion parameters. You are not running this on your machine, and neither is this node. Nothing that size fits on a consumer GPU; it needs a small cluster. What this node does is hand the job to Replicate's cloud, where the model is already hosted, and hand you back the text. Your own GPU sits there doing nothing.

So why would a text model live in an image-generation graph? Prompt work, mostly. You feed it a loose idea and it writes you a detailed, model-ready prompt. You give it one prompt and ask for ten variations to batch. You have it caption, rewrite, translate, or turn a paragraph of story into a shot list. The STRING it returns wires straight into a CLIP text-encode or any styler/prompt node that takes text, so the whole "think up a good prompt" step becomes part of the workflow instead of a tab you alt-tab to.

How it works

This is one of the ComfyUI-Replicate nodes - a thin bridge to replicate.com. When you queue, ComfyUI ships your prompt and settings to Replicate over the API, their hardware runs the 405B model, and the generated text comes back as a node output. No weights download, no CUDA, no VRAM used locally. The flip side: it isn't free (you pay Replicate per run), it isn't instant (a model this big has real latency, plus occasional cold-start), and your text leaves your machine.

Inputs and outputs that matter

The single output is STRING - the model's reply. Wire it wherever text goes.

• prompt - your actual instruction or question. This is the one you'll always touch.
• system_prompt - defaults to "You are a helpful assistant." Change it to set the persona and rules, e.g. "You write terse, comma-separated Stable Diffusion prompts, no prose."
• max_tokens (512) - the length cap. Bump it if replies get cut off; keep it low to save money.
• temperature (0.6) - higher is more varied, lower is more deterministic.

top_p, top_k, presence_penalty, frequency_penalty, and stop_sequences are all here too, but the defaults are sane - leave them until you have a reason. Every node in this pack also has force_rerun: identical inputs get a cached result, so flip this to true when you want a fresh roll without changing anything else.

How to install it

Easiest path is ComfyUI Manager: search ComfyUI-Replicate, install, restart. Manually:

cd ComfyUI/custom_nodes
git clone https://github.com/replicate/comfyui-replicate
cd comfyui-replicate
pip install -r requirements.txt

Then restart ComfyUI. The only dependency is the small replicate Python package - this is one of the few custom-node installs that won't drag half of PyPI in behind it.

The one required step people miss: set a Replicate API token in your environment before launching ComfyUI. Grab one at replicate.com/account/api-tokens, then:

export REPLICATE_API_TOKEN="r8_************"; python main.py

Common issues

If the node errors the instant it runs, your token isn't set - that's the number one cause. Cost is the other thing to watch: 405B is the expensive tier. Per token it costs meaningfully more than the 70B node in this same pack, and dramatically more than a small 8B model. If you're only expanding prompts or writing captions, 405B is overkill - the 70B (or smaller) gives you 90% of the quality for a fraction of the bill. Save 405B for the jobs where the reasoning actually has to be top-tier. And expect a wait; this isn't a snappy local node, it's a cloud call to a very large model.