Meta_Llama3_8B

This is the node that makes the whole pack make sense. You feed it a request in plain words - "a cat in a spacesuit walking on the moon" - and it returns a finished prompt string that you wire straight into your CLIP Text Encode node. No API key, no browser tab, no third-party service. The model runs on your own GPU, and ComfyUI never leaves the canvas.

That "have an LLM write the prompt" pattern has quietly become one of the more sensible habits in this hobby. The KB's own prompt-engineering notes say it plainly: if your model's encoder is an LLM reading an instruction, having a different LLM write that instruction is translation between two things that speak the same language. This pack is exactly one of those dedicated local prompt-writer nodes, and Meta_Llama3_8B is its flagship.

How it works

The node is a thin wrapper around the HuggingFace transformers text-generation pipeline. It loads whatever repo_id you give it in bf16, sets device_map="auto" (so layers spill to CPU if you're tight on VRAM), applies the model's chat template to your system + user messages, generates with sampling, and then does a hard split on the assistant header to return just the model's reply. It's the same generation call you'd write in a notebook, minus the bookkeeping.

A couple of honest quirks from reading the source: the returned text gets a '*' replaced with ' *' - a tiny formatting fix so asterisks don't glue to words. And errors aren't thrown; they're caught and returned as the output string. When something goes wrong you'll see the OSError text sitting in the output box instead of a nice red node, so check what the node "answered" before assuming it worked.

The inputs that matter

There are eight inputs but only a few you'll touch day to day:

• repo_id - the model. Default is blank, so this is the field that makes or breaks you. See below.
• system_content and user_content - the persona and the request. The defaults are a Chinese catgirl persona ("何小喵") chatting in character, which tells you exactly who wrote this pack and that you'll want to replace them with your own.
• max_new_tokens - answer length, 32–4096. The 256 default is fine for a prompt.
• temperature and top_p - sampling controls. Lower temperature, more literal; this is a prompt writer, so 0.6-ish is a good place to stay.
• reply_language - appends "answer me in English" (or whatever) to your prompt, eight languages plus Original_language. Handy if you think in Chinese and your downstream prompt encoder is English-only.

The single output, prompt, is a STRING. Wire it into a text input (a CLIP Text Encode prompt box, or any text slot) and run.

Getting a model into it

The README's two paths both matter. meta-llama/Meta-Llama-3-8B-Instruct is the classic but it's a gated repo - you need a HuggingFace token (huggingface-cli login) and you must accept the license on the model page first, or the load fails with a 401. The README's other suggestion, gradientai/Llama-3-8B-Instruct-262k, downloads freely if you just want it to work. First run pulls ~16 GB into the HF cache (~/.cache/huggingface), which is not your ComfyUI models folder, and it can look frozen while it downloads - it isn't.

Install the pack the usual way: ComfyUI Manager, search ComfyUI_Llama3_8B, install, restart. Or:

cd ComfyUI/custom_nodes
git clone https://github.com/smthemex/ComfyUI_Llama3_8B

There's no requirements.txt - the node imports transformers at load time and expects it in your ComfyUI Python env. If the pack fails to load, pip install transformers in that env is the fix.

Where people get burned

The big one is VRAM. Llama 3 8B in bf16 is ~16 GB of weights, so this wants a 16 GB+ card to feel local; on 8 GB it will chug along via CPU offload or OOM entirely. There's no quantization option anywhere in the schema - this pack is from mid-2024, before GGUF LLM nodes became the norm. If you're on a small card, this is the wrong tool; a GGUF-based LLM node is the pragmatic pick. The author is a prolific custom-node maker who's still shipping nodes years later, but this one is a hobbyist wrapper: single commit, minimal maintenance, README in machine-translated English. It works, and it's free, and it's a genuinely pleasant way to batch out prompt variants without leaving ComfyUI - just don't expect it to babysit your VRAM.