ChatQA_1p5_8b

Sibling to the pack's Meta_Llama3_8B node, this one swaps in NVIDIA's Llama3-ChatQA-1.5-8B - a Llama 3 8B fine-tune built for conversational Q&A and retrieval-augmented generation. NVIDIA trained it to answer questions against a document context and beat GPT-4 on their own conversational QA benchmark. In this pack, though, you're not using it for document retrieval. You're using it the same way you use every node in ComfyUI_Llama3_8B: type a request, get a prompt string back, wire it into your text encoder.

The practical difference from Meta_Llama3_8B is the prompt format. ChatQA expects a specific structure - a system prompt, an instruction, an optional context block, then the conversation. The node bakes all of that in, which is a decent chunk of fiddly formatting you don't have to write yourself.

The honest catch

Read the source and you'll spot it: the context block is hardcoded to an empty string. document = "" is set in the node and never exposed as an input. So the "RAG" part of ChatQA - the entire reason NVIDIA fine-tuned it - isn't wired up here. You get the model's improved instruction-following and answer formatting, but you're not feeding it any documents. If you came here hoping for "ask questions about your downloaded image captions," that's not what this node does. It's a Q&A-flavored prompt generator, not a retrieval pipeline.

What it does well: ChatQA's formatted-input style (system + instruction + User:/Assistant: turns) tends to produce cleaner, more structured answers than a bare chat template, and the node strips the Assistant: prefix and collapses newlines out of the result, so what lands in your prompt box is a single tidy line.

Inputs and output

Same overall shape as its sibling, with a couple of ChatQA-specific extras:

• repo_id - set it to nvidia/Llama3-ChatQA-1.5-8B (or a local path). It's a plain STRING with no default, so the box starts empty.
• system and instruction - the two ChatQA-format fields. The defaults are the model's recommended system prompt and "Please give a full and complete answer for the question." Leave them alone unless you know why you're changing them.
• user_content - your request.
• max_new_tokens (128 default), temperature (0.6), top_p (0.9), reply_language - all behave like the rest of the pack.
• get_model_online - flip it off after your first download to force offline mode, which sets TRANSFORMERS_OFFLINE=1. Useful if you're behind a network that fights HuggingFace (the README itself warns about pre-downloading for this reason).

Output is a single STRING named prompt, feeding any text slot downstream.

Installing and the model question

The pack installs once for all four nodes - ComfyUI Manager, search ComfyUI_Llama3_8B, or:

cd ComfyUI/custom_nodes
git clone https://github.com/smthemex/ComfyUI_Llama3_8B

then restart. No requirements.txt ships, so make sure transformers is in your ComfyUI Python environment; the node imports it at load and the whole pack silently fails to register if it's missing.

The model itself downloads through HuggingFace on first run into ~/.cache/huggingface, and it's roughly 16 GB in the fp16 this node loads. Expect to accept a license agreement on the model page and, if you're on a Meta-gated account flow, to have huggingface-cli login done first. Which is the other thing to know: this is 8B in fp16, so a 16 GB+ GPU makes it feel native. On less, device_map="auto" offloads and it turns into a slow, sad CPU text generator.

One genuine quirk worth remembering: like all three generation nodes in this pack, errors are caught and returned as the output string rather than failing the node. And ChatQA's post-processing splits the reply on the first : - so an answer that begins "Sure:" loses the word before the colon. Cosmetic, but it'll confuse you the first time you see it.