SambaNova LLM

Let's be honest about what this node is: a thin wrapper around SambaNova's Cloud API that returns generated text into your ComfyUI graph. No GPU, no local model download, no llama.cpp - you type a prompt, it fires a POST to api.sambanova.ai/v1, and a string comes back. The interesting part is what SambaNova runs and how fast: Llama 3.1 70B at around 460 tokens per second on their hardware, with a free tier that includes the full 405B. The README calls itself "the second fastest token output LLM" and that's roughly right - Cerebras edges it out on raw speed, and the author literally has a "Getting Cerebras API (WIP)" note in the same README, so the benchmark jockeying is part of the pitch.

Why would an image person care? Because ComfyUI is an image pipeline, and the genuinely useful place for an LLM node is the front of it: generating, translating, or cleaning up the prompt that feeds your text encoder. That matters more now that so many current models are LLM-encoded - they read your prompt as an instruction rather than a bag of CLIP tags, so having an actual LLM write well-structured sentences instead of wrestling tag soup is a real workflow upgrade.

How it works

The node loads your API key from a config file at startup, then on every execution builds a request and POSTs it with requests (the only dependency the code actually imports, more on that below). It retries with exponential backoff and honors Retry-After on 429s - good, because you will hit rate limits.

The request_type toggle picks which endpoint you hit. completion (the default) formats your prompt into a mock System: ... / Human: ... / AI: chat and calls /completions. chat sends a proper messages array (system message, full history, then your prompt) to /chat/completions. Both work; chat is the one that behaves like a real conversation, completion is the author's "fake it" mode and he admits he hasn't stress-tested it.

Conversation history is persisted to nodes/Nova/Nova.json in the pack folder, keyed by a UUID. That's the whole trick behind the node's one genuinely clever feature: continuity. You get a conversation_id out, paste it back into the conversation_id input, and the next call carries the context forward. No wiring extra state nodes.

Inputs and outputs that matter

Most fields you can leave alone. The ones you'll actually touch:

• model - five Llama 3.1/3.2 instruct models, default Meta-Llama-3.1-8B-Instruct. The 8B is fine for short prompt generation; 405B is a flex.
• max_tokens (default 100) - how long the answer can be. Bump it if your output keeps getting cut off.
• temperature (default 0.7) - the usual creativity knob.
• system_message - set the persona, e.g. "you are a prompt engineer writing one paragraph for a photorealistic image".
• conversation_id - paste the output from a previous run to continue that context.

It returns three things: generated_text (the string you wire into a Show Text node or straight into your text encoder), token_count (real usage.total_tokens in non-streaming mode; a chunk-by-chunk guess in streaming), and conversation_id again so you can loop it back.

Installing it

ComfyUI Manager, search "SambaNova", install, restart. Or:

cd ComfyUI/custom_nodes
git clone https://github.com/Apache0ne/SambaNova

Then grab a key at https://cloud.sambanova.ai/apis. Here's the trap: the repo doesn't ship the config file the node expects. The code looks for nodes/Nova/SambaNovaConfig.ini, and if it's not there, the node raises "API key is not set in the SambaNovaConfig.ini file." You have to create it yourself:

[API]
key = your_key_here
base_url = https://api.sambanova.ai/v1
max_retries = 3

The README glosses over this in one line, so expect to trip on it. Also ignore the scary requirements.txt - it declares pydantic, fastapi, sentry_sdk, openai, tiktoken and anthropic, but the shipped code imports none of them. Only requests is actually needed; Manager installs the whole list anyway.

Gotchas

The 405B's 8K context cap is genuinely low, and at 10 RPM you can blow your budget in one loop. The 70B has a 64K cap but the README is honest that quality fades around 16K and falls off a cliff at 32K - so keep conversations short. stream exists as an input but it's pointless in ComfyUI: nodes run linearly, so you can't watch tokens appear anyway; the author marks it WIP. And one serious note: this is an API-key-holding LLM node, the exact category that got people burned in the ComfyUI_LLMVISION incident. Install from the real repo above, not from some random "improved fork," and don't expose your ComfyUI to the internet.