ROCm LLM Chat

What this thing actually is

ROCmLLMChat is an HTTP client with a ComfyUI face bolted on. You point it at an OpenAI-compatible chat-completions server running somewhere - llama.cpp's server, vLLM, Ollama's /v1, Halogen's flash-server - type a prompt, and it hands back the model's answer as a STRING you can wire into anything. No model is loaded inside ComfyUI. No weights, no VRAM.

The name is a little misleading: nothing about the chat call is AMD-specific, and it runs fine on an NVIDIA box. What it buys you is an LLM in the graph instead of in another tab.

It's a different job from the text encoder already inside your checkpoint - Qwen3 on Z-Image, T5 on Flux. This one sits upstream of the sampler, writing the prompt the encoder then reads. Don't expect image quality to improve just by wiring it in; it fixes the blank-page problem.

How it works

Plain stdlib urllib - no requests, no openai package, no new dependency. It normalises server into a /v1/chat/completions URL (forgiving a bare host or a full endpoint), POSTs one non-streaming JSON body, and reads back choices[0].message.

The structural thing to internalise: the request comes from the ComfyUI backend process, not your browser. So server resolves on the machine running ComfyUI - if your UI is remote, 127.0.0.1 means the ComfyUI host, not your laptop. Use the LLM host's LAN name when they differ.

The inputs that matter, and the outputs

You set server and prompt; everything else can be ignored until something breaks. system_prompt is added as a system message only if the history doesn't already carry one - which matters when you chain nodes. model is the name the server advertises (GET /v1/models) - empty takes its default, and the node warns if you asked for one name and got served another. temperature (0.7) and top_p (0.95) are ordinary sampling knobs. max_tokens (2048) includes reasoning tokens - the cause of the most confusing failure below. api_key goes out as a bearer token (or from ROCM_LLM_API_KEY); blank is fine for an unauthenticated local server.

seed gets sent to the server - some honour it, many ignore it - but what you can count on is it busting ComfyUI's execution cache. Bump it when you want a fresh answer to an unchanged prompt.

Three outputs. response is the assistant text, ready for CLIPTextEncode. thinking is the server's reasoning_content across all steps, empty on models that don't reason. chat_history is the message list as sent plus this reply, typed CHAT_HISTORY so another ROCmLLMChat can consume it - no loops exist in a graph, so a conversation is a chain of nodes.

The tools bit is the interesting part - and the scary one

tools starts at none and steps up through read, read_command, read_write, full. With anything but none the node ships OpenAI-format tool definitions and executes whatever the model asks for until it answers or max_tool_steps (default 8) runs out. read is read_file / list_dir / search_files; full adds write_file and a persistent run_command shell where cd and export carry between calls.

File tools are confined to path_root (empty = the ComfyUI directory) with realpath containment, so .. and symlinks can't escape - and if system_prompt is empty the node injects its own agent prompt naming the real host, root and tools. The shell is not confined. It runs with your ComfyUI user's permissions and no sandbox, so only enable full for prompts you wrote yourself.

Installing it

ComfyUI Manager, search "RocM Ninodes". Or:

cd ComfyUI/custom_nodes
git clone https://github.com/iGavroche/rocm-ninodes

Restart. It lands under ROCm Ninodes → Generative AI; the CLI route is comfy node install rocm-ninodes. Upgrading from v1.x? Run uv run python tools/upgrade_to_v2.py first or the nodes may not register.

Honest caveat: the pack's requirements.txt pulls numpy, Pillow, psutil, gguf, safetensors, librosa and scipy. None of that is needed for chat; you're installing a whole toolbox to get an HTTP client.

Where people get burned

Empty response, full thinking. The whole budget went to reasoning. Raise max_tokens; the node prints a hint when it sees this. And a history containing tool messages only replays into a node that enables the same tools.

Connection refused. Backend-resolution again, plus the container classic: if the LLM lives in podman or docker, check it binds a reachable interface and prefer 127.0.0.1 over localhost - IPv6 through docker port forwarding resets connections. A cold server also loads its model on request one, so raise timeout past the default 300s if the first call dies.

Chat scaffolding in your conditioning. The node returns content verbatim, stripping nothing but whitespace. Ask for a bare prompt in the system message, or "Sure! Here's your enhanced prompt:" lands in your text encoder as literal tokens - the classic self-inflicted wound of every prompt-enhancer node.