NInfer - Chat

This is the node the whole pack exists for: one chat completion against a local Qwen3.8-27B, with the reply coming back as a string you can wire into the rest of the graph.

A local model in the graph is uncensored, offline, and free per call, and it can read your reference images directly. The catch is VRAM, which is what the rest of this pack spends its time managing.

What actually happens on a run

The node builds an OpenAI-format messages array and posts it to NInfer's /v1/chat/completions. Three details worth knowing because they explain most surprises:

Settings get split in two. The openai SDK validates its kwargs against a fixed list and raises TypeError for anything it doesn't recognise. So the node separates your merged options into SDK-native parameters (temperature, top_p, max_tokens, reasoning_effort, seed, …) and everything else, which is forwarded through the SDK's extra_body escape hatch so it still reaches the server. That's how top_k gets through at all.

Images become PNG data URLs. Each connected image_N socket is converted from a ComfyUI IMAGE tensor into base64 PNG and appended to the user message as content parts, in slot order; batches are flattened. Each 512px image costs roughly 256 prompt tokens, and the author's testing had colour-role conditioning still accurate at ten images on a 4090 - four slots is the comfortable default.

Reasoning is a separate channel. choice.content becomes Response; message.reasoning_content becomes Reasoning. A thinking model's scratch work doesn't land in your prompt text. The node also prints a token-usage preview into its own body - prompt tokens, cached tokens, completion tokens, and how many of those went to reasoning. That preview is your first diagnostic when an answer comes back empty.

Inputs that matter

• Client - from the Client node. Required.
• Prompt - the user turn. Empty is a validation error, not an empty generation.
• Model - leave it blank. The node then asks the server what it's serving and uses the first id it reports. The id changes with the artifact (qwen3.8-27b for one build, a long Huihui name for another), so hardcoding it is how you get HTTP 404 - the same status you'd get from a missing route, which is why that error is so confusing. On the plus side, the old trap of wiring the Server Control status report into this field now raises a readable error instead of sending the text as a model id.
• Force Regen - on by default, and you want to understand it. ComfyUI caches hard; with this off, re-running a graph whose inputs haven't changed returns the cached answer instead of paying for a new generation. Internally the node's fingerprint returns the current time whenever this is on, which is the standard "always dirty" idiom for a node that must fire every run - and it costs you the cache for everything downstream of it.
• Options - the chain from the option nodes. Optional; drop it and you get server defaults.
• History - the conversation so far, from a previous Chat node's History output. Feed it back for multi-turn.

Optional image_1 … image_4 are cascading: wire them in order, since slot 2 does nothing useful until slot 1 is connected. NInfer ships with --vision off by default; this pack passes it, because the vision tower is only 0.38 GiB and the overlay residency mode borrows VRAM only while an image is being encoded.

Outputs

Response is the visible reply - this is the one you send to a prompt-enhancer consumer, a Save Text node, or anything expecting a STRING. Reasoning is the hidden thinking. History is a NINFER_HISTORY payload containing the whole conversation including this turn, designed to loop back into another Chat node's history input.

Because this node isn't an output node, ComfyUI won't run a graph that ends here. Wire Response into a Preview Any or Save Text node, or the run does nothing at all.

Install and first run

cd ComfyUI/custom_nodes
git clone https://github.com/kaerozhi/comfy-ninfer-api
# restart ComfyUI, then load workflows/minimal_ninfer_chat.json from the pack

Client → Chat → Preview Any, Model blank, everything else default. If that doesn't work, nothing bigger will.

Where people get burned

Empty reply, no error. The classic. Qwen3.8-27B thinks before it answers by default, and with a modest Max Tokens every token can be spent on hidden reasoning, leaving Response blank while Reasoning is full of text. Check the usage preview, then raise Max Tokens or lower Reasoning Effort.

Timeouts on long answers. The Client's 900 s default is there because these run for minutes; shorten it and you'll meet it.

A stopped server with Auto Start off. Then nothing answers and the call fails - that's the setting doing its job, not a bug.