Live Chat Stream

Most prompt-enhancer nodes do one thing: your rough idea goes in, a rewritten prompt comes out, you press Queue. Live Chat Stream is the weirder version. There's a real chat box inside the node, the reply streams in token by token, and the moment the model finishes its <prompt> block - while it's still typing the rest of the answer - the graph fires and your image starts sampling.

What it actually is

Not a model, and not a node that runs an LLM in-process. nodes.py is almost aggressively boring: run() strips whitespace off some widget strings and hands them back as STRINGs for CLIP Text Encode to eat. Nothing loads weights in there.

The real work is split in two. The browser half (web/live_chat_stream.js) owns the chat panel, parses the stream, writes the extracted prompt into the node's widgets, and calls app.queuePrompt(0, 1) itself when the prompt looks complete. The server half (server.py) registers /live_chat_stream/* routes and relays Ollama's /api/chat NDJSON stream through ComfyUI instead of letting the browser hit Ollama directly - that trip runs into CORS and mixed-content trouble. The name is backend-neutral on purpose; Ollama is the only backend today.

Two lanes, and that's the interesting idea

Lane A is the sane one: the <prompt> and <negative> blocks of the reply become positive / negative, a clean image prompt. Lane B is the fun one: chat_p / chat_n use the model's own thinking, reply, or <mind> block as the prompt, unedited, so the picture shows what the character is feeling rather than what it politely offered to draw. You pick the lane-B source in Settings, and there's a 400-character cap on it - a wall of prose makes CLIP fall apart.

Timing has three modes: when </prompt> closes (the default, and the one that earns the name), live while streaming, or manual.

Inputs and outputs you'll actually touch

Every input here is an optional widget, and the chat UI hides most of them from the canvas - they're plumbing, not knobs you set:

• prompt_text, negative_text - lane A, filled by the frontend from the reply.
• chat_p_text, chat_n_text - lane B, same deal.
• response_text, thinking_text - the full reply and the model's reasoning. thinking stays empty unless the model supports thinking (the request sends think: true; if the model rejects it, the frontend retries once without).
• image - an IMAGE socket for img2img, and it beats anything you drop or paste onto the node. input_image_name is the internal filename for a dropped image, uploaded to input/live_chat_stream/; you don't set that by hand.

Outputs are positive, negative, response, chat_p, chat_n, thinking (all STRING, straight into text encoders) and image, which goes to VAE Encode for i2i. One thing that looks like a bug and isn't: with no image available the image output is an ExecutionBlocker, so the i2i branch just doesn't run instead of sampling garbage.

Install

No Python dependencies at all - pyproject.toml declares an empty dependencies = [], and the only third-party imports are aiohttp, PIL, numpy and torch, all of which a normal ComfyUI already has.

cd ComfyUI/custom_nodes
git clone https://github.com/ketle-man/ComfyUI-LiveChatStream
# restart ComfyUI - or search "ComfyUI-LiveChatStream" in ComfyUI Manager

What you do need is Ollama running with models pulled: a chat model at minimum, a vision-capable one if you want to attach images to the chat (VLM), and for the optional judge, a decision model - Ollama 0.35.1 or later, since the judge route is /v1/systemone. Then open workflows/live_chat_stream.json: an SDXL plus LCM/DMD-LoRA setup at 5 steps and 512×512 with both lanes wired to their own sampler, so you can see A and B side by side.

Where it bites

VRAM, twice over. You're now keeping a chat LLM resident on the same card as your diffusion model. The fix is VRAM: auto-free with a target (start at 8 GB): before each generation the server measures free VRAM from nvidia-smi, adds back ComfyUI's unused torch cache, then unloads the biggest Ollama models one at a time until the target is met - never the one that's mid-stream, because that would truncate your reply. On non-NVIDIA hardware it falls back to ComfyUI's own memory figure, which the author's comments say under-reports what Ollama holds, so auto-free is a guess there.

The Ollama address is server-side, always. It comes from LIVE_CHAT_STREAM_OLLAMA_URL or <ComfyUI user dir>/live_chat_stream/config.json ({"ollama_url": "http://192.168.1.20:11434"} - edits apply without a restart), never from the browser. That's the fix if Ollama lives on another box, and it's a deliberate hardening: anyone who can reach your ComfyUI can chat through your Ollama, but can't point it anywhere else, and redirects aren't followed. Don't run --listen on a network you don't trust.

Models that won't follow the format. Lane A depends entirely on the model emitting <prompt>...</prompt>. A chatty model that ignores your system prompt gives you an empty positive and a sampler running on nothing. Presets - including structured characters with personality, appearance and emotion rules, saved to presets.json in the user dir - are how you keep it on rails.

And keep the wider context in mind: the local-LLM-in-the-graph crowd keeps asking for in-graph GGUF loaders so they don't have to run a second process. This pack is the other camp - external Ollama server, two models, one card - and the VRAM juggling above is the price of that choice.