Muse Chat

If you've ever drafted a prompt in one tab, alt-tabbed back to ComfyUI, run it, hated the result, and repeated that for an hour, Muse Chat is aimed straight at you. It's a full local-LLM chat panel that lives inside a node on your canvas - LM Studio's chat window folded into the graph, so you brainstorm prompts without leaving the workflow or fighting two apps over VRAM.

What it actually is

Here's the thing that trips people up first: Muse Chat has no graph inputs or outputs. Open its info_schema and it's empty on both sides. That's not a bug - it's the design. The node is a window onto a chat service, not a data-processing step: you chat, you copy the prompt you like, you paste it into your prompt node. The class itself is a noop that returns instantly and is flagged as an output node, so it's safely ignored by Queue Prompt and never interferes with execution.

Under the hood it's two halves: the Python side registers aiohttp routes under /muse/* on ComfyUI's own PromptServer, and the web/ folder serves the chat UI. The node is the front door; chat logic and backend plumbing live in backends.py, chat_store.py, and friends. Chats are saved as JSON under comfyui-muse/chats/, so conversations survive restarts.

How it works

Three backends, switchable on the fly:

• LM Studio (http://localhost:1234) and Ollama (http://localhost:11434) - the "bring your own server" path. If you already run one, this is zero setup: pick the backend, hit ⟳ to list models, and start typing.
• Direct (GGUF) - a built-in loader that spawns llama.cpp's own llama-server and skips LM Studio/Ollama entirely. One click downloads a matching binary from the official llama.cpp releases, then you point it at a folder of .gguf files; mmproj projectors pair up automatically for vision/audio models. Since the engine is llama.cpp - the same one LM Studio wraps - you get the same speed, GGUF's Q4–Q8 quantization ladder doing its thing rather than a hacky reimplementation.

The genuinely clever part is the VRAM coordination - the classic "LLM in one app, diffusion in another" mess on a single GPU. Two toggles sort it out: Free ComfyUI VRAM unloads ComfyUI's models and clears its cache before each chat message, and Unload on Run drops the chat model - polling the backend to confirm it's actually freed - before your render proceeds. No more "remember to unload before you hit Run."

What you actually touch

No inputs or outputs to wire, so your settings live in the panel instead:

• Backend + base URL - the only thing you must get right. If you use Direct, click through the loader settings once to download llama-server and add a model folder.
• Per-chat system prompt, model, and Guide Materials - each session keeps its own; guide files in ComfyUI/input/ get prepended to every message, read fresh from disk.
• Max reply tokens (default 2048) - raise it when "write 5 prompts" gets cut off; a hint appears when a reply hits the cap.
• Context length / GPU layers - Direct loader only. Default context is 8192; sizing it to a model's full 128k advertised window is what actually OOMs small GPUs. GPU layers defaults to -1 (everything), with a Fit to GPU helper and an automatic retry at lower offload levels if a load fails.

The output is the one thing there's no field for: text, on your clipboard.

Installing it

Via ComfyUI Manager, search comfyui-muse (or "Muse"). Or the manual way:

cd ComfyUI/custom_nodes
git clone https://github.com/RudySen/comfyui-muse

Then restart ComfyUI and add the node via Add Node → utils → muse → Muse Chat. Notably, there's no requirements.txt to worry about - the core panel only needs aiohttp, which already ships with ComfyUI. Two things are opt-in: video attachments need opencv-python in ComfyUI's Python environment, and the Direct loader grabs its own llama-server binary on first use.

Where people get burned

• Expecting graph wiring. It has none - that's the feature. If you want an LLM that outputs a string into the graph, this isn't it.
• A backend you forgot to start. LM Studio and Ollama must actually be running and reachable at their default ports.
• Silent slowness on the Direct loader. If generation is laggy, check the Log panel: the classic culprit is a KV cache sized to a huge advertised context spilling into system RAM, especially on CUDA/Windows.
• Audio only works on Direct. As of this writing, neither LM Studio's nor Ollama's API accepts audio - Muse tells you in-chat and leaves it out.

It's a small pack from a solo author (posted as "Muse" on r/comfyui, Ko-fi in the README), so think of it as a sharp tool rather than a platform. But for its one job - iterating on prompts with a local model, next to your workflow, without the VRAM dance - it's a genuine quality-of-life upgrade.