Muse (GGUF)

"Muse (GGUF)" isn't a describing node - it's a brainstorming node. Where the VLM Prompt nodes look at your media and tell you what's there, Muse invents. You pick a genre, a mood, a shot type, an artist, and a subject; it writes you a full concept with a ready-to-use image prompt. It's the "I have writer's block at 11pm" node, and it's genuinely good at shaking something loose.

The whole thing is constraint-driven. Every dropdown defaults to "Random", and the pools are deep - 79 genres, 50 moods, 114 artists, 63 subjects - so a straight run is a lottery ticket. The fun is narrowing it: pick "Noir Photography" + "Melancholy" + "Dutch Angle" + "Vivian Maier" and let the model do the rest.

The inputs that matter

• genre, mood, shot_type, artist, subject - the five constraint dropdowns, each with a "Random" default.
• free_hint - a freeform word, phrase, or constraint ("cyberpunk detective with a broken umbrella"). Leave empty to dream freely, per the tooltip.
• aesthetic_bias - 12 output styles, from "Stable Diffusion / Tag Style" to "3D Render / Octane". The standout is "Midjourney v6/v7 - Painterly Realism", which the tooltip recommends pairing with the MidjourneyV1Z LoRA.
• output_format - JSON + Prompt (default), Prompt Only, or JSON Only. This is how the two outputs split.
• image (optional) - a style reference. reference_mode then decides: Style Only invents a new scene matching the visual language, Style + Subject may keep the subject too.
• max_tokens (default 900), keep_model_loaded, seed.

It has two outputs: prompt (STRING) - the extracted prompt, ready to paste or wire forward - and json (STRING), the full concept structure. In Prompt Only mode the prompt pin carries the text and JSON mode returns the structure alone.

How it works

Muse reuses the same GGUF engine as the VLM Prompt nodes, in text-only mode - the default model is the same Gemma4-E4B-uncensored-Q4_K_M, and it shares the whole GGUF dependency situation. It builds your constraints into a structured request, asks the model for a concept (in JSON), then extracts the prompt field for the STRING pin. On a video-style reference the engine can also pull a random pose/action hint from its built-in pose bank to keep sequences from going stale.

Install

ComfyUI Manager, or:

cd ComfyUI/custom_nodes
git clone https://github.com/mexxmillion/ComfyUI-VLMPrompt.git
pip install -r ComfyUI-VLMPrompt/requirements.txt

Restart ComfyUI. The model and its mmproj auto-download to ComfyUI/models/llm/GGUF on first run, and - same caveat as every GGUF node here - llama-cpp-python is not in requirements.txt, so you need a multimodal build with the VLM chat handlers for this node to load at all. The muse data (genres, moods, artists…) lives in muse_data.json and pose_bank.json, so you can curate the pools without touching Python.

Common issues

• Everything feels samey after a while - the pools are finite and a small 4B model lands in familiar ruts. Swap model_name to the Qwen options or add a free_hint to break out.
• Node missing after install - the llama-cpp-python handler problem again, not Muse itself.
• Reference image ignored - a style reference only does something when the mmproj is actually loaded; check the console for the "no mmproj" warning.