M3 Song Planner (Local VLM)

Let's clear the biggest misunderstanding off the table first: this node never makes a sound. The display name says "M3 Song Planner (Local VLM)" and the pack name says MiniMax M3, but no audio comes out of it. What it does is automate the fiddly upstream chore MiniMax Music 3 demands of you: writing the structured music caption and the original lyrics that feed MiniMaxMusic3TextEncode. If you've opened a Music 3 workflow, stared at its caption format - three labeled sections, hundreds of words of production detail - and quietly closed the tab, this node is the local fix for that blank page.

MiniMax Music 3 is a text-to-music model, and like most good ones it wants a proper production brief, not a one-line "make a sad song." Writing captions by hand is a mini-job of its own, and the community's standing answer to "who writes the hard input" is a small LLM inside the graph - the same trick as the popular LTX-2 prompt node, just scoped to music. M3 Song Planner is that idea minus the prompt-guessing: you type an idea, it writes both ingredients, and both drop straight into the encode node.

How it works

It's a two-stage pipeline, and each stage is one run of a local VLM:

1. Caption generation - the VLM reads your idea plus an optional genre hint and writes the structured Music 3 caption: Global Metadata, Vocal Details, and a section-by-section Arrangement timeline.
2. Lyrics generation - the VLM reads your idea and the caption it just wrote, and produces section-tagged lyrics ([Intro], [Verse], [Chorus]…). Structure scales with duration_seconds, so a 120-second brief doesn't get a five-minute song's sections.

The mechanism is what keeps this cheap and dependency-free. The node uses ComfyUI's native CLIP text-generation stack - clip.tokenize → clip.generate → clip.decode, the same path as the core Generate Text node (comfy_extras/nodes_textgen.py). No API calls, no Ollama, no second process. It sniffs whether your checkpoint is a Qwen or Gemma, builds the right chat template, and strips the echoed prompt. The two system prompts live in plain text files (prompts/caption_system.txt, lyrics_system.txt) that hot-reload on every run, so you can retune the writing style without restarting ComfyUI.

The inputs that matter

You'll actually set three of these, and the rest you'll probably leave alone:

• idea - the only thing you must type. A mood, a concept, a story fragment; it raises a clear error if empty.
• text_encoder - pick a generative VLM checkpoint, or wire the optional clip input from a CLIPLoader and the dropdown is overridden.
• duration_seconds (30–300, default 120) - sets the target length and the structure/token budget.

Beyond those: genre_hint steers the caption, vocal_config (female/male/duet/instrumental/auto) and language (English/Chinese/Korean/Japanese/auto) do what they say, and the sampling cluster - seed, temperature, top_p, top_k, max_tokens - behaves like every other text-gen node you've met. The lyrics stage uses seed + 1, so a seed change ripples both stages.

Outputs: caption and lyrics (both STRING), wired to the caption and lyrics inputs of a MiniMaxMusic3TextEncode in a Music 3 workflow, plus a debug STRING holding a JSON dump of timings, clip source, model family, and warnings. Stage failures degrade to warnings instead of killing the run.

Installing it

cd ComfyUI/custom_nodes
git clone https://github.com/benjiyaya/ComfyUI-MiniMax-M3-SongPlanner.git minimax-m3-planner

Restart ComfyUI and it appears under M3/SongPlanner (ComfyUI Manager finds it by pack title too). requirements.txt is comments only - genuinely nothing to pip install.

The real requirement is the model. You need a generative VLM text-encoder repack (.safetensors) in ComfyUI/models/text_encoders/:

• Qwen3-VL instruct repacks - recommended, verified generative.
• Gemma-3-Vision instruct repacks (≤12B) - also work.

The trap that burns people: Qwen2.5-VL repacks are vision-tower-only and are not generative. Load one and you get The CLIP ... does not support native text generation - the error you'll actually hit. Plain SD/SDXL CLIP files fail the same way. And because it's a V3-API node on a recent textgen stack, an old ComfyUI throws clip.tokenize/generate rejected the native arguments - that one means "update ComfyUI," not "this node is broken."

Two more practical notes. A 4–7B VLM wants roughly 5–16 GB of VRAM on top of whatever the music model needs; if you're crammed, flip the advanced keep_model_loaded to False so the VLM unloads between runs instead of sitting cached. Second, this is a fresh, niche pack - same caveat as every small LLM node in the ecosystem, so read the handful of files before the first run. This one is refreshingly boring: no network calls, no pip, just local prompt files and ComfyUI's own stack.

It's not magic - a small VLM is writing your brief, so expect clunkers. But it closes the gap between "I have an idea" and "I have a valid Music 3 input," entirely on your machine.