Nodes/MiniMax Music3 Semantic Studio/Music3 Semantic Studio
ComfyUI Node

Music3 Semantic Studio

How Semantic Studio compiles a song plan

By ukr8b3g-cmyk·Created about a month ago·Updated 23 days ago· 4
Music3 Semantic Studio
  • clip
  • CONDITIONING
  • seconds
project_json{"schema_version":1,"project_id":"","global":{"title":"","genre":"Pop","subgenres":[],"bpm":120,"key":"","scale":"","meter":"4/4","mood":"","production":"","vocal":{"mode":"vocal","gender":"","timbre":"","delivery":"","harmony":"","effects":""}},"timeline":{"sections":[{"id":"intro-1","type":"Intro","label":"Intro","duration":8.0,"energy":0.2,"lyrics":"","instruments":["piano","pad"],"vocal":"instrumental","directives":"Sparse opening; establish the main tone without a full groove."},{"id":"verse-1","type":"Verse","label":"Verse 1","duration":24.0,"energy":0.38,"lyrics":"","instruments":["piano","bass","light drums"],"vocal":"soft","directives":"Keep the arrangement restrained and leave space for the lead vocal."},{"id":"chorus-1","type":"Chorus","label":"Chorus 1","duration":24.0,"energy":0.82,"lyrics":"","instruments":["full drums","bass","guitar","piano","pad"],"vocal":"power","directives":"Open into a wider, fuller arrangement with a clear melodic lift."},{"id":"verse-2","type":"Verse","label":"Verse 2","duration":24.0,"energy":0.48,"lyrics":"","instruments":["piano","bass","drums","guitar"],"vocal":"soft","directives":"Retain momentum from the chorus while returning to a lighter texture."},{"id":"chorus-2","type":"Chorus","label":"Chorus 2","duration":24.0,"energy":0.88,"lyrics":"","instruments":["full drums","bass","guitar","piano","pad"],"vocal":"power","directives":"Repeat the chorus identity with slightly more density and backing support."},{"id":"bridge-1","type":"Bridge","label":"Bridge","duration":16.0,"energy":0.45,"lyrics":"","instruments":["piano","strings","pad"],"vocal":"intimate","directives":"Pull back the groove and create contrast before the final lift."},{"id":"chorus-3","type":"Chorus","label":"Final Chorus","duration":28.0,"energy":1.0,"lyrics":"","instruments":["full drums","bass","guitar","piano","strings","pad"],"vocal":"power","directives":"Peak arrangement density and emotional intensity; broaden the stereo image."},{"id":"outro-1","type":"Outro","label":"Outro","duration":12.0,"energy":0.3,"lyrics":"","instruments":["piano","pad"],"vocal":"fade","directives":"Release the energy and finish with a clean, natural decay."}]},"audio_edits":[],"takes":[],"conditioning_tracks":[]}
seed0
max_duration160.00
cfg_scale1.5
top_k50

The hard part of local music generation was never the audio, it was describing the song. You get a text box, you cram "dreamy indie pop, 120bpm, verse then chorus, female vocal, warm bass" into it, and you re-roll until something lands. Music3 Semantic Studio is the opposite approach: it's an authoring workspace where you build the song as a structure - sections, durations, energy, instruments, lyrics - and the node compiles all of it into the conditioning MiniMax Music 3 actually consumes. One good thing to internalize up front: this is an editor, not a generator. It writes the brief and hands it over. Everything else in your graph is unchanged.

What it's doing under the hood

MiniMax Music 3 is native to ComfyUI, and this node doesn't reimplement any of it. The source imports the core AR constants (comfy.ldm.minimax_music.ar) for its limits and passes your compiled text to the connected CLIP's Music3 tokenize, then encode_from_tokens_scheduled. It deliberately has no model allowlist: if the CLIP you wired in speaks the Music3 contract, it runs; if it doesn't, the failure surfaces from the core call.

The compile step is where the design work pays off. Global fields (genre + subgenres, BPM/meter, key, mood, an energy arc generated from your section energies, production profile) become a ### Global Metadata block; vocal settings become ### Vocal Details; then every section is rendered as one line - label, start–end target time, an energy phrase, the instrument list, vocal treatment, your own directives. That caption goes out as the CONDITIONING output, and your per-section lyrics become a tagged lyrics document ([Verse], [Chorus], …) with duplicate leading tags stripped so you don't double-tag.

Inputs you'll actually touch

  • clip (CLIP) - required, and it must be a Music3-compatible text encoder. This is the one wire that decides whether the node works at all.
  • project_json ("Studio Project JSON") - the project state, defaulting to a working eight-section pop template. It's advanced/hidden behind the UI in practice: Open Semantic Studio is where you edit it. Malformed or unknown fields get normalized and warned about at execution; unparseable JSON is ignored for that run and replaced with a minimal project rather than crashing your queue.
  • seed ("Music Seed (AR)") - AR-stage randomness, with a control-after-generate so you get the randomize/fixed/increment widget.
  • max_duration ("Duration") - the AR ceiling in seconds, default 160, capped at 360, step 0.04 (audio frames). The model can finish earlier; the Generation tab also has an Auto Sync toggle that keeps this in step with your timeline total.
  • cfg_scale ("Music CFG (AR)", default 1.5) and top_k ("Music Top-K", default 50) - advanced, and honestly: leave them. You'll get more out of rewriting a section than out of nudging these.

Outputs: CONDITIONING → KSampler positive, and seconds → the duration input of Empty MiniMax Music3 Latent Audio, so your latent matches the length the model planned. Negative conditioning is the classic Conditioning Zero Out; the README's own diagram has no negative-prompt slot for this node.

Install

cd ComfyUI/custom_nodes
git clone https://github.com/ukr8b3g-cmyk/MiniMax-Music3-Semantic-Studio.git

Restart. ComfyUI Manager will install it too - search the pack title, MiniMax Music3 Semantic Studio - it clones the same repo. No mandatory Python dependency - the UI is bundled web assets, and the pack's pyproject.toml lists none. There's a ready-made workflow in the repo (workflows/MiniMax_Music3_Semantic_Studio_V1.json) if you'd rather not wire the graph by hand.

Traps worth knowing

Two different seeds, two different knobs. Music Seed (AR) controls the autoregressive token pass; KSampler's seed controls diffusion noise. Changing one does not replace the other, and blaming the wrong one is the most common confusion here. Same for CFG, same for duration.

Everything is a target, not a contract. BPM, key, section boundaries, energy and instrument lanes are guidance. The instrument lanes are not stems - switching off "guitar" in the chorus means the model isn't asked for guitar, it doesn't mute a guitar track. The energy curve is pre-generation intent, not a fader.

A couple of hard boundaries. Section structure is capped at 32 sections, and section types resolve to canonical lyric tags - anything unrecognized lands as [Instrumental]. Set global vocal mode to instrumental and every lyric you typed is dropped by design. Cross 300 seconds of timeline and the compiler warns you that long-form behavior depends on the implementation under it.

Categorymodel/conditioning/minimax music

Inputs (6)

NameTypeDefaultDescription
clipCLIP
project_jsonSTRING{"schema_version":1,"project_id":"","global":{"title":"","genre":"Pop","subgenres":[],"bpm":120,"key":"","scale":"","meter":"4/4","mood":"","production":"","vocal":{"mode":"vocal","gender":"","timbre":"","delivery":"","harmony":"","effects":""}},"timeline":{"sections":[{"id":"intro-1","type":"Intro","label":"Intro","duration":8.0,"energy":0.2,"lyrics":"","instruments":["piano","pad"],"vocal":"instrumental","directives":"Sparse opening; establish the main tone without a full groove."},{"id":"verse-1","type":"Verse","label":"Verse 1","duration":24.0,"energy":0.38,"lyrics":"","instruments":["piano","bass","light drums"],"vocal":"soft","directives":"Keep the arrangement restrained and leave space for the lead vocal."},{"id":"chorus-1","type":"Chorus","label":"Chorus 1","duration":24.0,"energy":0.82,"lyrics":"","instruments":["full drums","bass","guitar","piano","pad"],"vocal":"power","directives":"Open into a wider, fuller arrangement with a clear melodic lift."},{"id":"verse-2","type":"Verse","label":"Verse 2","duration":24.0,"energy":0.48,"lyrics":"","instruments":["piano","bass","drums","guitar"],"vocal":"soft","directives":"Retain momentum from the chorus while returning to a lighter texture."},{"id":"chorus-2","type":"Chorus","label":"Chorus 2","duration":24.0,"energy":0.88,"lyrics":"","instruments":["full drums","bass","guitar","piano","pad"],"vocal":"power","directives":"Repeat the chorus identity with slightly more density and backing support."},{"id":"bridge-1","type":"Bridge","label":"Bridge","duration":16.0,"energy":0.45,"lyrics":"","instruments":["piano","strings","pad"],"vocal":"intimate","directives":"Pull back the groove and create contrast before the final lift."},{"id":"chorus-3","type":"Chorus","label":"Final Chorus","duration":28.0,"energy":1.0,"lyrics":"","instruments":["full drums","bass","guitar","piano","strings","pad"],"vocal":"power","directives":"Peak arrangement density and emotional intensity; broaden the stereo image."},{"id":"outro-1","type":"Outro","label":"Outro","duration":12.0,"energy":0.3,"lyrics":"","instruments":["piano","pad"],"vocal":"fade","directives":"Release the energy and finish with a clean, natural decay."}]},"audio_edits":[],"takes":[],"conditioning_tracks":[]}Semantic Studio project state. Recoverable malformed or future fields are normalized at execution time.
seedINT00–18446744073709550000AR-stage randomness. This is separate from the KSampler noise seed.
max_durationFLOAT160.000.04–360Maximum duration in seconds; the model can end the song earlier. Saving the Studio timeline can synchronize this to the section-duration total.
cfg_scaleFLOAT1.50–100
top_kINT501–16384

Outputs (2)

NameTypeDescription
CONDITIONINGCONDITIONING
secondsFLOAT