Chat Musician

This is the pack's strangest node and one of its most fun. ChatMusician is an open-source LLM that was fine-tuned to compose music - it outputs ABC notation, a text format for sheet music - and this node takes that text and actually renders it into a playable waveform. You describe what you want in words ("a sad waltz in A minor") and get back both a text response and audio. There's a demo page linked in the README with example prompts worth stealing.

What it's for

Music idea generation, especially when you want something a text-to-audio model like AudioLDM-2 won't give you: structured music with actual chords, tempo, and key. Because it composes symbolically (notation) rather than synthesizing sound directly, it can write a coherent melody with a chord progression, and the node renders that to audio for you. It won't be studio quality - it's a novelty with real utility for sketching.

How it works

You hand it a model handle from LLMLoader - the node itself has no model download; you supply the ChatMusician GGUF. The README points at ChatMusician GGUF files with Q5_K_M or Q5_K_S recommended, dropped into models/LLavacheckpoints like any other GGUF. The node wraps your prompt with instructions to "write exactly one complete ABC notation tune" with headers and body, then parses the ABC text and renders it to a waveform at your chosen sample_rate (44100 default).

The inputs that matter

• prompt - your musical request. Chord-conditioned prompts work great: "write a tune over Am, F, C, G" is the author's own example.
• model - the ChatMusician GGUF via LLMLoader. This is the load-bearing input.
• max_tokens - 512 default; a full ABC tune can need more, so don't be shy.
• seed - standard reproducibility.
• The sampling knobs (temperature, top_p, top_k, penalties) are the usual LLM set.

Outputs

Three of them: response (the ABC notation text), wave_form (audio), and sample_rate. Wire the audio pair into SaveAudioNode to write a file.

Install

cd ComfyUI/custom_nodes
git clone https://github.com/gokayfem/ComfyUI_VLM_nodes
python -m pip install -r ComfyUI/custom_nodes/ComfyUI_VLM_nodes/requirements.txt

Plus the llama.cpp backend wheel, and the GGUF itself.

The README's own warning, verbatim

It does NOT work perfectly, if you got errors accept the error and queue prompt again with the same settings!!

That's the author writing a BIG BIG BIG warning, and it's real: generation can fail or produce malformed ABC on the first attempt. The fix is literally to queue again - this is one of the few nodes where "press run twice" is the documented troubleshooting step. If the ABC renders to silence or errors, try again, and lower temperature if the output keeps coming back structurally broken. It's a fun toy that occasionally needs a nudge, and the README is refreshingly honest about it.