MuScriptor Transcribe Audio to MIDI

If you've ever wanted to grab a real song - or a generated clip, or a hummed demo - and turn it into an editable MIDI file with separate instrument tracks, MuScriptor Transcribe Audio to MIDI is the node you've been missing. It wraps MuScriptor, the multi-instrument transcription model from Kyutai and Mirelo, and exposes it as a single ComfyUI node. No API, no key, no cloud upload: the transcription runs locally on your machine. Drop in a WAV or MP3 (or wire in another node's AUDIO output), and out come a .mid file plus a structured JSON of every note event.

It's a genuinely different flavor of ComfyUI work - instead of pixels, you're making music data that other steps can consume. Pipe the MIDI into a soundfont renderer or a synthesizer node later in your graph and you've got a generative-audio loop, not just a one-off conversion.

How it works

The node is a thin wrapper around the upstream MuScriptor package. On your first run it downloads the model weights from HuggingFace (MuScriptor/muscriptor-medium, plus small and large variants) and caches them locally. The model treats transcription as a token-generation problem: audio is chunked, each chunk is decoded into MIDI events, and the node stiches the chunk boundaries together into one continuous file. You'll see progress printed per chunk in the ComfyUI console.

Two things are worth knowing before you use it. First, the model weights are gated - you must accept the license on the HuggingFace model page and authenticate before the first download will work. Second, the model is cached in memory keyed by (model_size, device, dtype), so a second run with the same settings skips loading entirely and just transcribes. Switch sizes or precision and it reloads.

The inputs that matter

You only really touch a handful of these:

• audio / audio_path - feed one or the other. audio takes priority if connected (it's the standard ComfyUI AUDIO type); otherwise audio_path takes a local file path. If you give it neither, the node errors out.
• model_size - small (103M), medium (307M, default), or large (1.4B). Medium is the sensible default; large is for when you're chasing every last note and have the VRAM to spare.
• device and dtype - leave both on auto unless you know better. auto picks fp16 on Mac (MPS) and fp32 elsewhere, which is a sensible balance.
• use_sampling + temperature - greedy decoding by default, which is deterministic and usually fine. Flip sampling on with a temperature around 0.8–1.0 when you want to experiment.
• instruments - a comma-separated or newline list like electric_guitar, drums, voice to restrict what gets transcribed. Handy for isolating tracks from a full mix, and a real differentiator for this model.

The decoding knobs (cfg_coef, beam_size, prelude_forcing, strict_eos) mostly live at defaults and do what they say on the tin: beam search width ≥2 trades speed for quality, prelude_forcing (on by default) improves chunk-boundary continuity and forces a batch size of 1, and strict_eos turns a silent chunk failure into an error instead of a shrug.

The outputs

• midi_path - absolute path to the saved .mid file in ComfyUI's output directory (named muscriptor_transcription_<timestamp>.mid unless you change filename_prefix). This is what you hand to a player, a DAW, or a downstream MIDI-processing node.
• notes_json - a JSON string with every decoded note as {pitch, start_time, end_time, instrument}, sorted by start time then pitch. This is gold if you want to filter, quantize, or visualize the notes programmatically without parsing MIDI bytes.

Install

The standard route: ComfyUI Manager → search "ComfyUI-MuScriptor" → install, then restart. Or manually:

cd ComfyUI/custom_nodes
git clone https://github.com/martyyz-ai/ComfyUI-MuScriptor.git comfyui-muscriptor
pip install -r comfyui-muscriptor/requirements.txt

Then the part people skip: accept the license on the MuScriptor-medium HuggingFace page, and authenticate:

export HF_TOKEN=hf_...   # or: hf auth login

Restart ComfyUI and the node appears under the MuScriptor category as "MuScriptor Transcribe Audio to MIDI".

Gotchas

• The HF token is mandatory, not optional. Forget it and the first run dies with a gated-repo auth error. The README says this loudly; it's still the #1 way this node bites people.
• Dependencies are on the heavier side. The pack pulls in fastapi, uvicorn, and a bundled server alongside the transcription code - this is a full port, not a lean single-purpose node. If you'd rather avoid the extra packages, there's a competing "zero extra Python dependencies" MuScriptor port (by jtydhr88) that's worth comparing; quality is reportedly similar.
• First run is slow. It downloads the model and builds it on the fly, so budget a few minutes before you judge it. Subsequent runs are much faster thanks to the in-memory cache.
• Sample the official audio examples at muscriptor.github.io to sanity-check that your output quality is in the right ballpark before you start debugging your own files.