MIDI Transcribe Audio

The name undersells this one. "MIDI Transcribe Audio" sounds like a boring format converter, but it's actually the front half of the entire reason this pack exists: taking a song you like and turning it into something you can edit, then re-sing with a different voice. Drop any audio in, and you get back the pack's MIDI JSON - a JSON string, not a .mid file - describing what was sung: the lyrics, their phonemes, note pitches, durations, and a frame-level f0 contour. That string is the fuel for everything else in ComfyUI-MIDI-Edit.

What it actually runs

This isn't one model, it's five, chained together by SoulX-Singer's preprocessing pipeline: vocal separation (a mel-band-roformer karaoke model) to isolate the singer, a dereverb pass, RMVPE for f0 extraction, VAD to find where the voice actually is, ASR for the lyrics (Paraformer for Mandarin/Cantonese, Parakeet for English), and finally ROSVOT for note transcription. The result is merged into track objects that look like this:

[
  {
    "text": "<SP> 我 有 一 只 小 <SP> 毛 驴 <SP>",
    "phoneme": "<SP> zh_wo3 zh_you3 zh_yi1 zh_zhi1 zh_xiao3 <SP> zh_mao2 zh_lu:2 <SP>",
    "duration": "0.27 0.36 0.48 0.36 0.24 0.98 0.24 0.36",
    "note_pitch": "0 60 63 65 67 67 0 60",
    "note_type": "1 2 2 2 2 1 2 2",
    "f0": "0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0"
  }
]

<SP> is a pause marker, and every token in text has a matching phoneme, pitch, duration and f0. Keep this format in mind - all the edit nodes in the pack speak it.

Inputs and output

There are only three inputs, and you'll set two of them:

• audio - a normal ComfyUI AUDIO input. Load a file and wire it in.
• language - Mandarin / English / Cantonese, default Mandarin. Pick what the song is actually sung in; it selects the ASR engine.
• max_merge_duration - maximum segment-merge length in milliseconds (1000–120000, default 30000). Only touch this if you're getting phrases chopped into odd pieces.

The single output, midi_json, is a STRING. It's designed to feed straight into MIDI Edit Lyrics or MIDI Lyrics Alignment, and from there into MIDI Synthesize Audio for the actual singing.

Where people get burned

Models, mostly. This node is useless until the SoulX-Singer-Preprocess bundle (~7 GB) sits in ComfyUI/models/Soul-AILab/. The main SoulX-Singer model (~2.6 GB) is only needed later, at synthesis, so transcribe-only users can skip it.

The submodule. The pack vendors SoulX-Singer as a git submodule, and if yours didn't come through, the import fails with confusing errors. Don't clone without --recursive, and if you already did, fix it:

cd ComfyUI/custom_nodes/ComfyUI-MIDI-Edit
git submodule update --init --recursive
ls SoulX-Singer/soulxsinger/   # should NOT be empty

Install. Either grab it from ComfyUI Manager (search "ComfyUI MIDI Edit") or clone it and pip install -r requirements.txt. That requirements file is heavy - funasr, nemo_toolkit, librosa, pyworld, parselmouth, torch, torchaudio - and torch/torchaudio need to match your CUDA version, so don't be surprised if pip complains and you end up on the PyTorch install page. If the environment gets mangled, the standard fix is deleting the venv and reinstalling.

It's slow. A full transcribe runs five models over your whole file. A three-minute song takes a while, especially on the first run while models warm up. That's expected, not a bug.

One tip that saves real time: run this node once on your reference/prompt voice and route its midi_json into MIDI Synthesize Audio's prompt_metadata. Otherwise the synthesizer re-runs this whole pipeline internally on every single render.