Replicate zsxkib/realistic-voice-cloning

Feed it a song and it re-sings the vocals in a different voice - the "AI cover" thing. This is zsxkib's wrapper around RVC (Retrieval-based Voice Conversion), packaged as a Replicate node. It ships with a set of preset voice models and lets you point at your own via URL. Drop in a track, pick a voice, and it separates the vocals, converts the timbre, and remixes them back over the instrumental.

It's a Replicate node, so the heavy lifting is remote: your audio goes up to Replicate, the model runs on their GPU, the finished track comes back. You need a Replicate account and a REPLICATE_API_TOKEN, you pay per run, and nothing installs locally beyond the wrapper.

How it works

RVC doesn't generate singing from scratch - it converts existing vocals. The pipeline splits your input into vocals and instrumental, retrains the vocal's timbre toward the target voice model while keeping the melody and timing, then mixes it back with the backing track (with optional reverb and level tweaks). That's why the knobs are things like pitch detection, index rate, and "protect" rather than a text prompt.

The inputs and outputs that matter

• song_input (AUDIO) - the track to convert.
• rvc_model - the target voice. Presets include cartoon and public-figure voices, plus CUSTOM to load your own.
• custom_rvc_model_download_url - a URL to a custom RVC model, used when rvc_model is CUSTOM.
• pitch_change (no-change) - shift the octave, or use male-to-female / female-to-male when the source and target voices sit in different ranges.
• index_rate (0.5) - how strongly the target voice's character is imposed. Higher = more of the model's timbre.
• protect (0.33) - preserves consonants and breath sounds so they don't get smeared; lower it for a stronger conversion, raise it to keep clarity.

Output is a single AUDIO track (mp3 or wav via output_format). There's a full mixing panel too - rms_mix_rate, main_vocals_volume_change, instrumental_volume_change, reverb_size/reverb_wetness, and pitch_detection_algorithm (rmvpe). force_rerun bypasses Replicate's cache.

How to install it

One install covers every Replicate node.

• ComfyUI Manager: search ComfyUI-Replicate, install, restart.
• Manual:

  cd ComfyUI/custom_nodes
  git clone https://github.com/replicate/comfyui-replicate
  cd comfyui-replicate
  pip install -r requirements.txt
  

  then restart.

No weights download - just the replicate pip package. Set your token first: export REPLICATE_API_TOKEN="r8_..." (replicate.com/account/api-tokens).

Common issues & troubleshooting

The AUDIO output won't preview or save. The pack's README says it was built for "simple text or image outputs" and lists broader audio support as a roadmap item. This node returns real audio, but audio is the pack's least-mature area - update ComfyUI for its native AUDIO support and make sure a save-audio node is wired downstream.

The pitch sounds wrong or strained. The target voice's natural range probably differs from the source. Use pitch_change (male-to-female / female-to-male, or an octave shift) so the conversion isn't fighting the range.

The vocal is muddy, or consonants/breaths sound smeared. Adjust protect up to keep breathy detail, and index_rate to balance how much of the target voice is imposed vs how much of the original performance survives. Clean, well-separated source vocals convert far better than a dense mix.

Custom voice isn't loading. Set rvc_model to CUSTOM and provide a valid custom_rvc_model_download_url - the preset dropdown and the custom URL are an either/or. Same track on a re-run means the Replicate cache fired; set force_rerun.