Separate music from vocals: extract beat and vocals separately

How the vocal separator works

You upload a song, a GPU server runs the vocal separator over the whole track, and you get two MP3 files back: the beat (everything except the vocal) and the vocal (only the voice). The vocal separator looks at the song band by band and moment by moment, and decides which energy belongs to the human voice and which to the instruments.

One detail people rarely notice: the separated vocal keeps true silence. On the songs we measured, 23 to 39% of the separated vocal's duration was completely silent, exactly where the singer rests. So when you remix the separated vocal, you do not drag a layer of background noise through the whole song.

This is also the separator MuziAI uses for the first stage of AI cover, so the beat quality is the same in both tools: running the same file through both paths gives identical beats.

How long it takes

Separation takes about 60 to 75 seconds of processing per song. Including upload and download, the page estimates 1 to 2 minutes. Longer songs take longer, as these measurements show:

Each person can separate one song at a time. When the GPU is busy with other work, the job waits until there is enough memory before it starts, and if it fails for lack of memory it retries on its own, so there is sometimes a queue. Songs made on MuziAI can be separated straight from the song page, with no upload.

Separation quality: the measured numbers

To know how well a separator works you need a case with a known answer. We mixed a clean vocal into the real beat of a song, then had the separator pull it back out. The separated vocal matched the original clean vocal at 19.9 dB SDR. Put simply, the error is about one hundred times smaller than the real vocal in energy.

The other direction was measured too: when the beat of a test song was separated again, the original vocal left in the beat was only at −70 LUFS, which is practically inaudible.

We tried replacing it with a karaoke-specialised separator and with a separator combined with de-reverb. Both produced a dirtier vocal (spectral noise of 0.0085 versus 0.0070 for the current one). So the current separator is still the best option of everything we measured.

Limitations: when separation falls short

• Dense harmonies and choirs. With many voices stacked, the line between the lead vocal and the backing blurs; the beat may keep echoes of the harmonies.
• Electronic music in the same range as the voice. Synths or lead sounds sitting right in the vocal range are easily assigned to the wrong side, so you may hear faint bleed.
• Files with a cut-off bandwidth. Many files that were compressed several times were already cut off around 16 kHz before they reached you. The separator cannot add what the source does not have: a test file gave a beat reaching 18.0 kHz, exactly matching its source; a heavily compressed file gave a beat cut off at 16.0 kHz. If the beat sounds muffled, the source file is the reason.
• Re-recorded from speakers or heavily compressed. Room sound and compression artefacts stick to both the vocal and the music and cannot be separated out.
• No separation is perfect. Voice and music overlap in the same frequency range, so in places the beat may keep a trace of the vocal, or the vocal a trace of the music.

Tip: use a high-quality original file (MP3 at 320 kbps, M4A, WAV or FLAC). Acoustic recordings with guitar, piano and a single voice usually separate the cleanest.

What to do with the result

• Auto mix. Put the beat and a vocal recording into the auto mix page: it filters, applies EQ, compresses, adds reverb, balances vocal against music and masters to −14 LUFS. The vocal balance reference comes from 144 songs with real separate tracks. Everything runs in your browser; a 4-minute song takes about 12 seconds and your files never leave your device.
• Music mixer. The «Open in mixer» button preloads the beat and vocal you just separated. You set volume, pan, mute and solo per track and export a WAV, also right in the browser.
• Sing it again. Use the beat for karaoke, or take the song to AI cover to have it sung in your own voice over the original beat.
• Cleaning a vocal track with music bleed. MuziAI uses the separator this way itself: a vocal track with guitar and drum bleed had a beat-locking score of 0.736, and after the separator it dropped to 0.257, on par with a real human singer (0.254).
• Vocal practice and learning arrangement. Listen to the vocal alone to catch breathing and phrasing; listen to the beat alone to hear how bass, drums and harmony are built.

Privacy

The file you upload is deleted as soon as separation finishes. The two separated files delete themselves after 24 hours; they live at a private link, no public page points to them, and the files are tagged so search engines do not index them. You are responsible for having the rights to the recording you upload.

Frequently asked questions

Can I use the separated beat for karaoke right away?

Yes. The beat is an MP3 the same length as the original, so you can download it and sing along. Pop, ballad and acoustic songs are usually clean; songs with lots of harmonies may keep faint echoes in the chorus.

Why does my beat sound muffled?

Almost always because of the source file. Files compressed several times are often already cut off around 16 kHz, and the separated beat is cut off at the same point. Separating again from a high-quality original makes a clear difference.

Can it separate drums, bass or piano on their own?

Not yet. The tool returns two parts: the beat and the vocal. To adjust parts inside the beat, you can put it into the mixer together with other tracks.

How long are the separated files kept?

24 hours, then they delete themselves. Download them if you want to keep them. To separate again, just upload the song once more; each run takes about one to two minutes. More guides are on the blog.

Related research

• Mixing vocals in the browser: where the targets come from
• Why AI covers sound raspy, and how we measured it