Free AI cover, sung in your own voice

Three stages: separate, change the timbre, remix

An AI cover on MuziAI goes through three stages in a row. Stage one splits the song into a beat and a vocal with the vocal separator, the same one the vocal remover uses. Stage two sends only the vocal through the voice converter: pitch, timing and phrasing stay the same, and only the timbre is replaced with the timbre in your sample. Stage three mixes the new vocal back onto the original beat and balances the loudness of the whole track.

Why not have the machine sing the whole song from scratch? Because the beat is what listeners recognise first. When we ran the same file through the vocal remover and through the cover pipeline, the two beats came out identical and matched the bandwidth of the source file, so the music is not degraded any further. The performance also belongs to the original: when it gets louder or softer and where it slides between notes follow the original singer. In our first evaluation, the new vocal's melody differed from the original by a median of 0.00 semitones, with 92.6% of frames within one semitone.

After the voice change there are a few post-processing steps, and each one only removes something the original does not have:

• A gate driven by the original vocal: the voice converter never outputs true silence, its noise floor sits around −48 dB. Wherever the original singer is silent, the new vocal is muted too, bringing the floor down to −93.6 dB.
• Envelope matching: loudness follows the original singer, which reduces the small dropouts between words.
• De-whistling in three bands, 2-4, 4-8 and 8-16 kHz, when a frame jumps far above its own usual level.
• Stereo reverb: the converted vocal is mono, so reverb is added and tuned until its width matches the original vocal.
• Mastering against the original song: the tonal balance follows the track you uploaded (deviation down to 0.24 dB from 0.47 dB), at a standard −14 LUFS.

What makes a good voice sample

Your sample decides two separate things: whether the cover sounds like you, and whether it sounds clean. Each depends on a different factor.

Length: 24 seconds beats 12 seconds

The system uses about 24 seconds of singing after removing breaths and pauses. We originally used only 12 seconds, assuming a longer sample would not sound more alike. When we measured it, five runs per branch, every 24-second window sounded more like the singer than every 12-second window, at every position in the recording. A longer sample is more robust because it averages the good passages with the weaker ones. So record 20 to 40 seconds: the system needs some extra to choose from.

Position matters as much as length

The same 12 seconds taken from the start of a recording or from the middle give very different results: spectral noise in the output dropped almost threefold when the right section was used. So the system does not just cut the first part. It builds 24 seconds from the least rough one-second pieces of your sample, after trimming silence with a threshold that adapts to the recording's background noise. On a 37.2-second phone recording, this threshold removed 5.6 seconds of pauses, where a fixed threshold removed none.

A rough sample gives a rough cover

The voice converter copies the vocal quality of the sample. Same song, only the sample changed: a clean sung sample gave an output with 0.92% roughness, a rough phone recording gave 1.67%, which is 81% worse. A longer sample fixes whether it sounds like you, but not the gravelly texture. Only a cleaner sample fixes that.

That is why the system measures your sample's roughness at the very start of the job, about 11 seconds in, and shows a warning at 1.5% or above. That threshold sits between the clean singing voices we measured (0.94 to 1.03%) and the first rough sample we encountered (2.14%). The page also warns when the sample has under 15 seconds of singing, or almost no clear singing at all. When you re-record: sing, don't speak, hold your voice steady to the end of each phrase, alone, no backing music, a quiet room, phone about a hand's width from your mouth.

How long it takes

These are processing times, not counting the queue. The first run takes longer because the system learns your voice during that run; later runs reuse that model. You can close the tab, and the result page updates itself step by step.

Changing the language and choosing the AI voice gender

When you have a song sung in another language, the system transcribes the original lyrics, translates them keeping the same number of lines and syllables, has the singing model sing the new lyrics to the old melody, then places the result on the original beat. A 189-second song took 580 seconds. Eight target languages are available; Japanese is not open yet because the sung lyrics are still hard to understand.

You choose Same as original singer, Female voice or Male voice. The gender choice only changes the timbre; the singing model still sings at the original song's pitch. If the output lands far outside the chosen range (a male voice above 320 Hz, a female voice below 244 Hz), the whole vocal is shifted by exactly one octave so it stays in harmony with the beat. The cost: a male voice shifted down an octave sounds somewhat rough. For the cleanest vocal, choose the same gender as the original singer.

Limitations

• Vietnamese is weaker than English. Measured on the same voice converter, the share of frames that lose their sound was 2.9% for Vietnamese and 1.0% for English. This is a model limitation, not a problem with your file.
• Consonants can change. In our tests, «Gà gáy vang trời» became «cáo kẽ vang trời», and «hỏi» became «khỏi». The melody holds, the consonants do not always.
• Low-quality sources. Files that were compressed several times are often already cut off around 16 kHz. The output vocal is typically 2 to 3.6 times noisier than the separated vocal, so a dirty source drags the cover down with it. The result page tells you plainly when the source is below 17 kHz.
• The remaining haze is inside the harmonics. It is not a layer of noise on top but part of the harmonics the model generates, so denoising afterwards cannot fix it. We tried filtering the output three different ways, and none was better.
• Range mismatch. A high female song with a male sample: keeping the key forces your voice up into a female range and it sounds more like the original singer; dropping an octave sounds more like you but boomier. This is a physical limit, not specific to any model.
• The phrasing stays the original singer's. Slides, phrasing and breaths are kept; only the vocal colour changes.

Privacy

The song and the voice sample are deleted as soon as the job finishes. The voice model learned from your sample is kept privately for 24 hours after its last use, only so the next cover with the same sample runs faster, then it deletes itself; it is never used for anyone else. The cover is kept for 24 hours, only people with the link can hear it, and the audio files are blocked from search engine indexing.

When not to use it

• You want a cover in a singer's or another person's voice: this tool only accepts your own voice.
• You want the cover to sound like you singing the song your own way.
• You only have a low-quality file, or the song has heavy harmonies or a choir.
• You need every Vietnamese consonant pronounced correctly for a release.
• You just need the beat for karaoke: the vocal remover is much faster.
• You want a brand new song: create one with AI and come back here.

Frequently asked questions

Can I record the sample on my phone?

Yes. What to avoid is the gravelly texture: speaking instead of singing, dropping your voice at the end of phrases, a noisy room. If the page says your sample is 1.5% rough or more, one re-recording is usually worth more than any other tweak.

Why is the first run slower than the next ones?

On the first run the system learns your voice from the sample, so a 4-minute song takes about 10 minutes. That model is kept for 24 hours after its last use, so another cover with the same sample takes only about 6 minutes.

Which file should I use, and what are the limits?

Only use recordings you own or have the right to use; MuziAI does not download music from YouTube or any other platform and does not help copy copyrighted songs. On quality: a file that was compressed several times is often cut off around 16 kHz and sounds muffled, because the beat and vocal cannot be cleaner than the source. An original MP3 at 320 kbps, M4A, WAV or FLAC gives a clearly better result.

Why do some lines sound boomy or break up on low notes?

Usually the line drops below the range in your sample. The system finds such lines automatically, converts them again one octave higher and splices them back in at the nearest unvoiced spot. If the whole song still sounds boomy, run it again with Keep the original key.

Related research

• Why AI covers sound raspy, and how we measured it
• Vietnamese sung by AI: how many words survive?