# Neural Vocoder Architectures Published: August 11, 2025 · Author: Govind Kumar A neural vocoder is the last stage of an AI voice. It turns a mel spectrogram, the model's plan for the speech, into the audio waveform you hear. Neural vocoders play a central role in voice realism because they rebuild the fine detail the plan leaves out: breath, crisp consonants and a smooth, buzz-free tone. *Last updated: October 6, 2026. Every model claim below was checked against the original paper on arXiv on that date.* This guide is for creators and technical readers. It explains what the vocoder does, how the main designs differ, and how to spot vocoder problems. Each architecture links to its original paper. > **Key Takeaways** > > - **Role:** the acoustic model decides what is said and how. The vocoder decides how real it sounds at the level of the waveform. > - **Realism comes from detail:** phase, high-frequency texture and the repeating patterns of voiced sounds. Older signal-processing vocoders lost this detail, which caused a buzzy, muffled sound. > - **6 main families:** autoregressive (WaveNet, WaveRNN), distilled (Parallel WaveNet), flow-based (WaveGlow), GAN-based (MelGAN, HiFi-GAN, BigVGAN), diffusion (WaveGrad, DiffWave) and Fourier-based (Vocos). > - **Speed went from slow to far faster than real time.** HiFi-GAN generated 22.05 kHz audio 167.9 times faster than real time on one GPU ([arXiv](https://arxiv.org/abs/2010.05646), retrieved 2026-10-06). > - **For creators:** a metallic or buzzy voice usually points to the vocoder; a flat or oddly stressed voice points to the acoustic model. **On this page:** [What a neural vocoder is](#what-a-neural-vocoder-is) · [Role in AI voice synthesis](#vocoders-in-voice-synthesis) · [Role in voice realism](#vocoders-and-voice-realism) · [Architectures compared](#architectures-compared) · [Which to use](#which-vocoder-fits-which-job) · [Spot vocoder problems](#spotting-vocoder-problems) · [What's new](#what-is-new-in-vocoders) · [FAQ](#frequently-asked-questions) ## What a neural vocoder is A neural vocoder is a neural network that generates an audio waveform from acoustic features, usually a mel spectrogram. It's the "voice box" at the end of a text-to-speech system. The word vocoder comes from "voice coder." Classic vocoders were built for telephones and radio. An encoder side squeezed speech into a few parameters, and a decoder side rebuilt audio from them. That is why people sometimes search for a "vocoder encoder." The neural version only does the decoder job. It never sees your text. It receives a spectrogram and learns, from hours of real speech, how to fill in a natural-sounding waveform. Musicians also use "vocoder" for the robotic singing effect. Audio processed that way is called "vocoded." That effect shares the name and the basic encoder-decoder idea, but it's a sound effect, not a speech generator. ## Vocoders in voice synthesis Vocoders turn the acoustic model's plan into actual sound. Most AI voices are made in 2 steps: an acoustic model predicts a mel spectrogram from the text. A vocoder then converts it into thousands of audio samples per second.
Where the neural vocoder sits in text to speech Three stages from left to right. Text such as "Hello there" goes into the acoustic model. The acoustic model outputs a mel spectrogram, drawn as a grid of bars showing frequency energy over time with no phase information. The vocoder takes the spectrogram and outputs an audio waveform, drawn as a wavy line. A note under the vocoder says it adds phase, breath and fine detail. Acoustic model text to plan Mel spectrogram Neural vocoder plan to sound Waveform Adds what the spectrogram leaves out: phase, breath, fine detail
The spectrogram says which frequencies are loud at each moment, but not their exact timing within each wave. The vocoder has to supply that missing detail, which is why it shapes how real the voice sounds.
A useful picture: the mel spectrogram is sheet music, and the vocoder is the performer. The same sheet music can sound rich or thin depending on who plays it. The acoustic model owns pronunciation, timing and stress. Our guide to the [neural network architectures behind AI voices](https://kveeky.com/blog/neural-network-voice-synthesis) covers that side. For the full pipeline from text to audio, see [how text-to-speech AI works](https://kveeky.com/blog/how-does-text-to-speech-work). ## Vocoders and voice realism Neural vocoders decide how real a voice sounds at the level of the waveform. They rebuild the details a spectrogram drops, such as phase, breathiness, sharp consonants and the steady repeating cycles of vowels. When that detail is wrong, even perfect words sound buzzy, metallic or muffled. Here is what a good vocoder adds: - **Phase.** A mel spectrogram stores how loud each frequency is, not where each wave starts. Poor phase estimates are a common source of the "phasey," hollow sound in older systems. - **Periodic structure.** Voiced speech is built from repeating cycles. The HiFi-GAN authors showed that modeling these periodic patterns is key to sample quality ([Kong et al., arXiv](https://arxiv.org/abs/2010.05646), retrieved 2026-10-06). - **High-frequency texture.** Sounds like "s," "f" and breath noise live in the upper frequencies. Poor vocoders smear them, which makes speech sound dull. - **Clean transitions.** Natural speech glides between sounds. Artifacts at those joins are what listeners hear as robotic. The research record shows how much the vocoder matters. Tacotron 2 paired its acoustic model with a modified WaveNet vocoder. It scored 4.53 out of 5, against 4.58 for professionally recorded speech ([Shen et al., arXiv](https://arxiv.org/abs/1712.05884), retrieved 2026-10-06). Realism has 2 layers, though. The vocoder controls sound quality, while the acoustic model controls prosody: rhythm, stress and pitch. A clean vocoder can't fix a sentence with the wrong word stressed. ## Architectures compared These models differ mainly in how they generate samples: one at a time, all at once, or step by step from noise. That choice sets the trade-off between speed and quality.
FamilyExamples (year)How it generates audioSpeedMain trade-off
AutoregressiveWaveNet (2016), WaveRNN (2018)One sample at a time, each based on the ones beforeSlowHigh quality, hard to run in real time
DistilledParallel WaveNet (2017)A fast student network copies a slow WaveNet teacherFastComplex 2-stage training
Flow-basedWaveGlow (2018)Reversible steps turn noise into audio in parallelFast on GPULarge models
GAN-basedMelGAN (2019), Parallel WaveGAN (2019), HiFi-GAN (2020), BigVGAN (2022)A generator learns to fool a discriminator that spots fake audioVery fastTraining can be unstable
DiffusionWaveGrad (2020), DiffWave (2020)Starts from noise and removes it over several stepsMedium, set by step countMore steps, better audio, slower output
Fourier-basedVocos (2023)Predicts frequency coefficients, then converts them to soundVery fastNewer, less widely tested
### WaveNet vocoder architecture WaveNet models raw audio directly, predicting each sample from all previous samples ([van den Oord et al., arXiv](https://arxiv.org/abs/1609.03499), retrieved 2026-10-06). It uses **dilated causal convolutions**, which skip input values at growing steps so the model can "hear" further back in time. The paper compresses each sample with mu-law companding to 256 possible values and predicts the next one with a softmax. That gave excellent quality but slow generation, because every sample waits for the one before it. WaveRNN kept the one-sample-at-a-time idea in a single recurrent layer. It generated 24 kHz audio 4 times faster than real time on a GPU. A sparse version ran in real time on a mobile CPU ([Kalchbrenner et al., arXiv](https://arxiv.org/abs/1802.08435)). ### Parallel WaveNet: distillation Parallel WaveNet trained a fast feed-forward network to copy a trained WaveNet, a method its authors called Probability Density Distillation. It ran more than 20 times faster than real time and served Google Assistant voices ([arXiv 1711.10433](https://arxiv.org/abs/1711.10433)). ### WaveGlow: flow-based WaveGlow combines ideas from Glow and WaveNet into one flow-based network with no autoregression. Its authors reported more than 500 kHz generation on an NVIDIA V100 GPU, with quality as good as the best public WaveNet ([Prenger et al., arXiv](https://arxiv.org/abs/1811.00002)). ### GAN vocoders: MelGAN, Parallel WaveGAN, HiFi-GAN and BigVGAN GAN vocoders train 2 networks against each other. The generator makes audio from a spectrogram, and discriminators try to tell it from real speech. - **MelGAN** showed GANs could produce coherent waveforms reliably. It is fully convolutional and ran more than 100x faster than real time on a GTX 1080Ti GPU ([arXiv 1910.06711](https://arxiv.org/abs/1910.06711)). - **Parallel WaveGAN** added a multi-resolution spectrogram loss. With 1.44 million parameters it ran 28.68 times faster than real time and scored 4.16 MOS in a Transformer TTS setup ([arXiv 1910.11480](https://arxiv.org/abs/1910.11480)). - **Multi-band MelGAN** generates several frequency bands separately. It reached a real-time factor of 0.03 on CPU with 1.91 million parameters ([arXiv 2005.05106](https://arxiv.org/abs/2005.05106)). - **HiFi-GAN** focused on periodic patterns. Its quality was rated similar to human recordings in a single-speaker test, and a small version ran 13.4 times faster than real time on CPU ([arXiv 2010.05646](https://arxiv.org/abs/2010.05646)). - **BigVGAN** scaled the GAN approach to 112 million parameters with periodic activations and anti-aliasing. It aims to work across unseen speakers and recording conditions ([arXiv 2206.04658](https://arxiv.org/abs/2206.04658)). ### Diffusion vocoders: WaveGrad and DiffWave Diffusion vocoders start from random noise and clean it step by step, guided by the spectrogram. WaveGrad lets you trade speed for quality by changing the number of steps, and produced high-fidelity audio in as few as 6 ([arXiv 2009.00713](https://arxiv.org/abs/2009.00713)). DiffWave matched a strong WaveNet vocoder on speech quality, MOS 4.44 against 4.43, while generating audio far faster ([arXiv 2009.09761](https://arxiv.org/abs/2009.09761)). ### Vocos: Fourier-based Vocos predicts Fourier spectral coefficients instead of raw samples, then converts them to audio with fast standard math. Its author reports state-of-the-art quality and an order of magnitude more speed than common time-domain vocoders ([arXiv 2306.00814](https://arxiv.org/abs/2306.00814)).
Try it: what does "times faster than real time" mean?

Pick a speed reported in the papers above, enter a length of audio, and see the generation time. Each paper used its own hardware, so compare the idea, not the winners.

10 min of audio takes about 3.6 seconds to generate at that speed.

Without JavaScript, the worked example is: 10 minutes is 600 seconds, and 600 divided by 167.9 is about 3.6 seconds. A real-time factor of 0.03 means 600 x 0.03 = 18 seconds.

## Which vocoder fits which job The right vocoder depends on where the audio is generated and what it has to handle. This summary is based on what each paper reports; speeds come from different hardware and are not directly comparable.
If you need…Look atWhy
Fast output on a CPUMulti-band MelGAN, small HiFi-GANSmall models with faster-than-real-time CPU results
Top quality for one voiceHiFi-GAN, DiffWaveQuality reported close to recordings or to WaveNet
Many speakers and recording conditionsBigVGANTrained at scale for unseen speakers and conditions
Adjustable speed vs qualityWaveGradNumber of refinement steps sets the trade-off
Studying the original approachWaveNetThe reference design most later vocoders compare against
If you want to try these yourself, our overview of [open-source TTS toolkits](https://kveeky.com/blog/open-source-toolkit-for-text-to-speech-synthesis) lists projects that ship pretrained vocoders. ## Spotting vocoder problems You don't need to know which vocoder a tool uses to diagnose a bad line. Listen for the type of problem, then fix the stage that causes it.
What you hearLikely stageWhat to try
Buzzing, metallic or hollow toneVocoderTry another voice; compare the downloaded file to rule out compression
Hiss or crackle on "s" and breath soundsVocoderTry another voice; check the downloaded audio on headphones
Flat delivery or wrong word stressedAcoustic modelAdd punctuation, split the sentence, change emotion or speed
Wrong pronunciationText front endRespell the word the way it sounds
Vocoder check: 4 listening passes

Tick each one as you go, with headphones on and one paragraph of your own script.

In [Kveeky](https://kveeky.com/), you can switch between 700+ AI voices, adjust speed and volume, choose from 6 emotions, and add custom punctuation hints. Then download the audio to edit or share. For delivery problems, our checklist on how to [make an AI voiceover sound less robotic](https://kveeky.com/blog/how-to-make-your-ai-voiceover-sound-less-robotic-in-5-minutes) fixes the most common issues.
The Kveeky text-to-speech editor with a script in the text box and the Settings panel showing Voice, Transcript Language, Speed, Volume, Emotion and Custom Punctuation.
The Kveeky text-to-speech editor. To test the checklist above, paste a paragraph, pick a Voice, then change Speed or Emotion in the Settings tab and generate again. If the buzz stays when only the delivery changes, switch the Voice.
To measure quality more formally, our guide to [synthetic speech intelligibility metrics](https://kveeky.com/blog/synthetic-speech-intelligibility-metrics) explains MOS listening tests and objective scores. ## What is new in vocoders Recent work pushes in 3 directions: 1. **Universal vocoders** that handle any speaker, language or recording condition, such as BigVGAN. 2. **Faster designs** that work in the frequency domain, such as Vocos, instead of building the waveform sample by sample. 3. **Neural audio codecs** that compress audio into discrete codes and decode it back. EnCodec is one example, a real-time streaming encoder-decoder ([Défossez et al., arXiv](https://arxiv.org/abs/2210.13438)). Codecs matter for text to speech because newer models generate codec tokens instead of spectrograms. VALL-E, for instance, uses codes from an off-the-shelf neural audio codec ([arXiv 2301.02111](https://arxiv.org/abs/2301.02111)). In those systems, the codec's decoder does the job a vocoder used to do. ## Frequently asked questions ### Is a vocoder the same as an encoder? Not exactly. A classic vocoder, short for "voice coder," has both an encoder that compresses speech and a decoder that rebuilds it. The vocoder in a TTS system only does the decoding: spectrogram in, waveform out. ### What does vocoded mean? Vocoded audio has been passed through a vocoder. In music it usually means the robotic, synth-like voice effect. In speech technology it means audio that was rebuilt from compact features, such as a spectrogram. ### What is the difference between a neural vocoder and a traditional vocoder? A traditional vocoder rebuilds audio with fixed signal-processing rules, which often sounds buzzy or muffled. A neural vocoder learns the mapping from real speech, so it restores phase and fine detail much more naturally. ### Do all AI voice generators use a separate neural vocoder? No. Two-stage systems, such as Tacotron 2 with a WaveNet vocoder, do. End-to-end models such as VITS generate the waveform inside one model, and codec-based models such as VALL-E rely on a neural audio codec's decoder. ### Which vocoder is the fastest? Speeds come from different hardware, so papers aren't directly comparable. HiFi-GAN reported 167.9 times faster than real time on a V100 GPU, and Multi-band MelGAN reported a real-time factor of 0.03 on CPU. ### Is the vocoder why AI voices sometimes sound robotic? Sometimes. A buzzy or metallic tone usually comes from the vocoder. A flat, monotone delivery or odd stress usually comes from the acoustic model, which plans rhythm and pitch before the vocoder makes any sound. ## How we checked this guide This guide is written by Govind Kumar for the [Kveeky team](https://kveeky.com/). **Disclosure: Kveeky makes an AI voice generator.** We describe published research and don't claim any of it is the vocoder behind a specific product, including ours. - Model claims come from the original papers on arXiv, all retrieved October 6, 2026. - Papers: [WaveNet](https://arxiv.org/abs/1609.03499), [WaveRNN](https://arxiv.org/abs/1802.08435), [Parallel WaveNet](https://arxiv.org/abs/1711.10433), [Tacotron 2](https://arxiv.org/abs/1712.05884), [WaveGlow](https://arxiv.org/abs/1811.00002), [MelGAN](https://arxiv.org/abs/1910.06711), [Parallel WaveGAN](https://arxiv.org/abs/1910.11480), [Multi-band MelGAN](https://arxiv.org/abs/2005.05106), [HiFi-GAN](https://arxiv.org/abs/2010.05646), [WaveGrad](https://arxiv.org/abs/2009.00713), [DiffWave](https://arxiv.org/abs/2009.09761), [BigVGAN](https://arxiv.org/abs/2206.04658), [EnCodec](https://arxiv.org/abs/2210.13438), [Vocos](https://arxiv.org/abs/2306.00814) and [VALL-E](https://arxiv.org/abs/2301.02111). - MOS and speed figures are each paper's own tests, on its own data and hardware. - Kveeky features and the 700+ voice count come from [kveeky.com/pricing](https://kveeky.com/pricing/), retrieved October 6, 2026. No Kveeky usage data is used in this guide. Want to hear the difference a clean waveform makes? Generate one paragraph, download it, and listen on headphones for buzz on vowels and hiss on "s" sounds. Long listening exposes vocoder artifacts fastest, which is why it matters most for [AI voiceovers for audiobooks](https://kveeky.com/voiceover/audiobooks/).