# AI Voice Generators and Deepfake Detection Explained Published: September 2, 2025 · Author: Deepak Gupta Voice deepfake detection is the work of telling real speech from AI-generated speech that imitates a real person. It combines detection models that look for synthesis traces, provenance signals such as watermarks, and human checks such as calling back on a known number. No method is reliable alone; in one study, listeners spotted deepfakes only 73% of the time. *Last updated: October 7, 2026. We rechecked every study, regulator page and challenge result cited here, and added a section on SynthID and C2PA watermarking from official sources.* This guide is for creators, businesses and families who want to understand how fake voices are made and caught. It explains how an AI voice generator differs from a voice deepfake and how detection works. It also covers where detection fails and what to do when a voice seems off. > **Key Takeaways** > > - **A deepfake voice imitates a real person to mislead.** An AI voice generator used with stock voices, or your own consented clone, is not a deepfake. > - **People are poor detectors.** In a study of 529 listeners, deepfakes were spotted 73% of the time, and showing examples helped only slightly. > - **Detection models struggle with new fakes.** The ASVspoof 2021 evaluation found detectors lack generalization across different source datasets. > - **Provenance is the second layer.** The EU AI Act requires providers to mark synthetic audio in a machine-readable, detectable way. > - **Process beats ears.** The FBI recommends a family secret word, and the FTC says to call back on a number you know. **On this page:** [What it is](#what-is-a-deepfake-voice) · [Can people tell](#spotting-an-ai-generated-voice) · [How detection works](#how-detection-works) · [Why it's hard](#why-detection-is-hard) · [Detection tools](#existing-deepfake-detectors) · [If you suspect a fake](#if-you-suspect-a-deepfake) · [Generators vs deepfakes](#ai-vs-deepfake-voice-generators) · [Watermarking](#audio-watermarking-synthid-and-c2pa) · [FAQ](#frequently-asked-questions) ## What is a deepfake voice? A deepfake voice is synthetic speech made to sound like a specific real person, used without their consent or to mislead listeners. It is usually made with voice cloning or voice conversion. The technology is the same as in legitimate tools. The difference is consent and intent:
Legitimate AI voiceDeepfake voice
Whose voiceA stock voice, or your own or a consenting person's cloneA real person who did not agree
PurposeNarration, courses, ads, accessibilityScams, fake statements, impersonation
DisclosureClear, or not misleadingHidden on purpose
ExampleA creator voicing a video with a stock AI voiceA fake call from a "relative" asking for money
Regulators see the harm. The FCC ruled in February 2024 that AI-generated voices in robocalls count as "artificial" voices under the Telephone Consumer Protection Act ([FCC](https://docs.fcc.gov/public/attachments/DOC-400393A1.pdf), retrieved 2026-10-06). ## Spotting an AI-generated voice You often can't tell by listening. Researchers tested 529 people in English and Mandarin, and listeners correctly spotted speech deepfakes only 73% of the time. Showing them examples first improved results only slightly ([Mai et al., PLOS ONE, 2023](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0285333), retrieved 2026-10-06). Some signs can still raise suspicion. Tick the ones you notice on a call or voice note:
Spot a synthetic voice: tick what you notice

Warning signs only, never proof.

No signs ticked. That still doesn't prove the voice is real: verify any request for money through another channel.

Treat these as warning signs, not proof. The FBI also suggests you "listen closely to the tone and word choice" on calls from a supposed relative ([FBI IC3](https://www.ic3.gov/PSA/2024/PSA241203), retrieved 2026-10-06). Then verify through another channel. ## How detection works Detection works in 3 layers. Strong protection uses all of them.
LayerHow it worksStrengthWeakness
Detection modelsAI classifiers trained on real and synthetic speech look for traces of synthesisFast; can scan large volumes of audioWeaker on new generators and different datasets
Provenance and watermarksThe generating tool marks its output, or a file carries tamper-evident history such as Content CredentialsStrong proof when the mark is presentOnly works if the creator's tool adds it; bad actors can use tools that don't
Procedural checksCall back on a known number, use a secret word, confirm through a second channelWorks against any fake, however goodNeeds people to follow the process under pressure
On provenance, the EU AI Act requires providers of systems that generate synthetic audio to mark outputs "in a machine-readable format and detectable as artificially generated or manipulated" ([Article 50, AI Act Service Desk](https://ai-act-service-desk.ec.europa.eu/en/ai-act/article-50), retrieved 2026-10-06). The C2PA standard describes Content Credentials as a way to show "the origin and edits of digital content" ([C2PA](https://c2pa.org/), retrieved 2026-10-06). ## Why detection is hard Detectors learn from examples of fake speech. When a new voice generator appears, its traces can differ from anything in the training data. The ASVspoof challenge series measures this problem. Its 2021 edition added a deepfake task on "manipulated, compressed speech data posted online". The organizers found that detection "lack[s] generalization across different source datasets" ([ASVspoof 2021 paper](https://arxiv.org/abs/2210.02437), retrieved 2026-10-06). The series has continued with ASVspoof 5 ([ASVspoof 5 paper](https://arxiv.org/abs/2502.08857), retrieved 2026-10-26). Other practical limits: - **Compression.** Phone lines and social apps compress audio, which can hide or mimic traces. - **Short clips.** A 5-second voice note gives a detector little to work with. - **Languages.** Detectors trained mostly on a few languages may do worse on others. - **Real-time calls.** Checking live audio fast enough is harder than checking a file. ## Existing deepfake detectors There is no single standard detector. Approaches fall into a few groups, and the FTC's Voice Cloning Challenge shows the range. The FTC challenge asked for ideas in 3 areas: preventing malicious cloning, monitoring to tell synthetic from genuine voices, and evaluating protections ([FTC](https://www.ftc.gov/news-events/contests/ftc-voice-cloning-challenge), retrieved 2026-10-06). Its winners included: - **AI Detect (OmniSpeech):** AI algorithms that tell genuine from synthetic voice patterns. - **DeFake (Washington University):** adds protective perturbations to voice samples to make cloning harder. - **OriginStory:** measures biosignals to confirm a human voice at the time of recording. - **Pindrop Security** received a recognition award for liveness detection. Commercial detection services also exist. Resemble AI's pricing page, for example, now lists "Deepfake detection for audio, image, and video" ([Resemble AI pricing](https://www.resemble.ai/pricing/), retrieved 2026-10-06). Test any detector on your own audio before you rely on it, and never treat one score as proof. ## If you suspect a deepfake Act on process, not on how real the voice sounds.
Six steps if you suspect a deepfake voice Six steps in order: 1, pause and send no money, codes or data; 2, hang up and call back on a number you know; 3, ask for the secret word agreed in advance; 4, check with someone who knows the person; 5, save the recording, number and time; 6, report it at ReportFraud.ftc.gov or to the platform. If a voice seems off: 6 steps 1 Pause Send no money, codes or data 2 Call back Hang up, then call a number you know 3 Secret word Ask for the word agreed in advance 4 Cross-check Ask someone who knows the person 5 Save evidence Keep the recording, number and time 6 Report it ReportFraud.ftc.gov or the platform Act on process, not on how real the voice sounds.
Verify through a fixed process instead of trusting your ears: every step works however real the voice sounds.
1. **Pause.** Don't send money, codes or data during the call. 2. **Hang up and call back** on a number you already know. The FTC's advice: "Don't trust the voice" ([FTC](https://consumer.ftc.gov/consumer-alerts/2023/03/scammers-use-ai-enhance-their-family-emergency-schemes), retrieved 2026-10-06). 3. **Ask for the secret word** your family or team agreed in advance. 4. **Check with someone else** who knows the person. 5. **Save the evidence.** Keep the recording, number and time. 6. **Report it.** In the US, report scams at ReportFraud.ftc.gov. Report fake videos or audio to the platform where they appeared. For businesses, add a rule that no payment or password change happens on a voice request alone.
Try it: send this to your family or team today

Steps 2 and 3 only work if you agree on them before a scam call. Copy this message, fill in the blanks and send it.

Voice scams can now copy anyone's voice, so let's agree two things. First, our secret word is ______. If I ever call or send a voice note asking for money, codes or passwords, ask me for it. Second, if anything sounds off, hang up and call me back on the number you already have saved: ______. I will never mind being checked, and I will do the same for you.
## AI vs deepfake voice generators Many people search for a "deepfake voice generator" when they really want a realistic voice for their own content. You don't need to imitate a real person for that. - **Use stock AI voices.** Kveeky has 700+ AI voices in 40+ languages, with speed, volume and emotion controls. - **Clone only your own voice**, or one you have written permission to use. Our [voice cloning guide](https://kveeky.com/blog/voice-cloning-duplicate-your-voice-in-minutes) covers how. - **Disclose AI voices** where viewers could think a real person is speaking.
The Voices page in the Kveeky app: a search box, gender and language filters, the tabs Featured, All voices, My voices and Saved, and a list of stock voices, each with a preview button and a FREE or PRO tag.
The stock-voice route in the Kveeky app. Filter by gender or language, press the play button to preview a voice, and look for the FREE or PRO tag before you pick one.
Responsible tools set rules too. Kveeky's Terms of Service prohibit impersonating any person or entity. They also prohibit using AI outputs "to create misleading content presented as human-created without disclosure". For the legal limits, read [is voice cloning legal?](https://kveeky.com/blog/is-voice-cloning-legal-ai-rights-guide), and for the ethical side, our guide to the [ethics of voice cloning in video production](https://kveeky.com/blog/ethical-implications-of-voice-cloning-for-video-production). Want legitimate AI voices to sound natural instead of fake? Our guide to [AI voice pacing, emphasis and breath](https://kveeky.com/resources/ai-voice-pacing-emphasis-breath/) shows how. ## Audio watermarking: SynthID and C2PA Watermarking marks AI audio at the moment it is made, so it can be identified later without guessing from the sound. The 2 systems you'll hear about most are Google DeepMind's SynthID, a watermark inside the audio, and C2PA Content Credentials, signed metadata attached to a file. ### What is SynthID audio watermarking? SynthID is Google DeepMind's watermark for AI-generated content. For audio, it was introduced with the Lyria music model on November 16, 2023, as a mark "inaudible to the human ear" ([Google DeepMind](https://deepmind.google/blog/transforming-the-future-of-music-creation/), retrieved 2026-10-07). DeepMind says it stays detectable after common edits such as added noise, MP3 compression and speeding up or slowing down. Because the mark lives in the sound itself, it survives when metadata is stripped. It only proves anything for audio made by a tool that adds it, so a missing watermark doesn't mean a voice is real. ### Does Google's text-to-speech watermark its voices? Yes. Google says all audio from Gemini 3.1 Flash TTS, announced April 15, 2026, is watermarked with SynthID ([Google](https://blog.google/innovation-and-ai/models-and-research/gemini-models/gemini-3-1-flash-tts/), retrieved 2026-10-07). Its September 23, 2026 post on Gemini 3.8 TTS says every clip from its Gemini audio models carries the mark ([Google](https://blog.google/innovation-and-ai/models-and-research/gemini-models/gemini-3-8-text-to-speech/), retrieved 2026-10-07). To check content, Google launched the SynthID Detector portal on May 20, 2025. It scans images, audio, video and text, and it opened first to early testers, with a waitlist for journalists and researchers ([Google](https://blog.google/innovation-and-ai/products/google-synthid-ai-content-detector/), retrieved 2026-10-07). ### Which other companies are adopting SynthID? On May 19, 2026, Google said OpenAI, Kakao and ElevenLabs are bringing SynthID to more of their AI-generated content ([Google](https://blog.google/innovation-and-ai/products/identifying-ai-generated-media-online/), retrieved 2026-10-07). The same post says Google has watermarked 60,000 years of audio and is partnering with NVIDIA on video. OpenAI's developer docs describe a provenance check that looks for SynthID in images and audio ([OpenAI](https://developers.openai.com/api/docs/guides/content-provenance), retrieved 2026-10-07). We couldn't confirm exact rollout dates for each company on their own pages, so check a vendor's documentation before you rely on its watermark. ### What is C2PA, and how does OpenAI use it? C2PA is an open technical standard for showing "the origin and edits of digital content" through Content Credentials, signed metadata that travels with a file ([C2PA](https://c2pa.org/), retrieved 2026-10-07). OpenAI joined the C2PA steering committee on May 7, 2024, and was already attaching Content Credentials to DALL-E 3 images ([C2PA](https://c2pa.org/openai-joins-c2pa-steering-committee/), retrieved 2026-10-07). In OpenAI's current developer docs, the C2PA check covers images only, while audio is checked for SynthID. Metadata can be removed when a file is re-encoded or screen-recorded, which is why watermarks and metadata are used together. ### What does watermarking mean for creators? A watermark doesn't stop you from publishing AI audio; it labels it. If your audience could be misled about who is speaking, say the voice is AI in the caption or description as well, since listeners can't hear a watermark. Under the EU AI Act's Article 50, covered above, providers must mark synthetic audio in a machine-readable way. If you're worried about scams rather than detection tools, read [is AI voice safe? A guide to voice-clone fraud and protection](https://kveeky.com/blog/is-ai-voice-safe). ## Frequently asked questions ### What is a deepfake voice? A deepfake voice is AI-generated speech made to sound like a real person without their consent, usually to mislead. It is made with voice cloning or voice conversion. A stock AI voice or your own consented clone is not a deepfake. ### Can AI detect AI-generated voices reliably? Not on its own. Detection models work well on fakes like those they were trained on, but the ASVspoof 2021 evaluation found they generalize poorly to new datasets. Combine detection with watermarks and callback checks. ### How accurate are people at spotting voice deepfakes? Not very. In a 2023 study of 529 listeners in English and Mandarin, people spotted speech deepfakes only 73% of the time, and seeing examples first helped only slightly. ### Is it illegal to make a deepfake voice? It depends on use and place. Using a cloned voice to defraud or impersonate is illegal, and the FCC treats AI voices in robocalls as artificial voices under the TCPA. This is general information, not legal advice. ### Is AI-generated audio watermarked? Some of it. Google says all audio from its Gemini text-to-speech models carries an inaudible SynthID watermark, and OpenAI, Kakao and ElevenLabs are adopting SynthID. Audio from tools that don't add a mark carries none, so a missing watermark doesn't prove a voice is real. ### How can I protect myself from voice deepfake scams? Agree a secret word with family, hang up and call back on a known number, and never send money on a voice request alone. Report scams to the FTC at ReportFraud.ftc.gov. ## How we checked this guide This guide is written by Deepak Gupta for the [Kveeky team](https://kveeky.com/). **Disclosure: Kveeky makes an AI voice generator.** This is general information, not legal or security advice. - Research: [Mai et al., PLOS ONE (2023)](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0285333) and the [ASVspoof 2021 paper](https://arxiv.org/abs/2210.02437), retrieved October 6, 2026. - Watermarking facts come from official pages, all retrieved October 7, 2026: Google DeepMind, ["Transforming the future of music creation"](https://deepmind.google/blog/transforming-the-future-of-music-creation/) (November 16, 2023); Google, ["SynthID Detector — a new portal to help identify AI-generated content"](https://blog.google/innovation-and-ai/products/google-synthid-ai-content-detector/) (May 20, 2025); Google, ["Making it easier to understand how content was created and edited"](https://blog.google/innovation-and-ai/products/identifying-ai-generated-media-online/) (May 19, 2026); Google, ["Gemini 3.1 Flash TTS: the next generation of expressive AI speech"](https://blog.google/innovation-and-ai/models-and-research/gemini-models/gemini-3-1-flash-tts/) (April 15, 2026) and ["Gemini 3.8 text-to-speech says hello"](https://blog.google/innovation-and-ai/models-and-research/gemini-models/gemini-3-8-text-to-speech/) (September 23, 2026); C2PA, ["OpenAI Joins C2PA Steering Committee"](https://c2pa.org/openai-joins-c2pa-steering-committee/) (May 7, 2024); and OpenAI, ["Content provenance"](https://developers.openai.com/api/docs/guides/content-provenance) developer guide (undated). - Regulators and standards: [FCC](https://docs.fcc.gov/public/attachments/DOC-400393A1.pdf), [FTC Voice Cloning Challenge](https://www.ftc.gov/news-events/contests/ftc-voice-cloning-challenge), [FTC consumer alert](https://consumer.ftc.gov/consumer-alerts/2023/03/scammers-use-ai-enhance-their-family-emergency-schemes), [FBI IC3](https://www.ic3.gov/PSA/2024/PSA241203), [EU AI Act Service Desk](https://ai-act-service-desk.ec.europa.eu/en/ai-act/article-50) and [C2PA](https://c2pa.org/), retrieved October 6, 2026. - Kveeky's terms were read on kveeky.com on October 6, 2026. No Kveeky usage data is used in this guide. Next step: agree a secret word with your family or team this week, and write down the callback numbers you would use. If you make AI voiceovers yourself, our [AI voiceover guide for social media videos](https://kveeky.com/voiceover/social-media-videos/) shows how to use stock voices openly and well.