4.3 Azure AI Speech Service
Key Takeaways
Azure AI Speech provides speech-to-text (STT), text-to-speech (TTS) and speech translation via the Speech SDK and REST APIs; Speaker Recognition was retired on September 30, 2025.
Speech-to-text supports real-time and batch transcription, custom speech for domain vocabulary, and speaker diarization to separate speakers.
Text-to-speech uses 600+ neural voices across 150+ languages/locales, supports SSML for fine control, and offers gated custom neural voice.
Speech translation runs recognize -> translate -> (optionally) synthesize, so it can output translated text or translated speech.
Speaker diarization separates who spoke when in a transcript but does not identify the speakers; Speaker Recognition, which verified or identified people by voice, was retired on September 30, 2025.
Quick Answer: Azure AI Speech provides three core capabilities: speech-to-text (convert audio to text), text-to-speech (convert text to audio), and speech translation (translate spoken language). All are available via REST APIs and the Speech SDK. Speaker Recognition (identifying who is speaking) was retired on September 30, 2025.
What Is Azure AI Speech?
Azure AI Speech is a cloud-based service that provides a comprehensive set of speech-related AI capabilities. It handles the audio side of NLP — converting between speech and text, translating spoken language, and separating speakers in a transcript. Where Azure AI Language works on text, Azure AI Speech works on the spoken word.
Core Capabilities
1. Speech-to-Text (STT)
What it does: Converts spoken audio into written text (transcription).
| Feature | Description |
|---|---|
| Real-time transcription | Transcribe audio as it is spoken (live captioning, voice commands) |
| Fast / batch transcription | Transcribe pre-recorded audio files (meeting recordings, podcasts) |
| Custom speech | Train on your vocabulary and acoustic environment for better accuracy |
| Multi-language | Supports 100+ languages and dialects |
| Speaker diarization | Separate speakers in a conversation ("Speaker 1:", "Speaker 2:") |
Use cases: live meeting captioning, voice-controlled apps, call-center transcription for quality analysis, accessibility captioning for deaf or hard-of-hearing users, and dictation.
2. Text-to-Speech (TTS)
What it does: Converts written text into natural-sounding speech audio using neural voices.
| Feature | Description |
|---|---|
| Neural voices | 600+ natural-sounding prebuilt voices across 150+ languages and locales |
| Custom neural voice | Create a unique voice for your brand (limited/gated access, requires responsible-AI review) |
| SSML support | Fine-tune pronunciation, pitch, rate, and emphasis using Speech Synthesis Markup Language |
| Visemes | Synchronized mouth shapes for animated avatars |
Use cases: screen readers and accessibility tools, audiobook generation, Interactive Voice Response (IVR) systems, voice-enabled chatbots, e-learning narration, and public-announcement systems.
Two voice tiers worth knowing: standard neural voices (the broad prebuilt catalog) and newer high-definition (HD) / Dragon HD voices that add greater expressiveness and emotional range. For the exam, the headline fact is simply that TTS uses neural voices that sound human, not the older robotic concatenative voices.
3. Speech Translation
What it does: Translates spoken audio from one language to another, in near real time.
| Feature | Description |
|---|---|
| Speech-to-text translation | Spoken language -> translated text |
| Speech-to-speech translation | Spoken language -> translated spoken audio |
| Multi-language | Many languages supported for speech translation |
| Real-time | Near-instant translation for live conversations |
How it works — three steps:
- Recognize the spoken input (speech-to-text)
- Translate the recognized text to the target language
- Synthesize the translated text (text-to-speech) — only for speech-to-speech translation
Use cases: real-time translation in international meetings, travel assistance (speak in your language, output in the local language), multilingual customer service, and cross-language collaboration tools.
Retired: Speaker Recognition
Speaker Recognition identified or verified a person by voice (verification was 1:1, identification 1:many). Microsoft retired it on September 30, 2025, so apps can no longer use it. Speaker diarization in speech-to-text still separates who spoke when, but it labels speakers generically rather than identifying them.
Custom Speech Models
Azure AI Speech can be customized for better accuracy in specific environments:
| Customization | What It Does | When to Use |
|---|---|---|
| Custom speech-to-text | Train on your domain vocabulary and acoustic conditions | Medical, legal, or technical terminology |
| Custom neural voice | Create a unique branded voice | Brand identity, specific character voices |
| Pronunciation assessment | Evaluate speech pronunciation accuracy | Language-learning applications |
A common exam scenario: a hospital wants accurate transcription of drug names and clinical terms. The default model struggles with rare vocabulary, so the answer is custom speech — train on the domain words, not switch services.
Azure AI Speech vs. Azure AI Language
| Aspect | Azure AI Speech | Azure AI Language |
|---|---|---|
| Input | Audio / speech | Text |
| Primary tasks | STT, TTS, speech translation | Sentiment, NER, key phrases, CLU |
| Focus | Converting between audio and text | Understanding text meaning |
The two are often chained: Speech converts a caller's audio to text -> Language analyzes intent and sentiment -> a response is generated -> Speech converts the response back to audio. Knowing this pipeline answers many "which combination of services?" questions.
Common Exam Scenarios
| Scenario | Capability |
|---|---|
| Add live captions to a webinar | Speech-to-text (real-time) |
| Read web content aloud for accessibility | Text-to-speech |
| Build an IVR phone menu with a natural voice | Text-to-speech (often with SSML) |
| Translate a spoken conversation between two people | Speech translation |
| Transcribe a podcast archive overnight | Speech-to-text (batch) |
| Improve recognition of legal jargon | Custom speech |
A subtle distinction the exam likes: speech translation (audio in, translated text or audio out) lives in Azure AI Speech, whereas Azure AI Translator only handles text. If the input is spoken, think Speech; if the input is written, think Translator.
On the Exam: Speech handles audio-to-text and text-to-audio conversions; Language analyzes text. Speech translation (audio) is different from Azure AI Translator (text). Speaker Recognition was retired on September 30, 2025; diarization separates speakers but does not identify them.
Which Azure AI Speech capability converts spoken audio into written text?
Text-to-speech
Speech-to-text
Speech translation
Speaker recognition
A company wants to create an accessibility feature that reads web page content aloud to visually impaired users. Which Azure AI Speech capability should they use?
Speech-to-text
Text-to-speech
Speech translation
Speaker recognition
A meeting app must label which parts of a recorded call came from different people (Speaker 1, Speaker 2) in the transcript. Which Azure AI Speech capability is needed?
Text-to-speech
Speech translation
Speech-to-text with diarization
Custom neural voice for each speaker
How does speech translation work in Azure AI Speech?
It directly translates audio waves from one language to another
It recognizes speech, translates the text, and optionally synthesizes the translation as speech
It uses a bilingual dictionary to replace words
It only translates written text, not spoken audio
A hospital needs Azure AI Speech to accurately transcribe rare drug names and clinical terms that the default model frequently mis-hears. What is the best approach?
Train a custom speech model on the medical vocabulary
Switch to Azure AI Translator
Use text-to-speech instead
Increase the audio sample rate only
Match each Azure AI Speech capability to its use case:
Match each item on the left with the correct item on the right
Speech-to-text
Text-to-speech
Speech translation
Custom speech
Sections you finish are checked off in the contents.