5.2 Azure OpenAI Service
Key Takeaways
- Azure OpenAI Service, delivered through Azure AI Foundry, gives API access to OpenAI models (GPT-4.1 and GPT-4o families, embeddings, image generation, Whisper) with Azure enterprise features.
- Azure OpenAI offers the SAME models as OpenAI but adds RBAC, managed identities, private endpoints, compliance certifications, and built-in content filtering.
- You must create a deployment (model + version + capacity) before calling a model; deployment types include Standard, Provisioned (PTU), and Batch.
- GPT-4.1 supports up to roughly 1,000,000 tokens of context; embeddings models produce vectors for semantic search and RAG.
- Your prompts and responses are NOT used to train OpenAI models — the most commonly tested data-privacy fact on the exam.
Quick Answer: Azure OpenAI Service (delivered through Azure AI Foundry / Microsoft Foundry) provides access to OpenAI models — the GPT-4.1 and GPT-4o families, embeddings, image generation, and Whisper — through Azure with enterprise-grade security, compliance, and content filtering. You create deployments of specific models and call them via REST API or SDK.
What Is Azure OpenAI Service?
Azure OpenAI Service gives you REST API access to OpenAI's generative models from within Azure. It is now part of Azure AI Foundry (Microsoft Foundry) Models, and it combines OpenAI's models with Azure's enterprise capabilities.
Azure OpenAI vs. OpenAI Direct
| Feature | Azure OpenAI | OpenAI Direct |
|---|---|---|
| Models | Same OpenAI models (GPT-4.1, GPT-4o, etc.) | Same models |
| Security | Azure RBAC, managed identities, private endpoints | API key authentication |
| Compliance | SOC 2, HIPAA, ISO 27001, GDPR certifications | Standard compliance |
| Content filtering | Built-in Azure AI Content Safety filters (on by default) | Moderation API |
| Data privacy | Your data is NOT used to train OpenAI models | Your data is NOT used to train OpenAI models |
| Networking | Virtual networks, private endpoints | Public internet |
| SLA | Enterprise SLA with guaranteed uptime | Best-effort availability |
| Billing | Azure subscription billing | OpenAI billing |
On the Exam: The key differentiator is ENTERPRISE FEATURES. Azure OpenAI offers the SAME models as OpenAI but adds Azure security, compliance, content filtering, and networking that enterprises require.
Available Models
Text Generation Models
| Model | Description | Best For |
|---|---|---|
| GPT-4.1 | Latest GPT-4 series; up to ~1,000,000-token context | Coding, instruction-following, long-context analysis |
| GPT-4.1 mini / nano | Smaller, faster, cheaper GPT-4.1 variants | Cost-sensitive and high-volume workloads |
| GPT-4o | Multimodal (text + image input, text output) | Complex reasoning, coding, multimodal analysis |
| GPT-4o mini | Smaller, faster GPT-4o | Quick, low-cost responses |
(Older models such as GPT-3.5-Turbo are being phased out; always create deployments from the current model catalog.)
Other Model Types
| Type | Example Model | Purpose |
|---|---|---|
| Image generation | GPT-image-1 | Generate images from text prompts |
| Embeddings | text-embedding-3-large / -small | Vector representations for semantic search and RAG |
| Speech | Whisper | Speech-to-text transcription |
Creating a Deployment
Before using a model you must create a deployment in your Azure OpenAI resource:
- Create an Azure OpenAI resource in the Azure portal
- Open Azure AI Foundry (the management portal)
- Select a model from the model catalog
- Create a deployment — choose the model version, capacity (TPM = tokens per minute), and a deployment name
- Use the deployment via the REST API or SDK with your endpoint and key (or managed identity)
Deployment Types
| Type | Description | When to Use |
|---|---|---|
| Standard | Pay-per-token, shared capacity | Development, testing, variable workloads |
| Provisioned (PTU) | Reserved capacity with guaranteed throughput | Production workloads with predictable demand |
| Batch | Asynchronous, lower-cost processing of large jobs | Bulk, non-interactive workloads |
A "global" routing option also exists for high availability across regions.
Enterprise Features
Security
- Azure RBAC — control who can access the resource and deployments
- Managed identities — authenticate without storing keys in code
- Private endpoints — keep traffic on your private network
- Customer-managed keys — encrypt data with your own keys
Compliance
Azure OpenAI inherits Azure's certifications: SOC 2 Type II, ISO 27001, HIPAA (with a BAA), GDPR, and FedRAMP in select regions.
Content Filtering
Built-in filters scan both prompts and responses for hate, violence, sexual, and self-harm content, plus Prompt Shields for jailbreak/injection detection. Filters are enabled by default.
Data Privacy
- Your prompts and responses are NOT used to train OpenAI models.
- Data is processed in the Azure region you select and is not shared with OpenAI for model improvement.
- You can opt in or out of abuse monitoring.
Common Workloads
Azure OpenAI is the engine behind most enterprise generative-AI applications on Azure. Typical workloads include:
| Workload | How Azure OpenAI is used |
|---|---|
| Enterprise chatbot / copilot | GPT chat completions, usually grounded with RAG via Azure AI Search |
| Summarization | Condense long documents, transcripts, or email threads |
| Content generation | Draft marketing copy, product descriptions, code |
| Semantic search | Embeddings models power vector search over your content |
| Classification and extraction | Use prompts to categorize tickets or pull structured fields |
| Image generation | Create images from text prompts with GPT-image-1 |
A recurring exam idea is that Azure OpenAI is rarely used alone — it is paired with Azure AI Search (for grounding/RAG), Azure AI Content Safety (for filtering), and Azure AI Foundry (for building and evaluating the application).
Access, Regions, and Responsible Use
Azure OpenAI is a Limited Access service: historically you applied for access, and Microsoft requires customers to follow a Code of Conduct and responsible-use commitments. Resources are created in specific Azure regions, and not every model or feature is available in every region — when you create a deployment you choose a region and a model that is supported there. Capacity is measured in tokens per minute (TPM), and you can request quota increases as your application grows.
Because the service is gated and governed, it is well suited to regulated industries (finance, healthcare, government) that need both cutting-edge models and strict control over data residency, access, and content safety. This combination — frontier models plus enterprise governance — is exactly what the AI-900 wants you to associate with Azure OpenAI.
On the Exam: The single most-tested Azure OpenAI fact is data privacy — YOUR DATA IS NOT USED TO TRAIN MODELS. Second is that you must create a deployment before calling a model. Third, remember it adds Azure's enterprise security, compliance, content filtering, and networking on top of the same OpenAI models. Access has historically required an application, though availability has broadened over time.
What is the primary advantage of Azure OpenAI Service over using OpenAI directly?
In Azure OpenAI Service, what must you create before you can use a model?
Is your prompt and response data used to train OpenAI models when using Azure OpenAI Service?
Which Azure OpenAI capability would you use to create numerical vector representations of text for semantic search in a RAG solution?
Which THREE are enterprise features that Azure OpenAI Service provides over OpenAI direct? (Select three)
Select all that apply