9.1 Service Bus Queues & Message Processing

Why Service Bus Queues Matter for AI Solutions

Azure AI workloads rarely finish in a single synchronous HTTP call. Uploading a PDF, extracting text, chunking, generating embeddings, running retrieval, and calling a chat model can take seconds to minutes. Azure Service Bus queues let you decouple producers from consumers: a web API or Function accepts the user request, enqueues a small work message, and returns quickly while a scaled worker fleet processes jobs asynchronously.

On the AI-200 exam, expect questions that mix messaging concepts with Azure AI scenarios—document intelligence pipelines, batch transcription, offline evaluation jobs, and agent tool invocations that must not be lost if a worker restarts.

Queues vs Other Azure Messaging Options (Exam Framing)

Use Service Bus when you need reliable processing guarantees, poison-message handling, sessions, scheduled messages, or topics (covered in the next section). Use Event Grid when something happened and many subscribers should react. Use Event Hubs when you ingest continuous telemetry at massive scale.

Message Anatomy for AI Jobs

A Service Bus message has a body (string/bytes/JSON) and user/application properties (key-value metadata). For AI pipelines, put identifiers and routing hints in properties, and keep the body as a compact work contract:

{
  "jobType": "embed-document",
  "documentId": "doc-7841",
  "blobUri": "https://stg.blob.core.windows.net/inbox/doc-7841.pdf",
  "correlationId": "req-9f2a",
  "priorityHint": "normal"
}

Best practices for AI back-end operations:

• Do not put multi-megabyte PDFs or base64 embeddings in the message body—store content in Blob Storage and pass a URI.
• Include a correlation ID so logs, Application Insights, and user status APIs can stitch the pipeline together.
• Prefer idempotent workers: if the same message is delivered twice, re-running embedding generation or writing the same vector with the same document ID should be safe.
• Use scheduled enqueue time when you want delayed retries or overnight batch scoring without a separate timer job.

Receive Modes: Peek-Lock vs Receive-and-Delete

Service Bus supports two receive modes. Choosing the wrong one is a common exam trap.

Peek-Lock (default for most SDKs): The broker delivers a message and places a lock on it for a lock duration (configurable; often around 30–60 seconds, extendable with lock renewal). While locked, other receivers cannot see the message. Your worker then:

1. Processes the AI job (download blob, call Document Intelligence, write vectors).
2. Calls Complete to permanently remove the message on success.
3. Calls Abandon to release the lock early so another worker can retry sooner.
4. Calls DeadLetter when the payload is invalid and retries will never succeed.
5. Optionally Defer to park the message for later retrieval by sequence number when a dependency is not ready yet.

If the lock expires before Complete, the message becomes available again (at-least-once delivery). That is why idempotency matters for embedding writes and status updates.

Receive-and-Delete: The broker removes the message as soon as it is delivered to the receiver. Throughput can be slightly higher and coding simpler, but if the worker crashes after receive and before finishing the AI job, the work is lost. Use this mode only for disposable telemetry-like work where loss is acceptable—almost never for paid inference jobs or compliance document processing.

Lock Duration, Prefetch, and Competing Consumers

AI jobs vary wildly in duration. OCR on a 100-page contract may exceed a short lock. Patterns that keep Peek-Lock reliable:

• Enable automatic lock renewal in the processor/SDK while work is in flight.
• Keep the queue message as a short-lived orchestration token; do long GPU inference on a separate compute path that checkpoints progress in durable storage.
• Scale competing consumers (multiple Function instances or container workers) against the same queue for parallelism; each message goes to one active lock holder.
• Tune prefetch carefully: high prefetch improves throughput for tiny jobs but can hold locks on messages your instance is not ready to process, increasing abandon/lock-expiry churn for heavy AI work.

Message Sessions for Ordered AI Workflows

Without sessions, Service Bus queues do not guarantee global FIFO across concurrent consumers. Message sessions group messages that share a SessionId. Only one consumer processes a given session at a time, preserving order within that session.

AI scenarios that need sessions:

• Process pages of the same document in order before merging results.
• Apply conversation-state updates for one conversationId sequentially so tool calls do not race.
• Ensure a user's profile enrichment jobs run in submission order.

Session-enabled receivers accept a session, process its messages, then close/release the session so another worker can take the next available session. Design session IDs as natural partition keys (document ID, tenant ID + user ID)—not a single global ID, which would serialize your entire farm.

End-to-End Pattern: Async Document Embedding Pipeline

1. API receives upload → stores PDF in Blob Storage → enqueues Service Bus message with blob URI and job type.
2. Worker receives with Peek-Lock → runs extraction/chunking → writes chunks to storage.
3. Worker enqueues a follow-up message (or uses the same message with a stage property) for embedding generation.
4. Embedding worker writes vectors to Azure AI Search / vector DB → Completes message → publishes a status event (Event Grid) so the UI updates.

At each stage, failures that are transient (throttling, 429s from the model endpoint) should Abandon or schedule a retry. Permanent failures (unsupported file type) should DeadLetter with a reason code your ops team can query.

Exam Watch-Outs

• Peek-Lock + Complete = reliable processing; Receive-and-Delete = delete-on-receive risk.
• Lock expiry without Complete causes redelivery (duplicate work risk).
• Sessions give per-session ordering, not global ordering of the whole queue.
• Large payloads belong in Blob Storage; messages carry pointers and metadata.
• Service Bus is the right choice for back-end operations that must be queued and processed with enterprise messaging features—exactly the official skill wording for this domain.