11.5 CloudWatch Metrics, Throttling & Operational Observability
Key Takeaways
- Use InvocationThrottles rather than the nonexistent InvocationsThrottled metric.
- Do not rely on a nonexistent universal ModelInvocationLatency metric to subtract internal processing stages.
- Quota accounting varies by model and endpoint, so diagnose the exact target before attributing a 429 to RPM or tokens.
11.5 CloudWatch Metrics, Throttling & Operational Observability
Amazon Bedrock runtime publishes CloudWatch metrics under the AWS/Bedrock namespace. Use the exact current metric names. Core metrics include Invocations, InvocationLatency, InvocationClientErrors, InvocationServerErrors, InvocationThrottles, InputTokenCount, and OutputTokenCount for supported operations. InvocationsThrottled and ModelInvocationLatency are not the documented runtime metric names and should not appear in alarms or exam answers.
Read latency with token context
InvocationLatency measures from request submission until the last token is received. A longer response naturally raises that metric. Diagnose latency together with input tokens, output tokens, output tokens per second, request mode, model target, and percentile. Bedrock documentation describes diagnosing latency changes through output-token rate; it does not expose a universal CloudWatch metric that cleanly subtracts raw transformer compute from Guardrail or serialization time.
| Symptom | Evidence to inspect | Likely action |
|---|---|---|
| Higher latency and more output tokens | OutputTokenCount, requested maximum, response length | Tighten output instructions or limits; stream when appropriate |
| 429 responses | InvocationThrottles, target model, request and token rates | Bound concurrency, back off with jitter, inspect exact quotas |
| 4xx increase | InvocationClientErrors and application logs | Validate IAM, request schema, model ID, token limits, and Guardrail configuration |
| 5xx or capacity errors | InvocationServerErrors, response codes, request IDs | Retry eligible transient errors and open support evidence if sustained |
| Cost increase without traffic growth | Input/output/cache usage and prompt versions | Find prompt bloat, loops, cache misses, or model changes |
CloudWatch aggregate metrics do not contain the full cause. Correlate them with structured application logs, X-Ray segments around application services, model invocation logging when permitted, and request IDs. Do not place raw sensitive prompts into ordinary logs by default.
Metric-to-alarm examples
Build a throttle-rate metric from InvocationThrottles divided by attempted eligible calls using service and application telemetry, then alarm on a sustained rate tied to the service objective. InvocationThrottles alone counts what Bedrock reports but does not describe user impact after retries. Track retry success and final failures separately.
For latency, chart p50, p95, and p99 InvocationLatency with OutputTokenCount and operation. Streaming requests measure through the last token, so add application time-to-first-token when user perception matters. An output-token increase after a prompt release can explain both latency and cost without any service degradation.
Client errors should be grouped by application error code such as access denied, validation, unsupported model, context limit, or Guardrail configuration. Server and transient-capacity errors need request IDs, UTC timestamps, target identifiers, and retry outcomes for support escalation. Avoid high-cardinality dimensions in CloudWatch metrics; store detailed IDs in controlled logs and correlate them.
Annotate dashboards with model, prompt, Guardrail, deployment, and quota changes. Without change markers, operators can mistake an intentional longer answer for an infrastructure incident.
Logging-delivery metrics and content controls
Model invocation logging delivery has its own success and failure telemetry. Alarm on failed delivery when the organization relies on those logs for audit or troubleshooting. Verify the destination policy, KMS permissions, and service configuration; an inference success does not prove its optional payload log was delivered.
Invocation logging can contain prompts, responses, images, or other sensitive material depending on configuration. Enable only the modalities and destinations the policy permits, encrypt them, restrict readers, and apply retention. CloudTrail records management and supported data events, while model invocation logging captures configured inference payloads; neither should be described as a substitute for the other.
Use metric filters cautiously on payload logs. Natural language is high-cardinality and can contain personal data. Prefer structured application outcome fields and sampled controlled review for quality metrics.
Business and model telemetry
Add task completion, validated-output rate, retrieval success, tool success, fallback use, human escalation, and user correction beside service metrics. Correlate by release and model target without exposing sensitive payloads. This reveals cases where infrastructure remains healthy while model behavior deteriorates.
Use sampled quality evaluation rather than attempting to infer hallucination directly from InvocationLatency or token counts. Operational metrics can identify when and where to investigate; they do not by themselves judge truth or responsibility.
Alarm ownership
Every alarm needs an owner, runbook, severity, and expected action. Test notifications and permissions. An unactionable alarm that fires on normal retries trains operators to ignore the signal, while a missing-data state can conceal disabled telemetry. Review thresholds after load or model changes and record why they still represent the service objective.
Which documented CloudWatch metric counts throttled Bedrock runtime invocations?