9.1 Managing and Monitoring Agents
Key Takeaways
- Agentforce observability has three layers: infrastructure that captures telemetry, build-and-test tools, and observe tools for production monitoring.
- Agentforce Session Tracing captures turns, reasoning steps, actions, prompt and gateway inputs and outputs, errors, and responses, and it must be enabled before other observability tools have data.
- Agent Health Monitoring sends email alerts when metrics such as error rate, interaction latency, escalation rate, or response rate breach thresholds.
- Monitor Agent Guardrails tracks instruction adherence and task resolution trends separately from broader analytics.
- Users with Manage AI Agents can manage, activate, deactivate, and monitor all agents org-wide, so assign it only to those who need that access.
9.1 Managing and Monitoring Agents
Quick Answer: Managing agents covers who can build and change them (Manage AI Agents, Service agent managers), who can use them (Employee agent access), and what version is live (draft, commit, activate, deactivate). Monitoring relies on a layered observability toolchain: Agentforce Session Tracing and audit and feedback data capture telemetry, then Agent Analytics, Agent Optimization, session traces, Agent Health Monitoring alerts, and guardrail monitoring show how agents perform in production.
Managing Agents: Access and Lifecycle Controls
| Control | What it governs | Key detail |
|---|---|---|
| Manage AI Agents permission | Access to Agentforce Builder and management of agents | Org-wide. Users can manage, activate, deactivate, customize subagents and actions, and monitor all agents |
| Manage Agentforce Service Agents permission set | Creating and editing Service agents | Includes Manage AI Agents. Admins can assign it to define Service agent managers |
| Agent Access (Employee agents) | Which employees can use an agent | Permission sets or profiles on the Agent Access page or in Enabled Agent Access |
| Agent user (Service agents) | What data Service agents can reach | Least-privilege permissions; only admins can view or edit agent users |
| Versions | What's live | Draft → commit → activate. Compare versions, create drafts from committed versions, delete versions (not recoverable) |
| Activate / Deactivate | Availability on connected channels | Activating makes the agent immediately available to users on its channels |
| Model provider access | Which LLM providers can be used | Einstein Setup → Configure Model Providers (section 5.4) |
| Audit and feedback data | Record of prompts, responses, and trust signals | Stored in Data 360 (section 5.3) |
Good governance practice: build in sandboxes, test with Testing Center, deploy through a pipeline, keep the previous committed version for rollback, and restrict management permissions to a small group.
The Observability Toolchain
Salesforce describes observability as a layered toolchain, not a single tool.
Layer 1: Infrastructure (turn these on first)
| Tool | Where | What it captures |
|---|---|---|
| Agentforce Session Tracing | Setup → Agentforce Session Tracing | Every turn, reasoning step, action, and message for every session. Must be enabled before other observability tools have data |
| Agent Platform Tracing | Einstein Audit, Analytics, and Monitoring Setup | Service-level telemetry across the Atlas reasoning engine, Apex, Flows, Prompt Builder, invocable actions, AI Gateway, LLM Gateway, and query federation |
| Audit and Feedback | Einstein Audit, Analytics, and Monitoring Setup | LLM inputs and outputs, retrieval quality data, trust evaluations, guardrail outputs, user feedback. Powers quality and trust metrics |
| Knowledge/RAG Quality Data and Metrics | Same Setup page | Retrieval scores such as answer faithfulness, answer relevance, and context relevance |
| Einstein Trust Layer | Einstein Trust Layer Setup | Instruction adherence and toxicity scores from LLM Gateway outputs |
Layer 2: Build and Test
| Tool | Purpose |
|---|---|
| Agentforce Builder (Simulate and Live Test) | Iterative testing |
| Interaction Summary | Subagent selection, actions, and reasoning for a test conversation |
| Testing Center | Batch evaluation before deployment. Custom scorers later surface in Agent Analytics |
Layer 3: Observe (production)
| Tool | Purpose |
|---|---|
| Agent Analytics | Aggregated trends across sessions in Tableau Next dashboards (section 9.2) |
| Agent Optimization | Session-level review with intents, outcomes, and quality scores (section 9.2) |
| Interaction Summary and Trace | Per-session breakdown and a time-ordered step record with timing and errors |
| Agent Health Monitoring | Alerts on operational metrics |
| Monitor Agent Guardrails | Instruction adherence and task resolution trends |
| Omni Supervisor – Agentforce tab | Near-real-time view of active voice calls with live transcripts |
| Data 360 Query Editor | SQL over the underlying data for custom analysis |
The toolchain works best in sequence: make sure infrastructure data is flowing, catch issues in build and test, then track and investigate in production.
Agentforce Session Tracing
Session tracing combines data from sources such as reasoning engine logs under a session ID. It supports all agent types, and you can query it through DMOs and reports. Use it to:
- Debug configuration issues at build time
- Learn why certain batch test cases failed
- Explain why an agent can't handle a set of questions or goes off topic
Key Session Tracing Data Model (STDM) objects:
| DMO | Contains |
|---|---|
AiAgentSessionDmo | Session facts: channel type, how the session ended, related messaging or voice session IDs |
AiAgentSessionParticipantDmo | Who joined: contact, lead, user, agent API name |
AiAgentInteractionDmo | One turn, including TopicApiName (which subagent handled it) |
AiAgentInteractionStepDmo | One step with start and end timestamps and ErrorMessageText |
AiAgentInteractionMessageDmo | The actual input and output text |
Session tracing data is stored in Data 360 and consumes credits. A beta Session Trace OTel API exports a single session's trace in OpenTelemetry format for observability platforms such as Splunk, Datadog, or New Relic.
Agent Health Monitoring
Agent Health Monitoring detects silent failures, such as spiking error rates or high latency, and notifies you in near real time.
| Metric | What it tells you |
|---|---|
| Agent Error Rate | Percentage of interactions with an error |
| Average Interaction Latency | Average milliseconds to respond to an interaction |
| Escalation Rate | Percentage of interactions escalated to a human |
| Agent Response Rate | Percentage of user turns that got a response within 60 seconds |
- Each alert monitors one metric for one agent and sends an email notification when a threshold is breached.
- Notifications link directly to the sessions that triggered them, so you can go straight to the trace.
- Health metrics also appear on the Health tab of Agent Analytics.
- It's built on Agentforce Session Tracing, visualized with Tableau Next, and available in sandbox and Data Cloud One orgs.
Monitoring Guardrails
Monitor Agent Guardrails gives a focused view of two guardrail metrics:
- Instruction adherence: how well agents interpret subagent instructions when generating responses
- Task resolution: how well agents understand the user's intent and resolve the task
This matters where guardrail compliance is a tracked requirement, such as regulated industries, enterprise policy enforcement, and brand safety.
Monitoring Usage and Cost
Agents, data libraries, session tracing, and Testing Center all consume credits. Digital Wallet is the source of truth for consumption. Sandbox agent usage is written to the production org's usage data, tagged with the sandbox org ID and environment type.
Scenario Practice
| Scenario | Tool or control |
|---|---|
| Agent Analytics dashboards show no data after go-live | Confirm Agentforce Session Tracing (and Audit and Feedback) is enabled |
| Operations wants an email when a Service agent's error rate spikes | Agent Health Monitoring alert on Agent Error Rate |
| A compliance team tracks whether agents follow instructions over time | Monitor Agent Guardrails (instruction adherence) |
| A supervisor wants to watch live voice calls handled by agents | Omni Supervisor, Agentforce tab |
| A contractor should adjust one agent's instructions but not manage other agents | Manage AI Agents is org-wide, so restrict who gets it and use process controls |
| A bad release needs rollback | Activate the previous committed version |
Exam Traps
- Session tracing comes first. Without it, analytics and optimization have no data.
- Manage AI Agents isn't scoped to one agent.
- Health Monitoring is for rapid alerts, while Agent Analytics is for trends over time.
- Live voice monitoring happens in Omni Supervisor, not in the Analytics Voice tab.
An org activates Agent Analytics after launching an agent, but the dashboards show no session data. Which infrastructure feature must be enabled for observability tools to have data?
An operations team wants an email notification when a Service agent's error rate rises above an acceptable level. Which feature should they configure?
Which statement about the Manage AI Agents permission is accurate?
A supervisor needs to watch active voice calls handled by an Agentforce Voice agent with live transcripts. Where do they do this?