10.1 Configure and Monitor Computer Use
Key Takeaways
- Computer use is a generative-orchestration tool that controls a Windows GUI with vision and reasoning so agents can automate apps and sites that lack APIs.
- Required configuration fields are Name, Description, Model, and Instructions; optional settings include inputs, machine, credentials, human supervision, stored credentials, access control, and Enforce HTTPS.
- Run targets include hosted browser (preview), Cloud PC pool (preview), and bring-your-own machines registered in Power Automate with computer use enabled.
- Prefer dedicated machines, least-privilege accounts, allow lists, and stored credentials (internal storage or Azure Key Vault) over sharing broad maker access.
- Monitor runs from agent Activity (Activity map and Transcript); advanced logging in Dataverse supports session replay, verbosity, retention, export, and optional Microsoft Purview audit.
Why computer use matters on AB-620
Configure and monitor computer use for an agent is a Domain 2 skill under Add tools to agents (about 40–45% of the exam). Microsoft expects you to know when computer use is the right tool, how to configure it safely, where it runs, and how to inspect what it did after a run—not only that the feature exists.
Computer use is a tool in Copilot Studio that lets an agent interact with websites and desktop applications by selecting buttons, choosing menus, and typing into fields. You describe the task in natural language; the agent runs that task on a configured Windows computer using a virtual mouse and keyboard. It is powered by Computer-Using Agents (CUA) models that combine vision with reasoning, so the agent can adapt when labels or layouts shift instead of relying on brittle pixel scripts alone.
Computer use requires generative orchestration turned on. Without it, the orchestrator cannot treat computer use as a selectable tool at runtime.
When to choose computer use vs API, connector, or MCP
| Situation | Prefer | Why |
|---|---|---|
| Public or private REST API with OpenAPI and auth you control | REST API tool or custom connector | Structured inputs/outputs, DLP-aware connectors, lower ambiguity |
| Service already has a Power Platform connector | Connector tool | Managed auth patterns, reuse across Power Automate and agents |
| Tool catalog already published as MCP tools/resources | MCP server tool | Dynamic discovery of tools; standardized protocol |
| System is GUI-only (legacy client, vendor portal, no API) | Computer use | If a person can click through the app, computer use can too |
| Multi-step deterministic process with known connectors | Agent flow | Fixed path, capacity-metered actions, human-in-the-loop approvals |
Exam trap: computer use is not a replacement for knowledge grounding. Use knowledge sources for Q&A over documents; use computer use for side-effecting UI work (submit forms, extract values from screens, navigate multi-page wizards).
Add computer use to an agent
- Open the agent → Tools → Add tool.
- Choose New tool → Computer use.
- Select Add and configure.
- Complete the four required fields, then Save.
| Required field | Role for the orchestrator and runtime |
|---|---|
| Name | Display name so makers (and sometimes users) distinguish this tool from others |
| Description | Short statement of what the tool does and when the agent should call it—critical for generative tool selection |
| Model | CUA execution model (for example OpenAI Computer-Using Agent standard GA, Anthropic Claude Sonnet variants standard/experimental, Claude Opus premium experimental). Anthropic models need admin access to external models for the environment |
| Instructions | Step-by-step task description including URLs and application names; this is the playbook the model follows |
Optional configuration that appears on exams
Beyond the four required fields, review:
- Inputs — Dynamic values that change each run (customer ID, invoice number). At execution, computer use combines instructions with input values.
- Machine — Target machine or pool; refresh list, manage machines, or open machine details in Power Automate.
- Connection — Connection identity used by the tool.
- Credentials to use — Maker-provided (default; good for autonomous agents) or End user (each user must have machine access). Sharing an agent with maker-provided credentials means users can act with the original author’s access on that machine—treat as a high-risk design choice.
- Human supervision — Email (Outlook) reviewers when the model detects potentially harmful instructions that could alter behavior. Response time limit ends the run if no one responds. Reviewers who did not initiate the run may not see the same activity context; assign authorized reviewers who understand the process.
- Stored credentials — Website or desktop app sign-in secrets in Power Platform internal storage or Azure Key Vault (subscription, resource group, vault name, secret name). Map username + password/secret to site host or desktop process name (Task Manager Processes tab for process names).
- Access control — Allow list of websites and desktop apps. Access control blocks interaction outside the list; it does not always prevent the model from opening another site (for example via a search bar). Design allow lists with that nuance in mind.
- Enforce HTTPS — When on, computer use does not interact with
http://sites, reducing cleartext exposure risk.
Where computer use runs
| Target | Intent | Notes |
|---|---|---|
| Hosted browser (preview) | Quick web automation in a Microsoft-managed environment | Not Entra-joined to your tenant; not for enterprise resource access or custom desktop apps |
| Cloud PC pool (preview) | Scalable Windows 365 for Agents compute without BYO hardware | Auto-scale, work/school account access, Entra joined and Intune enrolled for governance |
| Bring-your-own machine | Customer-managed Windows endpoints | Power Automate for desktop (supported version), register machine, turn on Enable for computer use |
BYO requirements to memorize: install Power Automate for desktop with machine-runtime connectivity, register to the same environment, enable computer use on the machine settings page, and prefer a dedicated machine so interactive users are not interrupted. When a machine is busy, additional runs queue (Queued → Next to run → running).
Test, publish, and conversational vs autonomous use
After Name, Description, and Instructions are saved, use Test. The test experience shows a step log of reasoning/actions and a live preview of the machine. Use Stop testing to halt mid-run. Refine instructions if outcomes drift.
Publish the agent after configuration. Computer use works especially well for autonomous background agents. In conversational channels:
- End-user credential mode requires each chat user to have valid machine credentials.
- Runs may surface reasoning messages and screenshots in chat—plan privacy and channel suitability.
Monitor computer use
- Open the agent Activity section and select a run.
- Switch between Activity map and Transcript.
- Transcript shows step-by-step reasoning and screenshots for each action.
Advanced activity (when logs are stored in Dataverse) opens a side panel from a computer use action with:
- Session replay (screenshot series)
- Per-action types, coordinates, user context, timestamps
- Summary: instructions, inputs, duration, action counts, escalations, machine name/login
- Websites and applications accessed
- Credentials used
- Export session logs for offline review
Admin configuration (Power Platform admin center → Environment → Settings → Products → Features → Computer Use):
| Setting | Purpose |
|---|---|
| Store logs in Dataverse | Default on when Computer Use feature is enabled; off leaves only default activity logs |
| Verbosity | All data (default), Data without screenshots, or Minimal |
| Log retention | Default 7 days; custom up to long durations; 0 or -1 for keep forever per product guidance |
| Send audit logs to Microsoft Purview | Independent; activity term CUAOperation |
Capacity note: advanced logs use Database (flowsession / binary metadata), Log (flowlog), and File storage—plan retention and verbosity with compliance and capacity together.
Transcripts can still include basic computer use logs and screenshots even when advanced Dataverse log settings differ—do not confuse agent transcript storage with the Computer Use advanced log toggles.
Scenario: Contoso legacy claims portal
Contoso’s carrier portal has no public API. Finance wants an agent to log in, open a claim, and copy status fields into Dataverse.
- Prefer computer use over inventing screen-scrape HTTP tools that violate terms of service or break weekly.
- Register a dedicated Windows machine (or Cloud PC pool for scale), enable computer use, least-privilege portal account.
- Store portal password in Key Vault-backed stored credentials; enable access control for the portal host and browser process; turn on Enforce HTTPS.
- Write instructions with exact URLs, field names, and failure recovery (“if MFA prompt appears, stop and escalate”).
- Assign human supervision reviewers for risky instruction detection; set a realistic response timeout.
- After pilot runs, inspect Transcript and session replay; tighten instructions where the model hesitated.
- If the vendor later ships an OpenAPI, migrate to a REST or custom connector tool and retire GUI automation for that path.
Exam checklist
- Generative orchestration must be on.
- Name + Description drive when the tool is selected; Instructions drive how it acts.
- Choose run target consciously (hosted browser vs Cloud PC pool vs BYO).
- Treat maker-provided credentials and shared agents as a privilege boundary problem.
- Use access control, HTTPS enforcement, dedicated machines, and stored secrets.
- Prove behavior with Test, then Activity/Transcript and advanced Dataverse monitoring for production readiness.
A maker wants an agent to complete weekly data entry in a vendor web app that has no API. Generative orchestration is already enabled. Which tool type is the best primary fit?
Which four fields must be configured when you add and configure a computer use tool before optional settings?
Operations needs step-by-step screenshots and session replay for computer use audits. Store logs in Dataverse is on. Where should a maker start inspecting a completed autonomous run inside Copilot Studio?