8.2 A.6 Deployment, Operation, Monitoring & Documentation

8.2 A.6 Deployment, Operation, Monitoring & Documentation

Auditor focus: After design and V&V, A.6 asks whether the organization releases, runs, observes, records, and retires AI systems under control. A perfect offline accuracy report with uncontrolled production deployment is a classic major gap.

---

Deployment (A.6.2.5)

Deployment control ensures only approved versions enter production under defined conditions. Typical elements:

Sampling probes:

1. Last three production promotions for a high-risk system: who approved, against which checklist, which model/data versions.
2. Evidence of rollback test or actual rollback; time-to-restore.
3. Hotfix exceptions: risk acceptance, compensating monitoring, retrospective V&V.
4. Vendor-hosted models: client-side acceptance and version pin/change control (links A.10).

Weak signals: “ML engineer has prod write access and ships when ready”; no link between model registry ID and production endpoint; canary process described but never used for AI releases; rollback plan is “retrain overnight.”

---

Operation and monitoring (A.6.2.6)

Operation control keeps the system within intended performance and risk envelope after go-live. Monitoring must be capable of detecting AI-specific degradation.

Ownership: Named on-call or product owner; thresholds; escalation path; analysis frequency. Unowned dashboards are decoration.

Link to Clause 9.1: A.6.2.6 operational monitoring for systems should align with AIMS monitoring design. Finding placement may reference both; avoid double-counting the same gap without clear requirement mapping.

Scenario: Credit model monitors only API 200-rate and CPU. Portfolio default rates for a protected subgroup spike after a data pipeline change; no alert. Deployment was clean; operation/monitoring design is inadequate. Finding against A.6.2.6 (and possibly 9.1 / A.7 if data quality broke).

---

Technical documentation (A.6.2.7)

Technical documentation supports transparency, continuity, and audit reconstructability. Typical artifacts:

• Model cards or system cards (intended use, metrics, limitations, ethical considerations).
• Architecture and data-flow descriptions.
• Evaluation reports and known failure modes.
• Runbooks for monitoring, incident response, and rollback.
• Version history and dependency inventory (libraries, foundation models, prompts).

Trap: Beautiful model card stale relative to production version. Always match document version ↔ deployed version ID.

---

Event logging and system recording (A.6.2.8)

Logging and recording under the life cycle objective typically include:

• Inference-relevant events (where proportionate): request metadata, model version, outputs or decision class, confidence, policy blocks.
• Human actions: overrides, approvals, manual labels fed back.
• Lifecycle events: training runs, evaluation completions, deployments, config changes, rollbacks, decommission.
• Security-relevant AI events: access to model artifacts, prompt/system-message changes, dataset access.

Integrity: Access control, retention period, time sync, protection against silent alteration (align with ISMS logging practices where integrated). Incomplete logs make incidents non-reconstructable—raise as control effectiveness failure, not a minor admin note.

Privacy note: Logging personal data in prompts or inputs may create legal obligations. Auditors look for defined logging scope and privacy risk handling, not “log everything forever” as automatic conformity.

---

Recording of AI system life cycle events

A practical life cycle event log (ticket system, registry, or CMDB-like AI inventory) should answer:

1. What system/version was created, changed, released, or retired?
2. When and by whom?
3. Against which approval and evaluation evidence?
4. What monitoring or incident followed?

Absence of life cycle event history for a multi-year production system is a strong indicator that A.6 is not effectively implemented.

---

Retirement and decommissioning — no dedicated Annex A control

There is no A.6.2.9. A.6.2 ends at A.6.2.8 (recording of event logs), so retirement is audited through other requirements rather than against a control of its own: the A.6.1.3 responsible design-and-development process should define the retirement stage, Clause 8.1 operational planning and control requires the process to be carried out as planned, A.4.2 resource documentation and the AI inventory must be updated, and A.7 plus Clause 7.5 govern what happens to data and records. An exam answer that cites "A.6.2.9" is wrong; an answer that raises retirement against the life cycle process and operational control is right.

Retirement controls prevent “zombie models” that still serve traffic or leak data:

• Decision and approval to retire; replacement or manual process.
• Disable endpoints, keys, and scheduled jobs.
• Archive or securely destroy models, prompts, and data per retention rules.
• Notify dependent teams and, where required, interested parties.
• Update inventory, SoA references, and monitoring coverage.

Probe: Sample one retired system—confirm no residual inference path and that records remain for the retention period required by the AIMS and law.

---

Auditor probe set: production reality

Use this checklist in Stage 2 interviews and demos:

1. Show the production model ID and open its registry/docs.
2. Show live or recent monitoring for quality/drift (not only uptime).
3. Pull one incident or override from logs; walk detection → response → record.
4. Demonstrate rollback procedure and last test/use.
5. Correlate a recent retrain/deploy event in the life cycle log with monitoring baselines reset.
6. Ask who is paged when fairness or drift thresholds breach—validate competence and authority.

Integrated ISMS/AIMS tip: Security logging may exist while decision logs do not. Conversely, ML platforms may log predictions without protecting log integrity. Test both layers when AI decisions are in scope.

Exam scenarios often contrast (a) strong offline V&V with uncontrolled deploy, (b) monitoring without ownership/escalation, (c) documentation that does not match production, and (d) missing rollback. Correct answers emphasize operational control and reconstructability, not tool brand names.