12.2 Evidence Analysis — Corroboration & Evaluation

12.2 Evidence Analysis — Corroboration & Evaluation

Quick Answer: After collecting evidence, auditors analyze it for sufficiency, appropriateness, and reliability; corroborate across independent sources; resolve contradictions; and evaluate results against audit criteria (ISO 14001, compliance obligations, and the organization's EMS). Findings must follow from evidence — not from auditor preference or incomplete trails.

Collection without analysis produces notebooks, not audit conclusions. Lead Auditor candidates are expected to show how evidence supports (or fails to support) conformity statements, when more evidence is needed, and how conflicting information should be handled before a finding is raised.

---

1. The Analysis Sequence

A practical sequence used in ISO 19011-style audits:

1. Organize evidence by process, clause, or risk theme
2. Test quality — is it relevant, reliable, and sufficient?
3. Corroborate — do independent sources agree?
4. Evaluate against stated audit criteria
5. Classify the outcome (conformity, nonconformity, or other finding type)
6. Record the rationale and remaining uncertainty

Skipping steps — especially corroboration — is how auditors raise unsupported nonconformities or miss systemic failures.

---

2. Sufficiency, Appropriateness, and Reliability

Sufficiency is risk-sensitive. A low-risk housekeeping control may need less depth than a significant aspect tied to a legal discharge limit. Exam scenarios often hide this distinction: the "right" answer increases evidence depth where environmental risk and compliance risk are high.

---

3. Corroboration — Triangulate Before You Conclude

Corroboration means checking whether independent evidence sources tell a consistent story.

Classic EMS triangulation:

• Say (interview): Operator explains secondary containment inspection
• Show (observation): Containment is cracked / drain valve open to stormwater
• Prove (record): Last inspection checklist marked "OK" the same morning

When sources conflict, do not average them into a vague feeling. Investigate:

• Is the record falsified, rushed, or poorly designed?
• Is the interviewee describing the intended process rather than actual practice?
• Is observation during an abnormal condition (maintenance shutdown)?
• Is documented information outdated relative to current operations?

Corroboration also works positively: interview + matching records + observed practice can justify a clear conformity statement for that control within the sampled scope.

---

4. Evaluating Against Audit Criteria

Audit criteria for an ISO 14001 audit typically include:

• Requirements of ISO 14001:2015 (and any CB interpretation rules)
• The organization's EMS documented information (scope, procedures, criteria it set for itself)
• Compliance obligations the organization identified as applicable (legal and other requirements)

Evaluation rule: compare evidence to criteria, not to the auditor's personal "best practice" wishlist. If practice differs from a procedure but still meets ISO 14001 and legal requirements, you may have a documentation/control issue under the organization's own requirements — still a potential nonconformity if the organization committed to that procedure — but you must cite the correct requirement.

Performance evaluation linkage (Clause 9 thinking): Evidence that monitoring occurred is not the same as evidence that results were analyzed, evaluated against criteria, and used in decisions. Trace whether exceedances or adverse trends triggered communication, investigation, or corrective action as the EMS requires.

---

5. Handling Incomplete or Ambiguous Evidence

Not every gap is immediately a nonconformity. Options:

More evidence needed is a legitimate interim conclusion. Raising a major nonconformity on a single ambiguous data point without follow-up is an exam trap answer.

---

6. Distinguishing Fact, Inference, and Opinion

• Fact: "Calibration certificate for pH meter SN-118 expired on 12 March; meter was in use on 20 March sampling run."
• Supported inference: "Monitoring results from 20 March are not demonstrated as reliable under the organization's calibration control."
• Opinion (avoid as a finding basis): "The lab looks disorganized, so results are probably wrong."

Findings should be written so another competent auditor could reach a similar conclusion from the same evidence. That is the fairness and evidence-based approach principles in action.

---

7. Systemic Patterns vs Isolated Lapses

Analysis includes deciding whether issues are isolated or indicative of system failure:

• Same failure across shifts, departments, or sites
• Multiple related minors against one process (e.g., competence, operational control, and monitoring all weak for one significant aspect)
• Absence of an entire required process (no compliance evaluation performed)
• Recurrence of a previously closed corrective action

Pattern recognition supports classification in the next section (major vs minor) and strengthens the audit conclusion. Document the pattern with multiple evidence points — do not label something "systemic" from a single slip without expansion of the sample.

---

8. Team Review Before Finalizing

Lead auditors facilitate team analysis: compare notes, challenge weak trails, align on criteria citations, and agree where evidence is still insufficient. Technical experts contribute interpretation; the team still decides conformity against criteria. Unresolved contradictions should be pursued before the closing meeting whenever feasible.

---

9. Exam Traps

• Equating "I don't like this method" with nonconformity without a criterion
• Accepting perfect documents when observation contradicts them
• Calling a single error systemic without expanded sampling
• Ignoring compliance obligations when only checking ISO clause text
• Writing conclusions that go beyond what the sample can support

Strong analysis turns raw notes into defensible findings: corroborated, criterion-linked, risk-aware, and honest about sampling limits.