15.3 Analytical Principles in Investigations: Avoiding Confirmation Bias and Other Traps
Key Takeaways
Confirmation bias is the tendency to seek and interpret evidence that confirms an existing belief while overlooking evidence against it.
Analysis of competing hypotheses, set out in Richards Heuer's 1999 book Psychology of Intelligence Analysis, tests evidence against several explanations at once.
Automation bias is over-reliance on a tool's score or alert, such as accepting a low risk score without examining the activity.
Investigation notes should separate facts, inferences and conclusions, so that a reviewer can test each step.
CAFCA sub-topic 4.3 covers analytical principles in an investigation, with confirmation bias as the named example. Investigators make judgements under uncertainty, time pressure and incomplete information. Bias leads to two kinds of failure. Some suspicious activity is wrongly closed. Some innocent customers are wrongly reported or exited. The exam tests whether you can recognise bias and apply structured methods.
Common biases in AML and fraud work
| Bias | What it looks like in an investigation | Countermeasure |
|---|---|---|
| Confirmation bias | Deciding early that the customer is a mule, then reading every transaction as proof and ignoring the payslips | List evidence against the working theory; test alternative hypotheses |
| Anchoring | Over-weighting the first fact seen, such as the alert's title or a high risk score | Review the full file before forming a view; reassess after each new item |
| Availability bias | "Last week's case was a scam, so this one is too", because recent cases come to mind easily | Use the evidence for this case and refer to base rates |
| Automation bias | Accepting a vendor "low risk" score or an auto-closure suggestion without looking | Treat scores as inputs, not conclusions; sample auto-closures |
| Affinity or halo bias | A polished corporate website or a senior job title lowers scrutiny | Apply the same tests to every customer |
| Stereotyping | Treating nationality, religion or ethnicity as evidence of crime | Base conclusions on behaviour and facts; follow fair-treatment rules |
| Overconfidence | Closing on a "gut feeling" with little evidence recorded | Require stated evidence for every conclusion |
| Hindsight bias | In QA, judging an earlier analyst as if later facts had been known at the time | Review decisions against what was knowable then |
| Groupthink | The team accepts the first view voiced in a case meeting | Assign a devil's advocate; collect independent views first |
| Sunk cost | Continuing an unproductive investigation because so much time has gone into it | Set review points; close or escalate on the evidence |
Core analytical principles
- Objectivity: let the evidence lead. The goal is the truth, not a SAR or a closure.
- Separate fact, inference and conclusion. For example: Fact: 14 inbound transfers from unrelated senders in 48 hours. Inference: consistent with funnel activity. Conclusion: suspicious, escalate.
- Generate alternative hypotheses. For a burst of P2P inflows, consider (a) legitimate group payments, (b) a mule account, (c) the customer being a scam victim, and (d) account takeover.
- Seek disconfirming evidence. What would we expect to see if the legitimate explanation were true? Look for it.
- Weigh the reliability of sources, giving more weight to independent and authoritative sources (Chapter 8).
- Be proportionate and timely. Gather enough evidence to decide reasonably within the deadlines. Certainty is not the standard.
- Document the reasoning so that a reviewer can follow and challenge it.
Analysis of competing hypotheses (ACH)
Richards Heuer's Psychology of Intelligence Analysis (1999) popularised ACH. List the hypotheses across the top and the evidence down the side. Mark whether each item is consistent or inconsistent with each hypothesis. The hypothesis with the least inconsistent evidence is usually the strongest, even if another has more consistent items.
| Evidence | H1: legitimate group refunds | H2: mule account | H3: customer is a scam victim |
|---|---|---|---|
| 14 senders, no common link | Inconsistent | Consistent | Neutral |
| Funds forwarded within 20 minutes to a crypto exchange | Inconsistent | Consistent | Consistent |
| Several senders reported scams | Inconsistent | Consistent | Neutral |
| Customer says a "job" told them to forward funds | Neutral | Consistent (unwitting mule) | Consistent |
| Device shared with two other accounts | Inconsistent | Consistent | Inconsistent |
Here H1 has the most inconsistent evidence. H2, an unwitting mule, fits best. That affects how the firm treats the customer, possibly as a vulnerable person who has been exploited, and what goes into the SAR narrative.
Safeguards built into the process
- Templates that require alternative explanations and supporting and contradicting evidence.
- Peer review or four-eyes for escalations and for closures of high-risk cases.
- QA calibration sessions where analysts compare decisions on the same cases.
- Training on bias, with real (anonymised) examples.
Realistic exam scenario
An analyst sees an alert titled "Possible terrorist financing – high-risk country". The customer's nationality matches the country, and the analyst begins drafting a SAR. A reviewer notes that the transfers are monthly payments of US$200 to a family member, consistent with the declared purpose and income, with no adverse information. The analyst anchored on the alert title and stereotyped by nationality. The right approach is to assess the actual activity and evidence. That points to closure with a documented rationale, unless other red flags emerge.
Common exam traps
- Equating a high risk score with suspicion. A score prompts a review; it is not evidence.
- Ignoring evidence that contradicts the working theory.
- Using nationality or ethnicity as evidence.
- Hindsight bias in QA.
An investigator decides early that a customer is a money mule and then dismisses the payslips and employer letter as 'probably fake' without checking them. Which bias is most evident?
Availability bias
Confirmation bias
Sunk cost bias
Hindsight bias
In analysis of competing hypotheses, which hypothesis is usually considered strongest?
The one with the least inconsistent evidence
The one matching the alert title
The one with the most consistent evidence
The one the senior investigator proposed
A vendor tool marks a customer as 'low risk', and the analyst closes an alert without reviewing the underlying transactions. Which bias does this show?
Affinity bias
Groupthink
Anchoring on the first transaction
Automation bias
Sections you finish are checked off in the contents.