Batch Screening and Fuzzy Logic

Batch Screening and Fuzzy Logic

Screening compares customer and counterparty names against sanctions lists, PEP lists, and watchlists. Two modes coexist. Real-time (single-record) screening checks one customer at onboarding or one payment in flight. Batch screening rescreens the entire existing customer or counterparty population — typically overnight or when a list changes — so that newly added sanctioned parties are caught even though those customers were screened clean when they joined.

Why batch screening is mandatory

Sanctions lists change frequently. The U.S. Office of Foreign Assets Control (OFAC), the United Nations, the EU, and the UK add and remove parties on no fixed schedule. Because OFAC sanctions are strict liability — a violation can be penalized regardless of intent or knowledge — an institution cannot wait for a customer's next periodic refresh to discover that an existing customer was added to a list yesterday. Best practice is to rescreen the affected population promptly after each list update.

Fuzzy logic: matching imperfect names

Exact-string matching fails on real-world names. "Mohammed," "Muhammad," and "Mohamad" are the same name; "Smith, John" and "John Smith" reorder tokens; data-entry typos abound. Fuzzy matching assigns a similarity score using techniques such as:

Tuning the match threshold

Each algorithm outputs a score (often 0–100%). The institution sets a threshold above which a potential match becomes an alert. This is the central engineering and compliance decision:

• A lower threshold (e.g., 75%) catches more variants — fewer false negatives — but generates many false positives, overwhelming analysts.
• A higher threshold (e.g., 95%) reduces alert volume but risks missing a true sanctioned party (a false negative), which is the more dangerous error.

Thresholds must be risk-based, documented, governed, and periodically tested. Lowering a threshold to reduce a backlog without governance approval is a classic exam-wrong answer.

Worked scenario

OFAC adds "Viktor Ivanov" to the SDN list. A batch rescreen of existing customers returns a 91% fuzzy match to a customer recorded as "Victor Ivanof." The correct action is not to dismiss it as a misspelling. The analyst escalates, confirms identity using date of birth, nationality, and address against the SDN entry, and — if it is the same person — freezes/blocks the property and files the required OFAC report. A near-but-not-exact spelling is exactly what fuzzy logic exists to catch.

List management and data quality

Screening is only as good as the lists and the data fed into it. Programs must consume multiple lists — OFAC's Specially Designated Nationals (SDN) and Consolidated lists, UN, EU, UK OFSI, and internal watchlists — and refresh them promptly when publishers update. Equally important is data quality on the customer side: missing dates of birth, truncated names, or mis-mapped country fields all degrade match accuracy. A high-quality date of birth or national ID dramatically improves the ability to confirm or dismiss a fuzzy hit, which is why screening effectiveness and CDD data completeness are tightly linked.

Reducing false positives without weakening coverage

The right way to cut false positives is not simply raising the threshold. Better levers include: adding secondary identifiers (date of birth, nationality) to disambiguate; maintaining a governed good-guy / whitelist of previously cleared true non-matches so the same false positive does not re-alert; suppressing obviously irrelevant list entries (e.g., entries with no overlapping data); and improving source data quality. Each of these must be documented and periodically tested.

Whitelisting must be controlled — blanket exclusions that suppress true matches are a serious failure, so whitelist entries are reviewed and re-screened when lists change.

Batch versus real-time, and what triggers a rescreen

The two modes serve different moments. Real-time screening prevents onboarding or paying a listed party at the instant of the event. Batch screening is the safety net for everyone already on the books. A rescreen of the existing population should be triggered whenever a watchlist is published or updated, whenever the institution's internal list changes, and whenever a customer record is materially updated (a new name, new beneficial owner, or new address must be re-screened). Programs also rescreen after fixing data-quality issues, because better customer data can surface matches that incomplete data previously hid.

Relying on onboarding screening alone — and never rescreening — is one of the most consequential control gaps, because list additions of existing customers would go undetected indefinitely.

Worked governance point

When tuning is proposed, the institution should sample alerts both above and below the chosen threshold to measure missed true matches before any change is approved — the same below-the-line discipline used for transaction monitoring. The output of every tuning exercise is documented, validated independently, and retained for examiners.

Common traps

• Assuming a clean onboarding screen is permanent — lists change, so batch rescreening is required.
• Raising thresholds to cut alert volume without governance, creating hidden false negatives.
• Dismissing fuzzy matches because the spelling differs — transliteration and typos are expected.
• Whitelisting matches in bulk without review, suppressing genuine hits.
• Treating false negatives and false positives as equally serious; under strict-liability sanctions regimes, a missed true match is far worse.