Section 5.3: Audit Sampling Methodologies (Judgmental vs. Statistical)

Section 5.3: Audit Sampling Methodologies (Judgmental vs. Statistical)

The Concept of Audit Sampling

Auditing is a sampling exercise. In a typical ISO 9001:2015 audit, it is physically and economically impossible for an auditor to inspect every single transaction, record, or activity. ISO 19011:2018 Annex B.3 defines audit sampling as the selection of less than 100% of the items within a population of relevance to the audit (the audit population) to enable the audit team to reach conclusions about the entire population.

Sampling introduces sampling risk: the risk that the audit team's conclusions based on a sample may be different from the conclusion that would be reached if the entire population were audited. To manage this risk, lead auditors must understand and apply two primary sampling methodologies: Judgmental (Non-statistical) Sampling and Statistical Sampling.

Judgmental (Non-statistical) Sampling

Judgmental sampling relies on the auditor's professional judgment, experience, and knowledge to select sample items. Rather than using mathematical probabilities, the auditor targets items that are most likely to contain errors, represent high risk, or are critical to the quality management system.

Determinants for Judgmental Selection

When designing a judgmental sample, the auditor considers:

• Process Complexity: Highly complex processes (e.g., software design) require more targeted sampling.
• Risk Assessment: Processes with high risk of product failure or customer complaints are heavily sampled.
• Process History: Areas that had nonconformities in previous audits are given closer attention.
• Process Maturity: Newer processes or those managed by newly hired staff are sampled more intensively.
• Key Transactions: Unusual or high-value transactions (e.g., high-dollar purchasing contracts) are prioritized.

Advantages and Disadvantages

• Advantages: It is highly cost-effective, directly targets areas of known risk, requires less time to execute, and allows the auditor to use their professional expertise to find hidden issues.
• Disadvantages: The sample is highly subjective and prone to auditor bias. Furthermore, the findings from a judgmental sample cannot be mathematically generalized to the entire population. For example, if an auditor judgmentally selects 5 high-risk training files and finds 2 errors, they cannot state that "40% of all company training records are nonconforming."

Statistical Sampling

Statistical sampling uses probability-based selection methods and mathematical models to evaluate sample results. It requires that every item in the population has a known, non-zero chance of being selected. This methodology is particularly suited for large, homogeneous populations, such as thousands of production test records, sales orders, or training logs.

Common Selection Methods

• Simple Random Sampling: Every item in the population has an equal chance of selection, often generated using a random number table or software.
• Systematic Sampling: Selecting every n-th item from a list (e.g., choosing every 50th invoice) starting from a random point.
• Stratified Sampling: Dividing the population into subpopulations (strata) based on a key characteristic (e.g., separating records by shift or department) and then sampling randomly from each stratum.

Sample Size and Confidence Levels

In statistical sampling, the sample size is determined by three variables:

1. The Confidence Level: The desired level of certainty that the sample represents the population (typically 95%).
2. The Tolerable Error Rate: The maximum rate of deviation the auditor will accept before concluding the process is ineffective.
3. The Expected Error Rate: The rate of error the auditor expects to find in the population before sampling begins.

Advantages and Disadvantages

• Advantages: It is highly objective, mathematically defensible, eliminates auditor bias, and allows the findings to be statistically generalized to the entire population with a defined margin of error.
• Disadvantages: It is resource-intensive, requires mathematical training, and can be ineffective if the population is small or non-homogeneous. Furthermore, it might miss specific high-risk items because every item has an equal probability of selection.

Practical Scenario: Auditing Competence Records (Clause 7.2)

Consider a multinational manufacturing company with 1,200 employees. The auditor is evaluating compliance with Clause 7.2 (Competence).

• Using a Judgmental Approach: The lead auditor targets 10 employees. They select 3 newly hired operators on the night shift, 2 engineers working on a newly launched product line, 2 quality inspectors, and 3 managers who recently assumed new roles. This sample directly targets high-risk training transitions.
• Using a Statistical Approach: The auditor obtains the roster of all 1,200 employees. Using a systematic sampling method with a 95% confidence level and a 5% tolerable error rate, the statistical table requires a sample of 59 employees. The auditor selects every 20th employee from the alphabetical roster. If the judgmental sample reveals a missing training record, it indicates a localized nonconformity that must be investigated. If the statistical sample reveals 6 missing records (exceeding the tolerable rate), the auditor can mathematically conclude that there is a systemic breakdown in the training record-keeping process across the entire organization.

Comparison of Sampling Methodologies

The table below compares Judgmental and Statistical sampling across key criteria: