12.3 Audit Tools and Techniques
Key Takeaways
- Working papers are everything the auditor prepares or uses -- checklists, forms, sampling plans, flowcharts, extracts of criteria, and contemporaneous notes -- and they are retained as part of the audit record and treated as confidential.
- A checklist is built from the criteria during preparation and uses open prompts with space to record which records, serial numbers, and gage IDs were examined; it guarantees coverage but must never stop the auditor from following unexpected evidence.
- Objective evidence is verifiable evidence another competent person could reproduce, so a finding records which record, which serial number, and which date -- and the strongest findings corroborate an interview, a physical observation, and a document that disagree with each other.
- Forward tracing starts at an input such as a material heat or an engineering change and follows it downstream to define the exposure, while backward tracing starts at a shipped unit or complaint and reconstructs its history upstream to test traceability; containment investigations need both.
- Audit sampling gathers evidence to support a conclusion about the system and is usually judgment-based, which is fundamentally different from acceptance sampling under ANSI/ASQ Z1.4 that makes a lot accept-or-reject decision with defined statistical risks.
12.3 Audit Tools and Techniques
Why the Tools Are Examined Separately
An auditor's conclusions are only as good as the evidence behind them, and the evidence is only as good as the tools used to gather it. The CQT Body of Knowledge calls out four specific tool sets: checklists and working papers, data gathering and objective evidence, forward and backward tracing, and audit sampling plans and procedural guidelines. A technician is far more likely to be asked to build a checklist, run a trace, or defend an audit sample than to lead a registrar audit.
Checklists and Working Papers
What Counts as a Working Paper
Working papers are all the documents an auditor prepares or uses to plan, perform, and record the audit. They include:
- Checklists and question sets
- Blank forms for recording observations, sample identification, and findings
- Sampling plans and the record of what was actually sampled
- Process flowcharts, turtle diagrams, and organization charts prepared for the audit
- Copies or extracts of criteria documents (procedures, drawings, control plans)
- The auditor's contemporaneous notes
- Previous audit reports and the status of open corrective actions
Working papers are retained as part of the audit record for the period defined in the audit programme, and they are the auditor's defense when a finding is challenged. They also carry confidentiality obligations: they routinely contain customer part numbers, process parameters, and personnel information, and they are not shared outside the audit team without authorization.
Building a Useful Checklist
A checklist is built from the criteria during preparation, working clause by clause or process step by process step.
| Good checklist practice | Why |
|---|---|
| Derive each line from a specific requirement and cite it | The line already contains the requirement half of a finding |
| Use open prompts ("Show me how...", "Walk me through...") rather than yes/no questions | Yes/no questions get yes/no answers and no evidence |
| Leave space to record what was examined (record IDs, serial numbers, dates) | Traceable evidence survives a challenge |
| Include the sample size intended for each line | Makes the audit reproducible and defensible |
| Sequence lines to follow the physical process flow | Minimizes walking and lets the auditor follow the product |
| Checklist failure mode | Consequence |
|---|---|
| Treating the checklist as a script | The auditor stops following interesting evidence and misses the real problem |
| Generic checklists copied between unrelated processes | Questions do not match the actual criteria; findings cannot be written |
| Recording only pass/fail marks | No evidence trail; the finding cannot be defended or reproduced |
| Never deviating from the checklist | The checklist becomes the ceiling of the audit rather than its floor |
[!IMPORTANT] A checklist is a memory aid, not a contract. It guarantees coverage of the criteria and keeps the auditor on schedule, but when evidence points somewhere unexpected the auditor follows the evidence and records the departure. An auditor who finds an uncalibrated gage and ignores it because "calibration is not on today's checklist" has failed the audit.
Data Gathering and Objective Evidence
The Four Sources of Evidence
| Source | What it proves | Characteristic weakness |
|---|---|---|
| Interviews | What people believe the process is, and whether they are competent in it | People describe the intended process, not necessarily the actual one; nervousness distorts answers |
| Observation of activity | What actually happens at the workstation right now | The observed behaviour may not be typical (the auditor is watching) |
| Examination of records and documents | What happened in the past and whether the system generated evidence | A record proves a form was filled in, not that the work was done |
| Re-performance / measurement | Whether the reported result is reproducible | Consumes time; may require a calibrated instrument and the auditor's competence |
What Makes Evidence "Objective"
Objective evidence is verifiable: another competent person, given the same identifiers, could find the same thing and reach the same conclusion. That means recording which record, which serial number, which gage ID, on which date.
- "Training records are incomplete" — not objective evidence.
- "Training records for operators J. Ruiz, T. Okafor, and M. Lindqvist show no sign-off for work instruction WI-40 Rev C, which was released 2026-06-01; all three performed operation 40 on job 55120 on 2026-09-12" — objective evidence.
Corroboration
Experienced auditors never rest a finding on a single source. The strongest evidence combines an interview statement, a physical observation, and a documented record that agree — or, more usefully, that disagree, which is where real findings live. If an operator says the first piece is always bought off by quality, the traveler shows no buy-off stamp, and the auditor watches a run start without a buy-off, the three sources together make the finding unarguable.
Forward and Backward Tracing
Tracing is the most powerful technique available to a technician-level auditor, because it tests the whole chain instead of one clause.
| Forward tracing (trace forward) | Backward tracing (trace back) | |
|---|---|---|
| Start point | An input: a raw-material lot, a receiving inspection record, a purchase order, a drawing revision | An output: a shipped unit, a finished-goods serial number, a customer complaint, a finished lot |
| Direction | Downstream, through each operation, to the finished and shipped product | Upstream, through each operation, back to raw material and its certifications |
| Question it answers | "Where did all of this material go, and was it controlled at every step?" | "Where did this specific unit come from, and can every requirement be evidenced?" |
| Typical use | Verifying that a nonconforming or suspect lot was fully contained; verifying that an engineering change reached every downstream operation | Verifying traceability and record completeness; investigating a field failure or customer complaint |
| What a failure reveals | Material escaped containment, or a change did not propagate | A gap in the traceability chain: a missing certification, an unrecorded operation, an unlisted gage |
Choosing the Direction
- Use backward tracing when you have a symptom and need the history: a customer return, a field failure, an out-of-calibration gage whose past measurements are suspect (Section 8.4 is a backward trace).
- Use forward tracing when you have a cause and need the exposure: a bad heat of material, a broken tool discovered at 14:00, a drawing revision that may not have reached the shop floor.
- A recall or containment investigation normally needs both: trace back from the failed unit to find the common input, then trace forward from that input to find every other unit affected.
Vertical and Horizontal Audits
Two related sampling strategies:
- A vertical audit takes one product, order, or job and audits every requirement applied to it, from contract review through shipment. It is the natural home of tracing.
- A horizontal audit takes one requirement — calibration, training, document control — and audits it across every department. It is the natural way to detect that five minors are really one major.
Audit Sampling Plans and Procedural Guidelines
Audit Sampling Is Not Lot-Acceptance Sampling
This is the distinction most often missed. An acceptance sampling plan (Section 11.2) is designed to make an accept/reject decision about a lot with known statistical risks. Audit sampling exists to gather enough evidence to conclude whether the system is working, and the audit conclusion is always accompanied by a statement that the audit was based on samples and therefore cannot guarantee that all nonconformities were detected.
| Acceptance sampling (Z1.4) | Audit sampling | |
|---|---|---|
| Object | A lot of product | A population of records, transactions, items, or activities |
| Purpose | Accept or reject the lot | Support a conclusion about system conformity and effectiveness |
| Sample size basis | Lot size, inspection level, AQL, from tables | Risk, prior performance, population size, available time, and auditor judgment |
| Output | Accept or reject | Findings with evidence, plus a conclusion |
Judgment Versus Statistical Audit Sampling
- Judgment-based (non-statistical) sampling relies on the auditor's knowledge to select the items most likely to reveal a weakness: the newest operator, the oldest open corrective action, the highest-risk part number, the shift with the least supervision. It is the normal method for a one-day internal audit. It is deliberately not random, so it cannot support a statistical inference about the population.
- Statistical sampling selects items at random with a defined sample size so the result can be projected to the population with stated confidence. It is used when a conclusion about the population must be defensible, for example when verifying containment of a suspect lot or demonstrating the effectiveness of a corrective action.
Practical Rules for Audit Sampling
- Decide the sample before you look. Choosing records after glancing at them invites bias in both directions.
- Record the population and the sample. "Ten of 240 travelers for June, selected by every 24th record" is reproducible. "A few travelers" is not.
- Expand the sample when you find something. One overdue gage in a sample of five is a reason to look at twenty, not a reason to stop. The size of the expanded sample often decides whether the finding is graded minor or major.
- Cover the edges of the population. Sample across shifts, operators, machines, and time. A sample drawn entirely from day shift proves nothing about night shift.
- Never sample only what the auditee offers. The escort who selects the records has selected the audit result.
Procedural Guidelines and Auditor Competence
The audit itself is governed by documented procedures: an audit procedure defining how audits are planned, conducted, reported, and closed; criteria for auditor selection, training, and competence evaluation; and the independence rule that an auditor may not audit their own work. ISO 19011 provides the guidance for auditing management systems, including auditor competence and evaluation, while certification bodies performing third-party audits operate under their own accreditation requirements. For an internal programme, the two questions an auditor must be able to answer are: how was I qualified to audit this? and how is my independence from this area established?
Worked Example: Designing the Trace
Situation: A customer returns a hydraulic fitting that failed a proof-pressure test at their assembly plant. The fitting carries lot code L-2608-14.
Step 1 — Backward trace from the returned part. Pull the job traveler for L-2608-14. Confirm each operation sign-off, the gages used at each operation and their calibration status on those dates, the material certification for the bar stock heat, the heat-treat lot and its furnace chart, and the final inspection record. Suppose the trace shows the bar stock came from heat H-71104, and the material certification on file reports a yield strength below the drawing minimum.
Step 2 — Forward trace from the suspect input. Now trace forward from heat H-71104: which bar lengths were issued, to which jobs, on which dates, producing which lot codes, shipped to which customers on which packing slips. This defines the containment population — the parts that must be recalled, sorted, or re-tested.
Step 3 — Close the loop. The backward trace found the cause; the forward trace found the exposure. Neither alone is sufficient: a backward trace tells you why one part failed, and a forward trace tells you how many more are out there.
Step 4 — Audit the control that failed. The real finding is usually not the bad heat; it is that receiving inspection accepted a certification whose reported value was below the drawing requirement. That is a horizontal audit question: how many other certifications were accepted without the values being checked?
Common Exam Traps for CQT Candidates
[!CAUTION] Trap 1: Treating the checklist as the whole audit. Coverage is the floor, not the ceiling; the auditor follows evidence wherever it leads.
Trap 2: Confusing the trace directions. Forward tracing starts at an input and finds the exposure downstream; backward tracing starts at an output and reconstructs its history upstream.
Trap 3: Applying Z1.4 tables to audit sampling. Acceptance sampling makes a lot decision with defined risks; audit sampling gathers evidence for a system conclusion, and audit reports must state the sampling-based uncertainty.
Trap 4: Accepting a sample chosen by the auditee. Whoever selects the records selects the result.
Trap 5: Recording conclusions instead of evidence. "Records incomplete" cannot be defended; record identifiers, dates, and serial numbers can.
Trap 6: Not expanding the sample after a hit. The expanded sample is what distinguishes an isolated minor from a systemic major.
A quality technician discovers at 14:00 that a broken locating pin in a stamping die has been producing mislocated holes, and cannot determine when the pin broke. Which tracing direction is required to define how much product must be contained, and what does that trace follow?
An auditor plans to examine training records in a department of 240 employees. The escort offers to pull "a good representative set" of files while the auditor reviews procedures. What is wrong with this arrangement, and what is the correct approach?
Which of the following is the strongest piece of objective evidence supporting a finding that a control plan requirement was not implemented?