6.3 Quality Control Plans & Audit Techniques (CoC Standard 6.1)
Key Takeaways
The Standard 6.1 protocol sets review criteria, a timetable, methods, sources, reviewers, benchmarks, and how quality control activity and conflict resolution are documented.
Casefinding, abstracting timeliness and the percentage of items coded unknown are evaluated for all cases each year.
Accuracy review covers at least 10% of the annual analytic caseload, up to 200 cases, for class of case, primary site, histology, grade, AJCC or other stage, first-course treatment and four recurrence/cancer status items.
Reviewers may be ODSs, advanced practice nurses, physician assistants, physicians, fellows or residents, but an ODS may not review their own cases.
The program sets its own benchmarks; the audit results, recommendations and outcomes must be presented to the cancer committee in the same calendar year and recorded in the minutes.
Quality control versus quality improvement
Quality control (QC) checks the data: are cases complete, timely and correctly coded? Quality improvement (Section 5.3) uses the data to improve care. The exam tests both, and Domain 2 lists "quality control plan elements, activities, and methods" and "casefinding and quality assurance audit techniques" as separate topics. Registry QC for CoC-accredited programs is governed by Standard 6.1, Cancer Registry Quality Control.
Standard 6.1: the quality control protocol
Each calendar year the cancer committee implements a protocol to evaluate the quality of registry data and activity. The Cancer Registry Quality Coordinator, working with registry staff and other departments, carries it out, recommends corrective action when an area falls below the protocol's measures, and presents the results to the committee.
| Protocol element | What it specifies |
|---|---|
| A. Review criteria | What counts as an error and how discrepancies are resolved |
| B. Timetable | When each QC activity happens during the year |
| C. Methods, sources and individuals | Random sampling of the annual analytic caseload; review by designated people (ODSs, advanced practice registered nurses, physician assistants, physicians, fellows or residents), with ODSs never reviewing their own cases; external audits, such as state or central registry casefinding audits, may fulfill part of the requirement |
| D. Activities evaluated for all cases each year | 1. Casefinding; 2. Abstracting timeliness; 3. The percentage of information coded as unknown (usually 9 or a string of 9s) |
| E. Accuracy review of abstracted data | A minimum of 10% of the annual analytic caseload, up to 200 cases, for: 1. Class of case; 2. Primary site; 3. Histology; 4. Grade; 5. AJCC stage or another appropriate staging system; 6. First course of treatment; 7. Follow-up items: date of first recurrence, type of first recurrence, cancer status, and date of last cancer status |
| F. Benchmarks | The minimum quality benchmarks and required accuracy, plus the requirement that data submitted to the NCDB meet its quality and timeliness criteria |
| G. Documentation | Review criteria, cases reviewed, errors found and their resolutions, and the percentage of accuracy for the items in D and E, reported to the committee each year |
Reporting rules: the results, recommendations and outcomes of recommendations are presented to the cancer committee during the current calendar year and recorded in the minutes. The protocol, including the process for resolving conflicts found in review, is submitted with the Pre-Review Questionnaire. Patient data reviewed under Standard 6.1 cannot be reused as the physician's in-depth study for Standard 7.2.
Worked example: sample size
| Annual analytic caseload | 10% | Minimum accuracy review |
|---|---|---|
| 450 | 45 | 45 cases |
| 1,600 | 160 | 160 cases |
| 2,000 | 200 | 200 cases |
| 4,200 | 420 | 200 cases (the cap) |
Casefinding audits (completeness)
A casefinding audit checks whether every reportable case was found.
- Choose independent sources, such as the pathology log, HIM disease index (ICD-10-CM codes), radiation oncology and infusion logs, and outpatient procedure logs, for a defined period.
- Match each reportable case from those sources against the registry accession list and suspense file.
- Investigate every miss. Was it truly reportable? Was the source screened? Was the case wrongly ruled out?
- Calculate completeness: cases found in the registry ÷ reportable cases found in the audit sources × 100.
- Fix the process, not only the missing cases. Add a source, change the ICD-10-CM code list, or retrain staff.
Central registries run the same kind of audit at facilities, and NPCR and SEER measure a central registry's completeness against expected counts. A state casefinding audit of the hospital can count toward part of the hospital's Standard 6.1 requirement.
Timeliness monitoring
CoC leaves the timeliness benchmark to the program's protocol, alongside the real deadlines: monthly RCRS submission (Standard 6.4) and the state reporting deadline. STORE's Date Case Completed – CoC [2092] is auto-calculated when the required items pass the "CoC Required – All" edit set, and STORE says it may be used for timeliness under Standard 6.1. Typical measures include the percentage of cases abstracted within the protocol's target interval from Date of First Contact, and an aging report of cases in suspense.
Accuracy (reabstracting) audits
| Technique | How it works | Strength |
|---|---|---|
| Reabstracting audit | A second qualified reviewer abstracts the case again from the source documents without seeing the original codes, then compares | Most rigorous estimate of coding accuracy |
| Visual editing (text-to-code review) | The reviewer checks whether each code is supported by the abstract's text fields and source documents | Fast; catches unsupported codes |
| Recoding audit | Reviewers code from a standardized set of records or text to test agreement between coders | Measures reliability between staff |
| Computerized edits | EDITS metafiles test code validity and logical consistency (Section 16.4) | Covers 100% of cases, but cannot detect a valid code that is wrong |
| Targeted review | Deliberately selects high-risk cases, such as new staff, new rules, complex sites or recent standard changes | Finds known weak spots, but does not estimate the overall error rate |
Standard 6.1 requires random sampling for the accuracy review. Programs may add targeted reviews on top of it.
Measuring accuracy
- Item accuracy = items coded correctly ÷ items reviewed × 100. For example, 25 cases × 10 items = 250 items. If 6 are wrong, accuracy is 244/250 = 97.6%.
- Case-level accuracy = cases with no errors ÷ cases reviewed. Some programs classify errors as major, where the error changes stage, primary site, histology or treatment category, or minor, and set separate benchmarks.
- Unknown-value rate = cases with an item coded unknown ÷ cases × 100, tracked for key items such as stage and treatment.
Closing the loop
For each QC finding: correct the records (and resubmit them to RCRS and the state), look for the cause, retrain or change procedures, re-audit the affected area, and report the results, recommendations and outcomes to the committee in the same year.
A CoC-accredited academic program has an annual analytic caseload of 4,200 cases. What is the minimum number of cases its Standard 6.1 accuracy review must cover?
42 cases
420 cases
300 cases
200 cases
Which reviewer arrangement violates CoC Standard 6.1?
A physician assistant reviews a random sample of breast cases for staging accuracy.
A state central registry casefinding audit is counted toward part of the program's requirement.
Each ODS on staff reviews a random sample of the cases they personally abstracted.
A surgical resident reviews a sample of colon cases for first-course treatment accuracy.
Which activities does Standard 6.1 require to be evaluated for all cases each year, rather than only for the random accuracy sample?
Primary site, histology and grade
AJCC stage and first course of treatment
Date of first recurrence and cancer status
Casefinding, abstracting timeliness and the percentage of information coded as unknown
Sections you finish are checked off in the contents.