11.4 Critical Appraisal, Bias and Research Governance
Key Takeaways
- Bias is systematic error that is not reduced by increasing sample size, whereas random error is.
- Confounding is distortion by a factor associated with both exposure and outcome, and only randomisation controls unknown confounders.
- CONSORT applies to randomised trials, PRISMA to systematic reviews and STROBE to observational studies.
- A fall in risk from 10% to 5% is a 50% relative risk reduction but only a 5 percentage point absolute reduction, giving a number needed to treat of 20.
- Research involving NHS patients, their tissue or their data requires Research Ethics Committee approval through the Health Research Authority, and stored human tissue is governed by the Human Tissue Act 2004.
Bias: Systematic Error
Bias is systematic error that pushes results consistently in one direction. Unlike random error, it is not reduced by increasing the sample size.
| Bias | Mechanism | Dental example |
|---|---|---|
| Selection bias | Groups differ systematically at entry | Recruiting the treatment arm from a specialist clinic and controls from general practice |
| Recall bias | Cases remember exposures more thoroughly than controls | Patients with oral cancer recalling tobacco and alcohol use more completely than healthy controls |
| Observer (detection) bias | Assessor knows the allocation | An unblinded examiner scoring plaque more favourably in the new toothbrush group |
| Performance bias | Groups receive different care apart from the intervention | The trial group also receiving longer oral hygiene instruction |
| Attrition bias | Differential loss to follow-up | Patients whose implants failed not returning for the five-year review |
| Hawthorne effect | Participants change behaviour because they are observed | Plaque scores improving in both arms of any toothbrushing trial |
| Publication bias | Positive studies preferentially published | Meta-analyses overestimating the effect of a material |
Confounding is different from bias. A confounder is an independent factor associated with both the exposure and the outcome, distorting the apparent relationship. The classic dental example is that alcohol appears to cause periodontitis until smoking is taken into account. Confounding is addressed by randomisation, restriction, matching, stratification or multivariable adjustment. Only randomisation controls unknown confounders.
Structured Appraisal
Appraisal is usually taught through a structured framework.
- PICO frames the question: Population, Intervention, Comparison, Outcome.
- CASP checklists and the Cochrane risk of bias tool provide domain-by-domain assessment.
- Reporting guidelines define what a paper must contain: CONSORT for randomised trials, PRISMA for systematic reviews, STROBE for observational studies.
Three questions do most of the work: Are the results valid? What are the results? Will they help my patient? The last is external validity or generalisability — a trial conducted in a university hospital on ASA I volunteers may not transfer to an elderly, medically complex NHS practice population.
Absolute and Relative Effect Measures
Relative measures exaggerate the apparent benefit of an intervention when the baseline risk is low.
If a preventive regimen reduces the risk of a new carious lesion from 10% to 5%, the relative risk reduction is 50% — which sounds dramatic — but the absolute risk reduction is 5 percentage points and the number needed to treat is 20. Twenty children must receive the regimen for one to avoid a lesion. Quoting only the relative figure is the commonest way in which marketing material overstates a benefit.
Research Governance and Ethics
UK dental research operates under a defined governance framework:
- Research Ethics Committee (REC) approval through the Health Research Authority is required for research involving NHS patients, their tissue or their data.
- Informed consent must be voluntary, informed and given by a participant with capacity, with a right to withdraw at any time without affecting their care. Research consent is separate from clinical consent.
- The Human Tissue Act 2004 governs storage and use of human tissue, including extracted teeth held for research.
- UK GDPR and the Data Protection Act 2018 govern personal data; research data should be pseudonymised or anonymised wherever possible.
- Clinical trial registration before recruitment is now expected, which is itself a measure against publication bias and selective outcome reporting.
The ethical foundations are the Declaration of Helsinki and the four principles of biomedical ethics: autonomy, beneficence, non-maleficence and justice. These are the same principles that underpin the consent and professionalism material in Paper B, which is why this chapter is not a detour from clinical practice but part of its foundation.
Exam link. A question that describes a trial where the examiner knew which toothpaste each participant received is testing observer or detection bias, and the correct remedy is blinded outcome assessment, not a larger sample.
Confounding, Effect Modification and Causality
Confounding occurs when a third variable is associated with both the exposure and the outcome and distorts the apparent relationship. The classic dental example is the association between tooth loss and cardiovascular disease: smoking causes both, so an unadjusted analysis overstates any direct link. Confounding is handled at the design stage by randomisation, restriction or matching, and at the analysis stage by stratification or multivariable regression. Effect modification is different: it means the true effect really does differ between subgroups, and it should be reported rather than adjusted away.
Because association is not causation, epidemiologists apply the Bradford Hill considerations — strength, consistency, specificity, temporality, biological gradient, plausibility, coherence, experimental evidence and analogy. Temporality is the only one that is strictly necessary: the exposure must precede the outcome. Candidates are often asked which consideration is essential, and temporality is the answer.
UK Research Governance in Practice
The framework names matter for Paper A. Research involving NHS patients, tissue or data requires approval from a Research Ethics Committee through the Health Research Authority, and NHS site permission. The Human Tissue Act 2004 governs the removal, storage and use of relevant material, with consent as its central principle, overseen by the Human Tissue Authority. The UK Policy Framework for Health and Social Care Research sets the responsibilities of sponsors and investigators. Data are governed by the UK GDPR and the Data Protection Act 2018. Clinical trial registration before recruitment and adherence to reporting guidelines — CONSORT for trials, PRISMA for systematic reviews, STROBE for observational studies — are the accepted defences against selective reporting and publication bias.
A manufacturer advertises that its varnish produces a 50% reduction in new carious lesions. The underlying trial reported that lesion incidence fell from 10% in the control group to 5% in the treated group over two years. Which statement best describes the clinical value of this result?