16.3 Appraising Current Research: Study Designs, Bias, Statistics & Landmark Renal Trials
Key Takeaways
Randomized controlled trials test cause and effect; cohort and case-control studies show associations that can reflect confounding or reverse causation.
A ratio (relative risk, odds ratio or hazard ratio) is statistically significant at the 95% level only when its confidence interval excludes 1.0.
The number needed to treat is 1 divided by the absolute risk reduction: a fall in event rate from 20% to 15% gives an ARR of 5% and an NNT of 20.
The HEMO trial found no survival benefit from a higher hemodialysis dose or high-flux membranes, and ADEMEX found no survival benefit from increasing PD small-solute clearance.
CHOIR and TREAT showed that targeting normal hemoglobin with ESAs raised cardiovascular risk (TREAT: stroke hazard ratio 1.92), which shaped the current Hb targets.
Why research appraisal is on the exam
The CSR outline's Domain 3 includes current research, clinical practices and protocols. A specialist must read new evidence critically, explain it to patients and colleagues, and decide whether it should change practice. Exam items may give a study summary and ask what the results show, or ask why a guideline grades a statement as weak.
The evidence hierarchy
From strongest to weakest for questions about treatment effects:
- Systematic reviews and meta-analyses of randomized controlled trials (RCTs)
- Individual RCTs
- Cohort studies, prospective or retrospective
- Case-control studies
- Cross-sectional studies (a snapshot of exposure and outcome together)
- Case series and case reports
- Expert opinion, including guideline OPINION statements and practice points
GRADE starts RCT evidence at "high" and observational evidence at "low." It then downgrades for risk of bias, inconsistency, indirectness, imprecision or publication bias, and can upgrade observational evidence for large effects or a dose-response. This is why so many nutrition statements in KDOQI 2020 are graded C or D, or given as OPINION.
Designs and what they can show
| Design | Strength | Typical limitation in renal nutrition |
|---|---|---|
| RCT | Randomization balances known and unknown confounders, so it can show causation | Short, small nutrition trials; poor adherence; hard to blind diet |
| Cohort | Follows exposed and unexposed people over time; good for rare exposures and many outcomes | Confounding and reverse causation |
| Case-control | Efficient for rare outcomes | Recall bias; choice of controls |
| Cross-sectional | Quick prevalence estimates | Cannot tell which came first |
Diagnostic accuracy studies compare a tool, such as a PEW screen, with a reference standard. They report sensitivity (the share of true cases the tool detects), specificity (the share of non-cases it correctly clears) and predictive values. A highly sensitive screen is best for ruling a condition out.
Bias and confounding in nephrology nutrition research
- Confounding: a third factor linked to both exposure and outcome. Sicker patients eat less and die sooner, so low intake looks lethal even if illness is the true cause.
- Reverse causation: the outcome process causes the "exposure." The obesity paradox in hemodialysis (higher BMI linked to lower mortality) may partly reflect wasting from illness lowering BMI before death. It is an observational association, and KDOQI 2020 accordingly grades the BMI-mortality statements as prognostic, not as a reason to promote weight gain.
- Selection and survivor bias: prevalent dialysis cohorts include only people who survived long enough to be counted.
- Measurement bias: albumin assays (BCG versus BCP) differ; food records underreport; nPCR reflects catabolism, not intake, in unstable patients.
- Attrition bias: dropouts in diet trials are often the sickest patients.
- Intention-to-treat versus per-protocol: intention-to-treat analysis keeps everyone in their randomized group and protects randomization. Per-protocol analysis can overstate benefit.
Reading the numbers
- Relative risk (RR) or hazard ratio (HR): below 1.0 means lower risk in the intervention group and above 1.0 means higher risk. An HR describes risk over time (from survival analysis).
- Odds ratio (OR): common in case-control studies. It approximates RR when outcomes are rare.
- 95% confidence interval (CI): the range of plausible true values. For ratios, a CI that includes 1.0 is not statistically significant. For differences, a CI that includes 0 is not significant.
- p-value: the probability of results at least this extreme if there were truly no effect. p < 0.05 is conventional, but it does not measure how large or important the effect is.
- Absolute risk reduction (ARR): control event rate minus intervention event rate.
- Number needed to treat (NNT): (with the ARR written as a proportion).
Worked example. In a hypothetical trial, 20% of control patients and 15% of treated patients were hospitalized in a year.
Twenty patients must be treated for a year to prevent one hospitalization. The relative risk reduction (25%) sounds larger than the absolute benefit (5 percentage points), so always look for both.
Statistical versus clinical significance. A large trial can find a statistically significant 0.1 g/dL albumin change that makes no clinical difference. A small trial can miss a real benefit because it is underpowered (a type II error).
Landmark trials that shaped renal nutrition and dialysis practice
| Trial (year) | Question | Main finding |
|---|---|---|
| MDRD (1994) | Protein restriction and BP targets in non-diabetic CKD | The primary analysis did not show a significant slowing of GFR decline; later analyses and meta-analyses informed today's supervised low-protein recommendations |
| HEMO (2002) | Higher versus standard HD dose, and high- versus low-flux membranes | No mortality benefit from either |
| ADEMEX (2002) | Higher peritoneal small-solute clearance | No survival benefit; this supported the weekly Kt/V minimum of 1.7 |
| CHOIR (2006), CREATE (2006), TREAT (2009) | Normal versus lower hemoglobin targets with ESAs | Higher targets gave no benefit and more harm; TREAT showed about double the stroke risk (HR 1.92) with darbepoetin targeting 13 g/dL |
| PIVOTAL (2019) | Proactive high-dose versus reactive low-dose IV iron in HD | Proactive iron (held at ferritin above 700 ng/mL or TSAT of 40% or more) was non-inferior for major cardiovascular events and death, lowered ESA doses and reduced transfusions |
| DAPA-CKD (2020), EMPA-KIDNEY (2023) | SGLT2 inhibitors in CKD | Slowed kidney disease progression across diabetic and non-diabetic CKD |
| FLOW (2024) | Semaglutide in type 2 diabetes with CKD | Fewer major kidney outcome events |
Two lessons run through these trials. First, intuitive biomarker targets can mislead: pushing hemoglobin, Kt/V or small-solute clearance higher did not improve survival. Second, observational associations need trial testing before they become practice.
Evaluating a new article: a quick checklist
- Question: population, intervention, comparison, outcome (PICO). Is it relevant to your patients?
- Design: is it the right design for the question, and was allocation randomized and concealed?
- Bias: blinding, follow-up completeness, intention-to-treat, funding and conflicts of interest.
- Results: effect size with CI, absolute as well as relative effects, and patient-important outcomes (death, hospitalization, quality of life) rather than surrogates alone.
- Applicability: do the participants resemble your patients (dialysis modality, diabetes, age)? Are harms and costs acceptable?
- Fit with guidelines: does it confirm, refine or conflict with KDOQI or KDIGO? One study rarely overturns a guideline.
A randomized trial in hemodialysis patients reports that an oral supplement reduced 1-year hospitalization from 30% to 24%. What are the absolute risk reduction and the number needed to treat?
ARR 20% and NNT 5.
ARR 6% and NNT about 17.
ARR 0.8 and NNT 1.25.
ARR 24% and NNT about 4.
An observational cohort of 10,000 hemodialysis patients finds that a BMI of 30 or more is associated with lower mortality (hazard ratio 0.82; 95% CI 0.74–0.91). Which conclusion is most appropriate?
The association is statistically significant, but the design cannot prove that raising BMI lowers mortality, because confounding and reverse causation (illness-related weight loss before death) may explain it.
The result proves that dietitians should encourage weight gain to BMI 30 in every dialysis patient.
The association is not significant, because the confidence interval includes 1.0.
The study is a randomized trial, so the hazard ratio shows causation.
Why do the HEMO and ADEMEX trials matter for how renal dietitians interpret dialysis adequacy targets?
They showed that doubling the dialysis dose markedly improved nutrition status and survival.
They proved that nPCR is unrelated to dialysis delivery in all patients.
They showed that raising small-solute clearance above standard minimums (higher HD Kt/V, or PD clearance above about 1.7 weekly) did not improve survival, so meeting adequacy minimums matters but pushing them higher is not a nutrition cure.
They established that high-flux dialyzers double survival compared with low-flux dialyzers.
Sections you finish are checked off in the contents.