Quantitative, Qualitative, and Mixed Evaluation Methods and Report Quality

Quick Answer: Quantitative evaluation answers "how many" and "did it work" using numerical data and statistical comparison; qualitative evaluation answers "why and how" through narrative, observation, and interview; mixed methods deliberately integrate both to triangulate findings. NBPHE Domain 4 task 2 asks you to determine when to use and apply each method; task 3 asks you to assess evaluation report quality and utility.

Choosing Between Quantitative and Qualitative Methods

The method you select hinges on the evaluation question. Quantitative methods suit questions about magnitude, frequency, dose-response, and whether outcomes changed — they require sufficiently large samples, validated instruments, and often a comparison group. Qualitative methods suit questions about meaning, context, acceptability, and mechanism — they thrive with small, purposively selected samples and open-ended data collection. A common CPH exam trap is assuming randomized controlled trials are always the gold standard; for a process evaluation of a community coalition, key informant interviews and meeting observation may be far more appropriate and feasible than a trial.

Within quantitative evaluation, common designs include randomized controlled trials, quasi-experimental designs (nonequivalent comparison group, interrupted time series, regression discontinuity), pre-post single group, and cross-sectional surveys. Quasi-experimental designs are used when randomization is impractical or unethical — a frequent reality in community public health. Interrupted time series is strong for policy evaluation, for example assessing a sugar-sweetened beverage tax's effect on consumption across multiple time points before and after implementation. Regression discontinuity is appropriate when a continuous assignment variable determines eligibility (e.g., income threshold for WIC) and you compare outcomes just above and below the cutoff.

Pre-post single-group designs are the weakest quantitative option because they cannot rule out secular trends or maturation effects, yet they remain common in small-scale community pilots where no comparison group is feasible. Cross-sectional surveys establish prevalence at a point in time but cannot establish temporality or causation, making them better suited to needs assessment and descriptive surveillance than to outcome evaluation.

Qualitative approaches include in-depth interviews, focus groups of roughly six to ten participants, participant observation, document analysis, and photovoice. Data saturation — the point at which new data no longer yield new themes — is the standard sample-size concept, not statistical power. Coding may be deductive, using a framework such as the Social Ecological Model, or inductive, as in grounded theory. Rigor in qualitative evaluation is assessed through credibility (prolonged engagement, triangulation, member checking), transferability (thick description), dependability (audit trail), and confirmability (reflexivity) — the Lincoln and Guba criteria that parallel quantitative validity and reliability.

Mixed Methods Evaluation Designs

Mixed methods integrate qualitative and quantitative data to offer triangulation, complementarity, and explanation that single-method designs cannot. The CPH exam expects you to recognize three core mixed-methods designs from the Creswell framework:

• Convergent (concurrent triangulation): collect quantitative and qualitative data simultaneously, then merge them to see whether findings converge or diverge. Use this when you want to validate one data type with the other.
• Explanatory sequential: quantitative first, then qualitative follows to explain the quantitative results. For example, a survey shows low program uptake; interviews explore why eligible clients did not enroll.
• Exploratory sequential: qualitative first to develop an instrument or taxonomy, then quantitative to test it at scale. For example, focus groups develop a culturally adapted dietary survey, which is then fielded to a large representative sample.

Integration is the defining feature of mixed methods — not merely collecting both types of data but deliberately merging them at the analysis or interpretation stage through joint displays, connected matrices, or narrative weaving. A joint display places quantitative and qualitative findings side by side in a single table so the reader can see where they confirm, complement, or contradict each other, making the integration visible rather than buried in prose. Failure to integrate is a common methodological weakness flagged in peer review and in the report-quality assessment that NBPHE task 3 addresses. Failure to integrate is a common methodological weakness flagged in peer review and in the report-quality assessment that NBPHE task 3 addresses.

Assessing Evaluation Report Quality and Utility

NBPHE task 3 requires assessing the quality and utility of evaluation reports. The CDC Framework for Program Evaluation in Public Health articulates four standards: utility, feasibility, propriety, and accuracy. A high-quality report should:

• State the evaluation questions and logic model clearly
• Describe methods so others can assess rigor and replicate
• Address stakeholders' information needs and decision timing (utility)
• Present findings linked to evidence, with limitations explicitly acknowledged
• Use clear visualizations rather than relying solely on dense tables
• Include actionable, prioritized recommendations
• Be delivered in time to inform decisions, not months after the program cycle ends

A report with strong statistical analysis but no connection to stakeholder decision timing fails the utility standard. Conversely, a timely, jargon-free report with weak methods fails the accuracy standard. The Joint Committee on Standards for Educational Evaluation's Program Evaluation Standards (2018 edition) articulates these dimensions in detail across utility, feasibility, propriety, and accuracy clusters.

Common report-quality red flags the exam may test include selective outcome reporting (highlighting only favorable findings), missing comparison group description, conflating correlation with causation, ignoring implementation fidelity data, and issuing recommendations not supported by the data presented. A strong evaluator assesses a report against all four standards, not just statistical sophistication.