Evaluating Coalition and Partnership Effectiveness
Key Takeaways
- Coalition evaluation should use the Community Coalition Action Theory, which links coalition characteristics, processes, and structures to health outcomes through intermediate outputs.
- Process evaluation assesses coalition functioning (membership, meeting quality, shared decision-making); outcome evaluation assesses health and systems change.
- Valid effectiveness indicators include member diversity, member satisfaction, action plan quality, resource sharing, policy or systems change, and population health outcomes.
- Partnership evaluation must include the community's own assessment of whether the partnership is working, not only the lead agency's internal metrics.
- Sustainability of outcomes—not the indefinite survival of the coalition itself—is the ultimate effectiveness criterion.
Quick Answer: Evaluating coalitions and partnerships means assessing both how well the partnership functions (process) and what it actually achieves (outcomes). The Community Coalition Action Theory links coalition characteristics and processes to intermediate outputs and ultimately to population health outcomes, giving evaluators a theory-based map of what to measure at each stage.
Why Coalition Evaluation Is Different
Evaluating a coalition is not the same as evaluating a program. A program evaluation asks 'did the intervention change the outcome?' A coalition evaluation asks two questions at once: 'is the coalition itself functioning as a collaboration?' and 'is the collaboration producing health and systems change?' The NBPHE task 'evaluate the effectiveness of coalitions and partnerships in achieving public health objectives' requires both. Skipping the process side means you cannot explain why a coalition that looked well-resourced failed; skipping the outcome side means you cannot justify continued investment.
The Community Coalition Action Theory (CCAT), developed by Frances Butterfoss and Michelle Kegler, is the dominant theory-based framework for coalition evaluation. CCAT posits that coalition characteristics (leadership, membership diversity, structure), processes (member engagement, shared decision-making, conflict resolution), and resources lead to intermediate outputs (action plans, pooled resources, member empowerment) that produce community and systems change and, ultimately, population health outcomes. The theory gives evaluators a chain of indicators to measure at each link, so weak links can be diagnosed.
Process Evaluation of Coalitions
Process evaluation asks whether the coalition is working as a collaboration. Indicators include:
| Process Indicator | What It Measures | Typical Method |
|---|---|---|
| Member diversity | Whether affected communities and key sectors are represented | Roster analysis vs community demographics |
| Meeting quality | Whether meetings use shared decision-making and surface dissent | Observation, member survey |
| Member satisfaction | Whether partners perceive value in participation | Annual survey, retention rate |
| Shared decision-making | Whether power is actually distributed, not concentrated in the lead agency | Member ratings, decision log review |
| Communication | Whether information flows in all directions, not just top-down | Communication audit, partner interviews |
| Action plan quality | Whether the plan has clear, measurable, theory-linked objectives | Plan review against SMART criteria |
| Resource pooling | Whether partners contribute and share funding, staff, data | Budget and in-kind contribution logs |
These process indicators are leading measures—they signal whether the coalition will be able to deliver before outcomes are observable. A coalition with low member diversity, top-down meetings, and no resource pooling is unlikely to produce sustained change even if its action plan looks strong on paper.
A coalition has an action plan with clear measurable objectives and pooled funding from three partners, but member surveys show that decisions are made by the lead agency with little input from others. Which process indicator most needs attention?
Outcome Evaluation of Coalitions
Outcome evaluation asks whether the coalition produced change. The outcome chain has three levels. Short-term outcomes include increased community capacity, new or strengthened partnerships, and changes in knowledge or awareness. Intermediate outcomes include policy change, systems change, environmental change, and behavior change—such as a new school wellness policy, a smoke-free housing rule, or a clinic referral protocol. Long-term outcomes are population health changes—reduced smoking prevalence, lower asthma hospitalizations, reduced injury rates. Long-term outcomes take years to appear and are influenced by many factors, so the coalition evaluation should not hinge on them alone; intermediate policy and systems changes are the truer signal of coalition effectiveness on a realistic timeline.
A common exam trap is judging a coalition solely on long-term health outcomes when the coalition is only two years old. The appropriate evaluation of a young coalition focuses on process health and short-term and intermediate outcomes—policy and systems change, capacity built, partnerships strengthened. Expecting population health change in 24 months is a measurement error, not a coalition failure.
Community-Validated Evaluation
Partnership evaluation must include the community's own assessment of whether the partnership is working. The lead agency's internal metrics—grant deliverables met, reports submitted—do not answer the effectiveness question for the people the partnership is meant to serve. Community-validated evaluation uses participatory methods: residents review draft findings, name what 'working' means to them, and confirm or challenge the lead agency's interpretation. This is consistent with the CBPR principle that those affected by the work should help evaluate it. A coalition evaluation that reports excellent process metrics but where the community says the partnership was extractive is not an effective partnership, regardless of the dashboard.
Sustainability and the Ultimate Criterion
Sustainability of outcomes—not the indefinite survival of the coalition itself—is the ultimate effectiveness criterion. A coalition that changes a school wellness policy that outlasts the coalition has been effective even if the coalition disbands. A coalition that meets for ten years, hosts well-attended conferences, and produces no policy, systems, or environmental change has not been effective regardless of how long it survived. Evaluators must distinguish coalition sustainability (does the coalition continue?) from outcome sustainability (do the changes the coalition produced continue?). The CPH treats the latter as the truer measure.
Common Evaluation Designs
Coalition evaluations commonly use a mixed-methods, pre-post with comparison design when feasible: quantitative indicators (policy adoption, behavior rates) paired with qualitative interviews and a matched comparison community without the coalition. When a comparison group is not feasible, a pre-post within the coalition's own community with a strong process evaluation is acceptable. The RE-AIM framework—reach, effectiveness, adoption, implementation, maintenance—can also structure coalition evaluation, with maintenance explicitly measuring sustainability. Whatever the design, the evaluation should be planned at coalition formation, not added at the end, so baseline indicators are captured before the coalition begins to act.
A coalition has been operating for 18 months and the funder asks for an effectiveness evaluation. Which outcome level is most appropriate to assess at this stage?
Which statement best captures the distinction between coalition sustainability and outcome sustainability in partnership evaluation?