5.3 Indicators, Measures, Reliability, and Validity

From Objectives to Indicators

An indicator is the observable sign that an objective has been met or that a process occurred. If an objective says participants will demonstrate correct inhaler technique, the indicator cannot be general satisfaction. It should be a checklist score, demonstration rating, or similar evidence. The exam often tests whether you can choose the indicator closest to the objective.

Indicators may be quantitative or qualitative. Quantitative indicators include counts, percentages, means, rates, and scores. Qualitative indicators include themes from interviews, observations, open-ended comments, or meeting notes. Neither type is automatically better. The best indicator answers the evaluation question with enough accuracy for the decision.

Process indicators may include recruitment contacts, attendance, completion, facilitator adherence, participant dose, materials distributed, referrals made, and partner contributions. Outcome indicators may include knowledge score, skill demonstration, intention, self-efficacy, behavior frequency, service use, or policy adoption. Impact indicators may include injury rates, disease incidence, absenteeism, or community-level conditions.

Reliability is consistency. A reliable scale gives similar results when the underlying trait has not changed. Interrater reliability matters when observers score the same behavior. Test-retest reliability matters when a stable concept is measured over time. Internal consistency matters when multiple items are intended to measure the same construct.

Validity is about meaning. Content validity asks whether the measure covers the full concept. Criterion validity asks whether scores relate to an accepted standard. Construct validity asks whether the measure behaves as expected in relation to theory and other variables. Face validity asks whether the measure appears appropriate to respondents and stakeholders, but appearance alone is not enough.

Operational definitions make measurement concrete. Instead of saying "participation," define whether participation means attending one session, attending four of six sessions, completing homework, or actively practicing a skill. Instead of saying "healthy food access," define distance, price, store type, hours, or availability of specific items.

Measurement must fit the priority population. A validated English instrument may not be valid after informal translation. A long survey may be unrealistic in a busy clinic. A digital survey may exclude people without reliable internet. The CHES-level judgment is to preserve measurement quality while respecting literacy, language, disability access, culture, burden, and confidentiality.

In applied questions, scan the verb in the objective. Words such as list, identify, demonstrate, attend, refer, adopt, and reduce point to different indicators. Then ask whether the measure would produce the same meaning across staff, sites, languages, and time points. A measure that is convenient but poorly matched can make an effective program look weak or make an ineffective program look successful.

Also consider burden. A ten-item validated tool may be better than a forty-item tool if the shorter version still measures the construct well and participants are completing it during a brief session. Measurement quality includes fit with the setting, not only psychometric strength on paper.

Scenario Review Checklist

• Identify the relevant CHES Area of Responsibility.
• Locate the program stage in the scenario.
• Match the answer to evidence, stakeholders, and ethics.
• Reject choices that are premature, unsupported, or outside scope.