4.2 Work Sample Tests, Performance Simulations, Assessment Centers & Physical Ability Tests

Work Sample Tests, Performance Simulations, Assessment Centers & Physical Ability Tests

Core Principle: Performance-based selection instruments evaluate candidates by requiring them to perform actual or simulated components of the job. By measuring observable behaviors rather than abstract psychological traits, work samples and assessment centers deliver superior predictive validity and high legal defensibility.

While written job knowledge and cognitive tests evaluate what an applicant knows, performance tests and simulations evaluate what an applicant can actually do. In public sector human resources, performance testing is widely deployed across diverse occupational tiers—from clerical typing and welding work samples to public safety physical agility obstacle courses and executive assessment centers for Police and Fire Chiefs.

---

Work Sample Tests & Performance Simulations

A Work Sample Test is an assessment instrument that requires candidates to execute physical, technical, or administrative tasks that are identical or highly similar to essential duties identified through job analysis.

Psychometric Advantages:

• High Criterion Validity: Meta-analytic research demonstrates work sample tests achieve strong predictive validity ($r \approx 0.54$), rivaling cognitive ability tests.
• Reduced Adverse Impact: Because they focus on specific acquired behavioral skills rather than general cognitive aptitude, work samples typically exhibit substantially lower racial and ethnic adverse impact.
• High Face Validity & Applicant Acceptance: Candidates perceive work samples as highly fair and directly relevant to the target job, reducing civil service examination appeals and litigation.
• Realistic Job Preview (RJP) Effect: Applicants gain firsthand exposure to actual working conditions, encouraging self-selection and reducing early post-hire turnover.

Common Public Sector Work Samples:

• Clerical & Administrative: Timed typing tests, transcription accuracy tests, spreadsheets and database management exercises, in-basket email triage.
• Skilled Crafts & Trades: Pipe threading and soldering for utility workers, blueprint reading and conduit bending for electricians, heavy equipment maneuverability courses for public works operators.
• Technical & Professional: Drafting a formal administrative policy briefing memo, writing code for a municipal GIS system, performing a laboratory water-quality titration.

---

Assessment Centers (AC) in Public Sector Leadership Selection

An Assessment Center (AC) is a standardized evaluation of behavior based on multiple inputs. Several trained observers and techniques are used. Judgments about behaviors are made, in major part, from specifically developed assessment simulations (International Taskforce on Assessment Center Guidelines).

+-----------------------------------------------------------------------------------+
|                         ASSESSMENT CENTER ARCHITECTURE                            |
+-----------------------------------------------------------------------------------+
|  1. IN-BASKET / INBOX EXERCISE:                                                   |
|     Candidate reviews incoming memos, emails, citizen complaints, and budget cuts |
|     • Evaluates: Decisiveness, administrative judgment, delegation, prioritization|
+-----------------------------------------------------------------------------------+
                                         │
                                         ▼
+-----------------------------------------------------------------------------------+
|  2. LEADERLESS GROUP DISCUSSION (LGD):                                            |
|     4 to 6 candidates collaborate to resolve a community crisis or allocate funds |
|     • Evaluates: Collaboration, persuasion, negotiation, interpersonal tact       |
+-----------------------------------------------------------------------------------+
                                         │
                                         ▼
+-----------------------------------------------------------------------------------+
|  3. ROLE-PLAY / CITIZEN SIMULATION:                                               |
|     Candidate interacts with a trained role-player (e.g., irate citizen, employee)|
|     • Evaluates: Conflict de-escalation, emotional intelligence, oral communication|
+-----------------------------------------------------------------------------------+
                                         │
                                         ▼
+-----------------------------------------------------------------------------------+
|  4. TACTICAL / TECHNICAL SIMULATION & ORAL BOARD:                                 |
|     Candidate presents emergency response plan or strategic agency restructuring |
|     • Evaluates: Strategic thinking, technical command, executive presence        |
+-----------------------------------------------------------------------------------+
                                         │
                                         ▼
+-----------------------------------------------------------------------------------+
|  ASSESSOR INTEGRATION & CONSENSUS MEETING:                                        |
|  Trained assessors pool behavioral observations and reach consensus ratings       |
+-----------------------------------------------------------------------------------+

Essential Requirements for a Legitimate Assessment Center:

1. Job Analysis Foundation: Explicit identification of observable behavioral dimensions (e.g., Leadership, Problem Analysis, Decisiveness, Conflict Management, Oral Communication).
2. Multiple Behavioral Simulations: Must include at least two or more distinct job-related simulation exercises.
3. Multiple Trained Assessors: Candidates must be evaluated by two or more trained assessors per exercise; assessors are typically senior public managers or external subject matter experts.
4. Behaviorally Anchored Rating Scales (BARS): Assessors record observable behaviors and evaluate performance against standardized behavioral rubrics rather than global impressions.
5. Assessor Integration: Final dimension and overall scores are determined through systematic consensus discussions or calibrated statistical aggregation models.

---

Physical Ability Testing (PAT) in Public Safety

Physical Ability Tests (PAT) are critical selection instruments for protective service classifications (police officers, firefighters, corrections officers, conservation rangers). Because physical tests frequently result in significant adverse impact against female applicants, they are subject to intense legal scrutiny under Title VII and UGESP.

Legal Precedents & Validation Standards:

• Berkman v. City of New York, 536 F. Supp. 177 (E.D.N.Y. 1982): The court struck down the FDNY's physical exam because it over-emphasized maximum anaerobic strength (carrying dead weight) rather than realistic stamina and pacing required on the fireground, resulting in unlawful disparate impact against women.
• Lanning v. Southeastern Pennsylvania Transportation Authority (SEPTA), 181 F.3d 478 (3d Cir. 1999): The Third Circuit established that a cutoff score on a physical test resulting in adverse impact must measure the minimum qualifications necessary for successful performance of the job, rather than an arbitrary elite athletic standard.

ADA Status of Physical Agility Tests vs. Medical Examinations

Under EEOC ADA Title I enforcement guidance:

• Physical Agility and Physical Fitness Tests ARE NOT Medical Examinations: An agency may administer agility/fitness tests at the pre-offer stage because they measure physical performance and endurance rather than medical conditions.
• Restrictions: The test cannot include physiological monitoring (e.g., heart rate, blood pressure, pulmonary spirometry) during or immediately following the test, as physiological monitoring converts the assessment into a pre-offer medical examination prohibited by 42 U.S.C. § 12112(d).

---

Setting Legally Defensible Cut Scores

Under UGESP Section 5H (29 CFR § 1607.5H), cut scores on selection instruments must be established to reflect the minimum level of competence necessary for acceptable job performance.

Standard Cut-Score Methodologies in Public HR:

1. The Modified Angoff Method: Subject matter experts (SMEs) review each test item or performance task and independently estimate the percentage of "Minimally Competent Candidates" (MCC) who would perform the task correctly. The individual estimates are averaged across SMEs to establish the cut score.
2. The Nedelsky Method: Used for multiple-choice items; SMEs identify the number of distractors a minimally competent candidate could eliminate as incorrect, calculating the guessing probability to set the passing mark.
3. Empirical Benchmarking / Contrast Groups Method: The test is administered to known groups of superior and borderline performers; the cut score is set at the point that maximizes differentiation between acceptable and unacceptable job performance.