4.4 Training & Experience (T&E) Evaluations, Point-Factor Scoring & Supplemental Questionnaires
Key Takeaways
- Training and Experience (T&E) evaluations systematically convert an applicant's education, certifications, and work history into competitive merit scores.
- While traditional Credential Counting exhibits low predictive validity (r ≈ 0.10–0.20), the Behavioral Consistency T&E method achieves substantially higher validity (r ≈ 0.45) by evaluating past major achievements against critical job competencies.
- Point-Factor and Supplemental Questionnaires must be directly derived from job analysis KSAOs, utilize graduated behavioral levels, and incorporate mandatory verification audits to combat applicant over-inflation.
- Civil service rules require transparent scoring rubrics, formal appeal mechanisms, and strict record retention for T&E evaluations under UGESP standards.
Training & Experience (T&E) Evaluations, Point-Factor Scoring & Supplemental Questionnaires
Core Principle: In public sector recruitment, Training and Experience (T&E) evaluations serve as formal competitive examinations. Rather than being passive resume reviews, T&E instruments systematically score applicant qualifications against predetermined, job-related criteria derived directly from formal job analysis.
Public HR departments frequently face high-volume recruitments for professional, administrative, or specialized technical classifications where administering an in-person written test or assessment center is logistically impractical or cost-prohibitive. In these situations, civil service commissions deploy Training and Experience (T&E) Evaluations as the competitive examination of record to rank-order candidates on eligible lists.
Methodologies of T&E Evaluation
Not all T&E evaluations are created equal. Industrial-organizational research demonstrates dramatic differences in psychometric validity across various T&E models.
| T&E Evaluation Method | Operational Mechanics | Psychometric Validity ($r$) | Best Applied In |
|---|---|---|---|
| Holistic Credential Review | Evaluators conduct an unstructured review of resumes/applications against general qualification guidelines. | Low ($r \approx 0.10 - 0.15$) | Basic minimum qualification (MQ) screening only; not defensible as a scored exam |
| Point-Factor / Rating Schedule Method | Explicit point values are awarded for specific increments of accredited education, certifications, and years of specialized experience exceeding MQs. | Low to Moderate ($r \approx 0.15 - 0.22$) | High-volume entry/journey-level civil service lists where experience quantity correlates with basic readiness |
| Grouping / Category Method | Candidates are placed into broad qualification tiers (e.g., Highly Qualified, Well Qualified, Qualified) based on defined experiential thresholds. | Moderate ($r \approx 0.25 - 0.30$) | Category Rating systems (5 U.S.C. § 3319) in federal and state merit hiring |
| Behavioral Consistency Method (Schmidt & Caplan) | Applicants submit detailed narrative descriptions of their top 2–3 past work achievements for specific KSAO competencies; SMEs score narratives using BARS. | High ($r \approx 0.45$) | Senior professional, technical, administrative, and managerial civil service classifications |
| Supplemental Questionnaire / Task-Based Self-Assessment | Applicants rate their frequency and skill level on specific job tasks, supported by mandatory narrative justifications and employer verification audits. | Moderate to High ($r \approx 0.35 - 0.42$ with audit controls) | Modern automated applicant tracking systems (e.g., NeoGov, USAJOBS) |
The Behavioral Consistency Method: Gold Standard in T&E
Developed by Frank Schmidt and colleagues, the Behavioral Consistency T&E Method is recognized as the most psychometrically valid and legally defensible T&E examination technique.
Core Tenets of Behavioral Consistency:
- Competency Identification: Job analysis identifies 4 to 6 critical competencies where past achievement differentiates superior from average performance.
- Achievement Narratives: Applicants complete a structured Achievement Record. For each competency, the applicant describes:
- The specific situation or challenge faced.
- What they personally did (actions taken).
- The verifiable outcome or impact (metrics, cost savings, policy implemented).
- The names and contact details of supervisors who can verify the achievement.
- SME Scoring with Benchmarks: Two independent Subject Matter Experts rate the narratives using detailed scoring rubrics with concrete behavioral benchmarks, achieving high inter-rater reliability.
+-----------------------------------------------------------------------------------+
| POINT-FACTOR SCORING ARCHITECTURE FOR T&E EXAMS |
+-----------------------------------------------------------------------------------+
| MINIMUM QUALIFICATIONS BASELINE (PASS/FAIL HURDLE): |
| Bachelor's Degree + 3 Years Professional HR Experience = BASELINE ELIGIBILITY |
+-----------------------------------------------------------------------------------+
│
▼
+-----------------------------------------------------------------------------------+
| POINT-FACTOR ADD-ONS (MAXIMUM 100 POINTS): |
| |
| 1. RELEVANT EDUCATION BEYOND MQ (Max 20 Points): |
| • Master's Degree in HR / Public Administration / Industrial Relations: 20 pts |
| • Post-Graduate Certificate in Public Management: 10 pts |
| |
| 2. PROFESSIONAL CERTIFICATIONS (Max 15 Points): |
| • PSHRA-CP / PSHRA-SCP / SHRM-CP / SPHR: 15 pts |
| • IPMA-HR / Professional Workplace Investigator Credential: 10 pts |
| |
| 3. SPECIALIZED EXPERIENCE BEYOND MQ (Max 45 Points): |
| • 1-2 additional years lead/supervisory HR experience: 15 pts |
| • 3-4 additional years lead/supervisory HR experience: 30 pts |
| • 5+ additional years lead/supervisory HR experience: 45 pts |
| |
| 4. SUPPLEMENTAL TECHNICAL COMPETENCY QUESTIONNAIRE (Max 20 Points): |
| • SME-scored technical writing / labor contract analysis sample: 20 pts |
+-----------------------------------------------------------------------------------+
│
▼
+-----------------------------------------------------------------------------------+
| TOTAL RAW T&E SCORE (Scaled to 70.00 - 100.00 Passing Score on Register) |
+-----------------------------------------------------------------------------------+
Designing Defensible Supplemental Questionnaires
Automated Supplemental Questionnaires are widely utilized in public sector Applicant Tracking Systems (ATS). To maintain merit integrity and withstand legal challenge, they must be constructed with rigorous psychometric safeguards.
Best Practices for Questionnaire Design:
- Graduated Behavioral Options: Avoid binary Yes/No questions. Use 4- or 5-level behavioral frequency/proficiency scales:
- Level A: I have no training or experience in this task.
- Level B: I have completed educational or classroom training in this task, but have not performed it on the job.
- Level C: I have performed this task under direct supervision or guidance.
- Level D: I have performed this task independently as a regular, core duty of my job.
- Level E: I have served as a lead worker, expert, or supervisor, training others in this task.
- Mandatory Narrative Substantiation: For every response at Level C, D, or E, the applicant must provide a text narrative identifying: (a) the employer where the experience was gained, (b) the specific job title held, and (c) the exact duties performed.
- Applicant Warning & Truth-in-Application Certification: Applicants must sign an explicit certification stating that over-inflation, exaggeration, or misrepresentation will result in immediate disqualification, removal from the eligible list, or termination of employment.
- HR & SME Verification Audits: Prior to certifying the final list, HR analysts or SMEs audit a sample of high-scoring questionnaires against the applicant's official work history and transcripts, adjusting inflated scores downward before finalizing rankings.
Legal Defensibility, Record Retention & Appeals
T&E examinations are subject to the same legal standards under UGESP (29 CFR Part 1607) and state civil service laws as written examinations.
Administrative & Legal Defensibility Rules:
- Job-Relatedness: Points can only be awarded for education, certifications, and experience that have a documented direct linkage to the KSAOs established in the formal job analysis.
- Objective Scoring Keys: The scoring key and point allocations must be established and frozen before applications are opened and inspected, preventing favoritism or tailored scoring.
- Appeal Process: Civil service rules typically grant applicants a formal examination appeal period (e.g., 10 to 14 calendar days from score notification) to challenge their T&E score. The agency must provide the scoring rubric and written explanation of point determinations upon request.
- Document Retention: All applicant submissions, rating sheets, SME notes, and scoring keys must be retained for a minimum of two years (or statutory state retention schedules) for Title VII / EEOC compliance.
Which Training and Experience (T&E) evaluation methodology demonstrates the highest predictive criterion validity (r ≈ 0.45) by having applicants submit detailed narratives of past major accomplishments that are scored by Subject Matter Experts using standardized rubrics?
A public agency implements an online self-rating Supplemental Questionnaire for a Senior Accountant civil service examination. To prevent candidate score over-inflation and ensure merit integrity, what administrative control must the HR department incorporate?
When constructing a Point-Factor T&E Rating Schedule, why is it legally essential that point allocations and scoring keys are finalized and approved BEFORE the recruitment closes and applications are reviewed?
An applicant for a civil service engineering position receives their T&E score notice, which places them in Rank 4. The applicant believes their post-graduate coursework in structural engineering was improperly credited by the scoring panel. What standard civil service administrative remedy is available to the applicant?