15.2 Administering Examinations and Evaluating Competency

15.2 Administering Examinations and Evaluating Competency

Quick Answer: Under employer programs, Level I/II candidates typically pass general, specific, and practical examinations that the Level III develops, approves, and/or administers under controlled conditions. Scores follow Written Practice minima (commonly discussed as ≥70% each and ≥80% composite under SNT-TC-1A-style programs—confirm the current document). Retests, security, and practical specimen design are program-level decisions the Basic exam expects you to reason about.

Section 15.1 built the Written Practice spine. This section is the examination engine: how you create tests that measure competence, run them fairly, grade them defensibly, and decide when someone is ready to be recommended for certification.

Why Examination Administration Is a Level III Core Skill

Domain 1 competencies include developing personnel certification examinations, administering them, and evaluating competency. On the shop floor, weak exams produce two failure modes:

1. False fails — competent people blocked by poorly written or irrelevant questions.
2. False passes — people who can game a leaked bank but cannot set up equipment, interpret real indications, or follow the procedure.

Customer and regulatory audits treat examination control as a quality system issue. A missing answer key log or an unproctored “open office” general exam can invalidate a certification chain even if training hours look complete.

The Three Classic Examination Types (Level I and Level II)

General examination

• Tests method fundamentals: physics principles, equipment types, technique theory, general capabilities and limitations, safety awareness for the method.
• Usually closed-book multiple-choice under controlled conditions.
• Content should align with the body of knowledge (including CP-105 topical structure) for the method and level—not random internet trivia.

Specific examination

• Tests employer/product-specific knowledge: codes, standards, specifications, procedures, acceptance criteria, reporting forms, and product-related discontinuities relevant to the facility’s work.
• Often allows use of referenced codes/procedures if the Written Practice and exam design say so—or remains closed-book if that is the program rule. Consistency matters more than folklore.
• This is where “our nuclear procedure vs. our commercial PT procedure” actually lives.

Practical examination

• Requires the candidate to demonstrate equipment setup/calibration, examination of specimens or hardware, data recording, and evaluation according to instructions or procedures appropriate to the level.
• Graded against defined criteria (checklist, point system, mandatory critical steps).
• Must be method-realistic: a UT practical without thickness/calibration tasks, or an MT practical without particle application and indication interpretation, fails the purpose of the exam.

Level III examinations under employer programs

Employer Level III designation often emphasizes procedure development/approval, interpretation of codes and specifications, method selection, training design, and examination oversight. ASNT Central Certification uses its own Basic + Method examination path. The Written Practice must state whether ASNT Level III is accepted in lieu of employer Level III exams, partially credited, or supplemented by employer-specific demonstrations.

Developing Examination Content

Item writing principles (exam-useful, not academic pedantry)

• One clear construct per item; avoid double negatives and “all of the following except” traps unless intentional and fair.
• Align difficulty to the level: Level I focuses on performance under instruction; Level II includes interpretation and setup judgment; Level III includes system and code interface.
• Cover the topical outline—do not overweight one favorite chapter of a training manual.
• Include safety-critical method items (for example RT radiation basics awareness appropriate to the role, PT flammability/chemical handling at the level expected).
• Avoid embedding the answer in another option; avoid local slang only one crew understands.

Specific exam blueprint

Map items to the actual procedures the candidate will use:

• Applicable codes/standards invoked by the facility.
• Technique sheets and procedure numbers.
• Acceptance criteria decision points.
• Reporting and hold-point rules.
• Product-family discontinuities (castings vs welds vs wrought, etc.).

When procedures change, specific exams must be revised—stale specific exams are a classic audit finding.

Practical specimen design concepts

Design practicals so they measure job competence:

1. Specimen population — enough variety that candidates cannot memorize “panel 3 always has the only crack.”
2. Known condition set — documented discontinuities or calibration targets with expected responses (master results controlled by Level III).
3. Task sequence — setup/calibration → examination → recording → evaluation/disposition per level authority.
4. Critical fails — steps that automatic-fail the practical if missed (for example failing to perform required calibration check, violating a safety interlock rule, or accepting a rejectable indication against procedure).
5. Time bounds — realistic production-relevant time without turning the test into pure speed typing.
6. Technique limits — if certification is limited (yoke MT only), the practical must not require a bench unit the person will never use—or must clearly expand the scope if that is intended.

Document specimen control: identification, storage, contamination control (especially PT), and who may recondition specimens after candidates deposit penetrant or magnetic ink.

Administering Examinations: Control and Proctoring

Before the exam

• Verify candidate identity and that training/experience prerequisites the Written Practice requires are complete (or clearly waived only if the practice allows and is documented).
• Confirm vision currency if required before certification exams.
• Select exam form versions from a controlled bank; record form ID issued.
• Brief candidates on rules: no unauthorized devices, no discussion, permitted references only as defined.

During the exam

• Proctor present or continuous remote proctoring meeting Written Practice rules.
• Maintain seating/spacing that prevents collusion for written exams.
• For practicals, prevent coaching beyond allowed clarifications; observe critical steps.
• Control timing; handle restroom breaks per security rules.
• Log irregularities (power loss, fire alarm, suspected cheating) immediately.

After the exam

• Collect all exam materials, notes, and scratch paper if controlled.
• Grade against the official key or practical checklist.
• Record scores, form ID, date, proctor, and grader.
• Secure keys and graded papers per retention rules.
• Communicate results through controlled channels; do not post scores on a public board that exposes privacy or enables bank reconstruction.

Security threats Level IIIs must anticipate

• Shared phone photos of questions.
• “Brain dump” files circulating among crews.
• Reusing the same practical specimen sequence for years without rotation.
• Supervisors sitting in and whispering answers “to help production.”
• Uncontrolled electronic copies of specific exams emailed company-wide.

Countermeasures: multi-form banks, controlled print counts, version tracking, rotation of practical setups, separation of duties where feasible, and disciplinary linkage in the Written Practice for integrity breaches.

Grading Concepts and Program-Level Retest Rules

Grading

Typical SNT-TC-1A-style discussion for Level I/II:

• Minimum score on each examination (often 70%).
• Composite average at or above a higher threshold (often 80%).
• Confirm current published document and write the exact rule into the Written Practice so graders do not invent local folklore.

Practical grading may use weighted checklists. Critical steps can zero the practical regardless of partial credit elsewhere—if that is how the program is designed, document it and brief candidates.

Retest rules at program level

The Written Practice should answer:

• How soon may a failed exam be retaken?
• How many attempts are allowed in a period?
• Must the candidate retrain on failed topics before retest?
• Does failure of one exam type require retest of only that type, or the full battery?
• Who approves retest authorization?
• Are alternate forms required so the same form is not simply memorized?

Informal “oral overrides” that replace a failed written exam without Written Practice authority are a program integrity failure. If oral or alternate methods are allowed, they must be defined before the crisis, not invented during a production crunch.

Evaluating Competency Beyond the Score Sheet

A passing score is necessary but not always sufficient for certification recommendation:

• Scope match — exams covered the techniques the certificate will authorize.
• OJT correlation — field performance logs do not contradict exam results (or, if they do, investigate).
• Vision and medical fitness elements required by the practice.
• Procedure currency — person trained and tested on current revisions.
• Ethics/integrity — no unresolved cheating investigation.

Level III recommendation language should be factual: method, level, technique limits, exam results, vision status, and any restrictions. Management then executes employer certification per the practice.

High-Yield Exam Traps for This Section

• Treating practical exams as optional if the person “has a lot of experience.”
• Confusing general (method fundamentals) with specific (employer procedures/codes).
• Allowing unproctored exams or uncontrolled bank distribution.
• Inventing retest “one free oral pass” rules not in the Written Practice.
• Using a single practical specimen forever so the answer is tribal knowledge.
• Believing ASNT central exam administration replaces the need to understand employer exam control for Basic Domain 1 items.
• Grading without documented criteria so two Level IIIs would not reach the same pass/fail.

Master the triad develop → administer securely → grade and retest by written rules, and you can answer most Domain 1 examination items with Level III judgment rather than memorized slogans.