10.2 Test Construction, Table of Specifications TOS, and Item Analysis

Planning the Test: The Table of Specifications (TOS)

A Table of Specifications (TOS)—also known as a test blueprint—is a two-way grid that aligns learning competencies and instructional time with cognitive demand levels. Developing a TOS is the single most critical step in ensuring content validity in classroom assessment. Without a TOS, test developers risk over-testing lower-level recall items or omitting major curriculum topics.

Alignment with Bloom's Revised Taxonomy

Modern test construction structures the TOS across the six cognitive levels of Anderson and Krathwohl's Revised Bloom's Taxonomy:

1. Remembering (Recall): Retrieving relevant knowledge from long-term memory.
2. Understanding (Comprehension): Constructing meaning from instructional messages.
3. Applying (Application): Carrying out or using a procedure in a given situation.
4. Analyzing (Analysis): Breaking material into constituent parts and determining relationships.
5. Evaluating (Evaluation): Making judgments based on criteria and standards.
6. Creating (Synthesis): Putting elements together to form a coherent, original functional whole.

Step-by-Step Procedure for Constructing a TOS

1. List Topics & Allocated Hours: Record all instructional topics and the actual number of class hours spent teaching each topic.
2. Calculate Topic Weight (%): Divide hours spent on a topic by total instructional hours: $\text{Weight (\%)} = \left( \frac{\text{Class Hours for Topic}}{\text{Total Class Hours}} \right) \times 100\%$
3. Determine Number of Test Items per Topic: Multiply the total number of items planned for the test by the topic weight: $\text{Items per Topic} = \text{Total Test Items} \times \text{Topic Weight}$
4. Distribute Across Cognitive Levels: Allocate items across Lower-Order Thinking Skills (LOTS: Remembering, Understanding) and Higher-Order Thinking Skills (HOTS: Applying, Analyzing, Evaluating, Creating) according to the target cognitive ratio (e.g., 60% LOTS / 40% HOTS).

---

Guidelines for Constructing Objective Test Items

To write high-quality test items that measure actual knowledge rather than test-wiseness, teachers must adhere to established psychometric guidelines.

Guidelines for Multiple-Choice Items

• The Stem: The stem must present a complete, unambiguous problem. A student should be able to understand the question without reading the choices. Avoid vague stems like 'Which of the following is true?'.
• Avoid Double Negatives: Stems containing negative phrasing ('Which of the following is NOT...') must emphasize the negative word in BOLD or CAPITALS. Avoid double negatives entirely (e.g., 'Which is NOT uncharacteristic of...').
• Distractor Construction: Distractors (incorrect options) must be plausible to unprepared students. They should be homogeneous in content, grammatical structure, and relative length. Avoid creating a correct option that is noticeably longer or more detailed than the distractors ('length cue').
• Eliminate Grammatical Clues: Ensure the stem connects grammatically to all options. For instance, ending a stem with 'an' clues that the correct answer begins with a vowel.
• Avoid Absolute Modifiers: Words like always, never, all, and none usually indicate false statements in true-false or distractor options.
• Avoid Overusing 'All of the Above': 'All of the above' allows students to guess correctly if they recognize just two correct options. 'None of the above' should not be used when evaluating relative best answers.

Guidelines for Matching-Type Items

• Keep the list of premises (Column A) and responses (Column B) homogeneous (e.g., Column A contains historical events; Column B contains dates).
• Use unequal numbers of premises and responses (more responses in Column B) to prevent correct guessing by process of elimination on the final item.

---

Quantitative Item Analysis

After administering a test, teachers perform Item Analysis to evaluate the quality of each item. Item analysis yields two primary psychometric indicators: the Item Difficulty Index ($P$) and the Item Discrimination Index ($D$).

Procedure for Grouping Test Papers

1. Rank all graded test papers from highest to lowest score.
2. Select the top 27% of papers to form the Upper Group ($U$).
3. Select the bottom 27% of papers to form the Lower Group ($L$).
4. Set aside the middle 46% of papers.

1. Item Difficulty Index ($P$)

The Difficulty Index represents the proportion of students who answered the item correctly. It is calculated using the formula: $P = \frac{R_U + R_L}{N_T}$ where $R_U$ is the number of correct responses in the Upper Group, $R_L$ is the number of correct responses in the Lower Group, and $N_T$ is the total number of students in both groups combined ($N_U + N_L$).

2. Item Discrimination Index ($D$)

The Discrimination Index measures an item's ability to differentiate between high-performing and low-performing students. It is calculated using the formula: $D = \frac{R_U - R_L}{n}$ where $n$ is the number of students in a single group ($n = N_U = N_L$).

              DISCRIMINATION INDEX (D) DECISION MATRIX
┌────────────────────────┬──────────────────────────────────────────┐
│ D ≥ +0.40              │ Excellent Item ➔ Retain                  │
├────────────────────────┼──────────────────────────────────────────┤
│ +0.20 ≤ D ≤ +0.39      │ Marginal Item  ➔ Revise                  │
├────────────────────────┼──────────────────────────────────────────┤
│ D < +0.20              │ Poor Item      ➔ Discard                 │
├────────────────────────┼──────────────────────────────────────────┤
│ D < 0.00 (Negative)    │ Defective      ➔ Discard / Check Key     │
└────────────────────────┴──────────────────────────────────────────┘

Worked Examples & LET Exam Scenario Traps

Worked Example: Item Analysis Calculation

Data: A teacher conducts an item analysis for Item #15 among 100 students. The upper 27% ($N_U = 27$) and lower 27% ($N_L = 27$) are selected. In the upper group, 24 students answered Item #15 correctly ($R_U = 24$). In the lower group, 9 students answered correctly ($R_L = 9$).

Calculations:

1. Total Sample ($N_T$): $N_U + N_L = 27 + 27 = 54$
2. Single Group Size ($n$): $27$
3. Difficulty Index ($P$): $P = \frac{24 + 9}{54} = \frac{33}{54} \approx 0.61$ Interpretation: $P = 0.61$ indicates Moderate Difficulty (Optimal range 0.31–0.80).
4. Discrimination Index ($D$): $D = \frac{24 - 9}{27} = \frac{15}{27} \approx +0.56$ Interpretation: $D = +0.56$ indicates Excellent Discrimination ($\ge +0.40$).

Final Decision: Retain Item #15 unconditionally in the test bank.

LET Exam Scenario Trap: Negative Discrimination

Trap: What does it mean when an item yields a discrimination index of $D = -0.25$?

Analysis: A negative discrimination index means that more students in the lower-performing group answered the item correctly than students in the upper-performing group ($R_L > R_U$). This indicates a severely flawed item caused by:

• An incorrect answer key (the item was miskeyed).
• Ambiguous phrasing that misled high-achieving students who over-thought the trick options.
• A distractor that was unexpectedly appealing to top performers.

Required Action: The item MUST be discarded or re-keyed immediately, and credit should be re-evaluated for affected students.