13.2 Creating and Selecting Assessment Formats

Choosing an Assessment Format to Fit the Objective

ETS asks candidates to know how to create assessments in a variety of formats, to select a format that meets a specific instructional objective, and to describe the uses, strengths, and limitations of essay, selected-response, portfolio, conference, observation, and performance formats.

Match the format's cognitive demand to the objective's. An objective to design an experiment calls for a performance task or a written design, not a matching quiz; an objective to identify organelles can be assessed efficiently with selected response.

Designing Selected-Response Items: Anatomy and Best Practices

Selected-response items require students to identify and choose the correct answer from a menu of options provided by the test designer. Common formats include multiple-choice, matching, and alternate-choice (true/false) items. When properly constructed, selected-response items allow teachers to sample broad curricular domains efficiently and score student responses objectively and quickly.

The Anatomy of a Multiple-Choice Item

A standard multiple-choice item consists of three distinct components:

1. The Stem: The introductory statement, question, or problem that presents the central cognitive task to the student.
2. The Keyed Response (The Key): The single correct, or indisputably best, answer among the alternatives.
3. Distractors: The incorrect alternatives designed to attract students who possess partial knowledge, harbor common misconceptions, or commit typical procedural errors.

Guidelines for Writing High-Quality Item Stems

• Present a Single, Clear Problem in the Stem: The stem should be fully formulated so that a student who knows the material can answer it in their mind before reading the choices. Stems that merely state a fragmented phrase (e.g., "William Shakespeare...") force students to evaluate four disconnected propositions, increasing cognitive load without testing disciplinary depth.
• Eliminate Irrelevant Window Dressing: Stems should be concise and focused, stripping away extraneous background narrative that does not contribute to the problem.
• Avoid Negative Stems Wherever Possible: Stems containing "NOT," "EXCEPT," or "LEAST" are cognitively disorienting, often measuring a student's reading speed or visual attention rather than their subject-matter competence. If a negative stem is pedagogically essential, the negative word must be explicitly emphasized using BOLD, CAPITALIZATION, or italics (e.g., "Which of the following is NOT a primary cause of...").

Guidelines for Writing Plausible Distractors

• Incorporate Realistic Student Misconceptions: Effective distractors are not bizarre, frivolous, or obviously absurd. High-quality distractors represent documented cognitive misunderstandings or common computational mistakes (e.g., in math, an option reflecting a sign error; in physics, an option reflecting the confusion between velocity and acceleration).
• Maintain Parallelism in Structure and Length: All options should be approximately identical in length, grammatical structure, vocabulary complexity, and degree of specificity. A notorious flaw in teacher-made tests is making the correct answer noticeably longer, more elaborate, and more carefully qualified than the hasty distractors.
• Eliminate Clues and Grammatical Inconsistencies: Ensure that every distractor aligns grammatically with the stem. A stem ending with the indefinite article "an" followed by three distractors beginning with consonants and one beginning with a vowel provides an unintended syntactic clue that invalidates the item.
• Ban Absolute Qualifiers: Test-savvy secondary students know that statements containing absolute words like "always," "never," "all," "none," or "must" are virtually always false in complex secondary subjects. Conversely, options with qualifying words like "often," "generally," "may," or "typically" are frequently correct. Eliminate these telltale markers.
• Avoid "All of the Above" and "None of the Above":
  • "All of the above" undermines measurement: a student who recognizes that choices A and B are correct can select "All of the above" without having any knowledge of choice C. Furthermore, if a student knows that just one option is false, they immediately eliminate both that option and "All of the above."
  • "None of the above" measures what students know is incorrect rather than what they know is correct, providing little diagnostic evidence of true conceptual mastery.

Designing Constructed-Response Items: Prompts, Rubrics, and Alignment

Constructed-response items require students to generate and express their own original answers. Formats range from short-answer completions (e.g., fill-in-the-blank, one-sentence explanations) to extended-response essays, Document-Based Questions (DBQs), and multi-step mathematical derivations.

Formulating Effective Constructed Prompts

• Define the Cognitive Task Explicitly: Novice teachers often write vague prompts like "Discuss the Great Depression." High-quality secondary prompts specify the precise analytical verbs aligned with Bloom's Taxonomy: "Compare and contrast the economic policies of Herbert Hoover and Franklin D. Roosevelt regarding direct federal relief, citing at least two specific legislative acts for each president."
• Specify Task Constraints: Transparently delineate the expected scope, including word count or time limits, required citation formats, and the specific structural components that must be present.
• Pre-Construct the Scoring Rubric Prior to Testing: A teacher should never administer an essay prompt without having already developed the corresponding analytic or holistic scoring rubric. Constructing the rubric alongside the prompt ensures that the prompt is clearly aligned with measurable learning objectives.

Performance-Based Assessment: Definition and Core Principles

A performance-based assessment (PBA) is any evaluation format that requires students to actively generate an original product or execute a complex, observable behavioral demonstration to prove their mastery of specific knowledge and skills. Rather than recognizing or selecting an answer from a pre-determined menu of choices, the student must construct, perform, or produce.

Distinctive Characteristics of Performance Tasks

1. Active Construction and Application: Students synthesize multiple concepts and apply procedural skills to create something new (e.g., writing a computer algorithm, conducting an empirical experiment, or choreographing a dramatic scene).
2. Emphasis on Higher-Order Thinking: Tasks are aligned with the upper tiers of Bloom's Revised Taxonomy (Analyze, Evaluate, Create) and Webb's Depth of Knowledge (Strategic and Extended Thinking).
3. Transparent Evaluation Criteria: Performance tasks are evaluated using clear, pre-distributed scoring rubrics or performance checklists that define varying levels of quality across distinct dimensions.
4. Integration of Process and Product: Performance assessments often evaluate both the procedural journey (e.g., adherence to scientific laboratory safety, collaboration skills, revision strategies) and the final culminating product (e.g., written lab report, working engineering model).

Secondary Disciplinary Examples

• Mathematics (Grade 10 Geometry): Students design an energy-efficient architectural model of a tiny home, calculating surface-area-to-volume ratios, optimizing solar exposure angles using trigonometry, and submitting detailed scaled blueprints.
• Science (Grade 11 Chemistry): Students are presented with four unlabelled clear aqueous solutions and must design and execute a qualitative chemical analysis scheme using precipitation reactions and flame tests to identify each unknown compound correctly.
• Social Studies (Grade 12 Government): Students participate in a simulated appellate court hearing, writing an appellate brief analyzing Fourth Amendment cell phone search precedents and delivering a five-minute oral argument before a panel of guest attorneys.
• World Languages (Grade 8 Spanish): Students conduct an unscripted, four-minute video-recorded conversation with a peer navigating an unexpected flight delay at a Madrid airport, negotiating hotel accommodations and ground transit entirely in the target language.

Grant Wiggins and Authentic Assessment: Real-World Ecological Validity

While all authentic assessments are performance-based, not all performance assessments are authentic. Grant Wiggins, a pioneer in backward curriculum design, established the critical distinction between generic performance tasks and authentic assessments.

A task is authentic when it possesses ecological validity—meaning it directly mirrors the actual intellectual challenges, realistic constraints, messy ambiguities, and collaborative standards encountered by adult professionals, civic leaders, and scientists working in the field.

Wiggins's Core Criteria for Authentic Assessment

In Educative Assessment (1998), Wiggins lists six characteristics of an authentic task (paraphrased here):

1. Realistic Context: The task is situated in a genuine real-world scenario with authentic constraints (e.g., limited budget, tight deadlines, competing stakeholder priorities).
2. Requires Judgment and Innovation: The student cannot simply follow a prescriptive, algorithmic formula or cookbook set of instructions; they must make strategic decisions, evaluate trade-offs, and justify their choices.
3. Asks the Student to "Do" the Discipline: The student does not merely read about what historians or scientists do; they act as a historian analyzing conflicting primary sources, or as an investigative journalist verifying claims.
4. Replicates Real Contexts: The task simulates the settings in which adults are "tested" at work, in civic life, or in personal life, with their messy constraints and purposes.
5. Requires an Efficient Repertoire: The student must combine knowledge and skills efficiently and effectively to negotiate a complex, multistage task rather than recall isolated facts.
6. Allows Rehearsal, Resources, and Feedback: Real-world professional work is rarely executed in a single, high-stakes, timed sitting. Authentic assessment embeds opportunities for drafting, receiving critical peer and expert critique, refining, and polishing.

Portfolio Assessment: Working, Showcase, and Assessment Portfolios

A portfolio is a purposeful, systematic, and longitudinal collection of student work that documents effort, progress, and academic achievement over time. Unlike a random folder of completed assignments, a portfolio is defined by intentional selection, explicit evaluative criteria, and structured student self-reflection.

Secondary educators utilize three primary portfolio models, each serving a distinct pedagogical function:

1. Working Portfolio (Process Portfolio)

• Purpose: To document the ongoing creative, drafting, and problem-solving process over the course of a unit or semester.
• Contents: Unfinished brainstorms, initial rough drafts, peer-critique rubrics, teacher conference notes, second revisions, and reflections detailing how the student overcame specific hurdles.
• Secondary Role: Working portfolios celebrate the iterative nature of learning. They help adolescents understand that intellectual growth involves making mistakes, analyzing feedback, and revising ideas. In an 8th-grade art class, a working portfolio documents preliminary thumbnail sketches, color-mixing experiments, and structural failures before the final ceramic sculpture.

2. Showcase Portfolio (Display / Best-Work Portfolio)

• Purpose: To exhibit the student's highest-quality completed artifacts, illustrating their pinnacle of achievement and personal pride.
• Contents: Carefully curated selections chosen primarily by the student, accompanied by written rationales explaining why each artifact represents their finest work.
• Secondary Role: Often used for culminating senior capstones, college admissions applications, vocational-technical certification showcases, or parent-teacher conferences. In a 12th-grade creative writing course, a showcase portfolio features three polished short stories, two revised poems, and an introductory essay articulating the student's evolving literary voice.

3. Assessment Portfolio (Evaluation Portfolio)

• Purpose: To provide systematic, verifiable evidence that the student has mastered specific state curriculum standards, graduation competencies, or course proficiencies.
• Contents: Standardized artifacts mandated by the department or curriculum committee, evaluated using validated analytic rubrics with strict scoring guidelines.
• Secondary Role: In many secondary districts, high school graduation requires passing an assessment portfolio in writing or digital literacy, ensuring all graduating seniors demonstrate baseline competency across multiple informational and argumentative genres.

The Indispensable Element: Metacognitive Reflection

Every authentic portfolio must include structured student reflections. A portfolio without reflection is merely a storage bin. When secondary students analyze their own growth—articulating why a particular piece was challenging, how their thinking evolved between Draft 1 and Draft 3, and what strategies they will apply to future tasks—they engage in high-level metacognitive self-regulation.

Complex Performance Formats: Simulations, Debates, Lab Practicals, and Capstones

Secondary educators deploy diverse performance formats to stimulate adolescent engagement:

• Simulations and Mock Trials: Immersing students in simulated societal roles (e.g., Model United Nations, mock legislative debate, mock epidemiological CDC outbreak team). Students must synthesize disciplinary knowledge under dynamic, unpredictable conditions.
• Oral Defenses and Socratic Panels: Moving beyond written testing by requiring students to verbally articulate and defend their scientific hypotheses or historical theses before a panel of peers and instructors, mirroring university dissertation defenses.
• Laboratory Practicals: In high school biology, chemistry, and physics, students are evaluated not on their ability to memorize lab procedures, but on their physical execution of experimental techniques—such as setting up a titration apparatus, calibrating spectrophotometers, or preparing microscopic slide specimens.
• Senior Capstones: Year-long interdisciplinary inquiry projects where high school seniors investigate a complex community problem, produce a substantive thesis paper, create a tangible physical or digital artifact, and present their findings to a community jury.

Advantages and Limitations of Performance Assessments

Secondary educators must weigh the psychometric and practical trade-offs when designing performance assessments:

Advantages

• Exceptional Ecological Validity: Measures genuine problem-solving in realistic contexts, providing direct evidence of skill transfer.
• Stimulates Adolescent Intrinsic Motivation: Students exhibit higher ownership, engagement, and pride when creating original products rather than filling out bubble sheets.
• Accommodates Diverse Learning Strengths: Enables students who experience test anxiety or who struggle with traditional reading-heavy multiple-choice formats to demonstrate deep conceptual competence.
• Promotes Collaboration and Communication: Reflects modern workplace demands by evaluating interpersonal discourse, team coordination, and project management.

Limitations & Practical Challenges

• Substantial Time Investment: Developing authentic tasks, administering simulations, and evaluating lengthy projects requires extensive instructional and professional time.
• Reduced Curricular Sampling Breadth: A two-week performance project can only evaluate a narrow slice of content standards, whereas a 60-item selected-response test samples an entire semester's curriculum.
• Vulnerability to Subjective Scoring Drift: Without rigorous, well-defined analytic rubrics and regular teacher calibration, scoring can be compromised by halo effects, personal bias, and inconsistent evaluative standards across teachers.

Common Exam Traps

• Format mismatch. A recall test cannot show whether students can evaluate or create.
• Telltale item flaws. Correct options that are longer or more qualified than the distractors, grammatical cues, and "all of the above" let test-wise students guess.
• No rubric for constructed responses. Write the rubric with the prompt.
• Hands-on equals authentic. An activity is an authentic assessment only when it requires real-world judgment, not just crafts.