15.2 Using Research, Evidence Tiers & Action Research

Why Research Literacy Is a Teaching Competency

ETS asks candidates to understand the implications of research, views, ideas, and debates on teaching practices: to know resources for accessing research, to interpret data, results, and conclusions from research on teaching, and to relate those findings to a variety of educational situations. ETS's discussion questions also expect candidates to be able to read and understand articles and books about current views, ideas, and debates regarding best teaching practices.

Where to Find Research and Professional Debate

Reading a Study Critically

1. Question and design. Is it an experiment with random assignment, a quasi-experiment with a comparison group, a correlational study, a qualitative study, or a meta-analysis? Only experimental and strong quasi-experimental designs support causal claims.
2. Sample. How many students, and how similar are they to yours?
3. Measures. Did the study measure the outcome you care about?
4. Results. How large was the effect, not just whether it was statistically significant? Cohen's rough conventions call effect sizes of about 0.2 small, 0.5 medium, and 0.8 large, although smaller effects can matter in education.
5. Limitations and funding. Did the authors acknowledge limits, and did a program's publisher fund the study?

Correlation is not causation. If students who complete more homework earn higher grades, homework may help, but motivation, prior achievement, and family support could explain both. Only a design that controls for these factors can show that homework causes the difference.

Views, Ideas, and Debates

Secondary teachers regularly encounter contested questions: how much homework helps; whether to track or detrack; grading practices such as minimum grades and standards-based grading; how much explicit instruction versus inquiry; limits on student phone use; and the role of AI tools. A professional response weighs the evidence, considers the local context, pilots a change, and collects data on the results rather than adopting or rejecting an idea on reputation alone.

Evaluating Educational Research Credibility: ESSA Evidence Tiers

Secondary educators are constantly bombarded by commercial vendors and curriculum publishers marketing "revolutionary" software, curricula, and instructional fads. To protect public funds and ensure student learning, educators must be equipped to critically evaluate the methodological soundness and empirical credibility of educational claims.

The federal Every Student Succeeds Act (ESSA) established a formal, four-tiered framework to define what constitutes an evidence-based intervention:

+-----------------------------------------------------------------------------------+
|                         ESSA TIERS OF EVIDENCE-BASED RESEARCH                     |
+-----------------------------------------------------------------------------------+
|  Tier 1: STRONG Evidence                                                          |
|  * Supported by at least one well-designed, well-implemented randomized            |
|    controlled trial (RCT). Demonstrates statistically significant positive effect.|
|                                                                                   |
|  Tier 2: MODERATE Evidence                                                        |
|  * Supported by at least one well-designed, well-implemented quasi-experimental   |
|    study (matched comparison group, no random assignment).                        |
|                                                                                   |
|  Tier 3: PROMISING Evidence                                                       |
|  * Supported by at least one well-designed correlational study with statistical   |
|    controls for selection bias.                                                   |
|                                                                                   |
|  Tier 4: DEMONSTRATES A RATIONALE                                                 |
|  * Well-defined logic model grounded in empirical literature, with an ongoing     |
|    formal evaluation plan to measure outcomes.                                    |
+-----------------------------------------------------------------------------------+

Unmasking Commercial Fads and Pseudoscience: The Learning Styles Myth

A classic example of an unscientific pedagogical fad is the VARK "Learning Styles" theory (the belief that individuals learn fundamentally better when instruction is tailored strictly to their self-reported visual, auditory, reading, or kinesthetic preference).

• The Scientific Consensus: Overwhelming empirical research across cognitive science and educational psychology has repeatedly demonstrated that catering to learning styles produces zero measurable improvement in academic achievement. While adolescents express personal preferences, human cognitive architecture processes conceptual meaning semantically.
• The Evidence-Based Alternative: Rather than attempting to categorize students into rigid visual or auditory buckets, cognitive science validates Dual Coding Theory (Allan Paivio) and Universal Design for Learning (UDL). Presenting information through integrated multimodal channels (e.g., combining verbal explanations with synchronized visual diagrams) benefits all adolescent learners, regardless of their self-perceived "style."

Practical Checklist for Evaluating Commercial Educational Programs

When evaluating a proposed instructional program, secondary educators should demand answers to five critical methodological questions:

1. Is the study published in a peer-reviewed academic journal? (Or is it an internal "white paper" funded and published by the software vendor itself?)
2. Was there an authentic comparison group? (Did researchers compare students using the product against an equivalent control group receiving high-quality standard instruction?)
3. Is the sample representative of our secondary student population? (Was the program tested on diverse adolescent learners in public middle and high schools, including English learners and students with disabilities?)
4. What was the effect size (e.g., Cohen's d)? (Is the demonstrated academic gain meaningful in practice, or merely a trivial difference that achieved statistical significance due to an enormous sample size?)
5. Have independent researchers replicated the results? (Has any researcher without a financial stake in the commercial program successfully reproduced the positive findings?)

---

Action Research in the Secondary Classroom

Action research is a systematic, reflective inquiry conducted by classroom practitioners in their own instructional settings to gather information about how their particular school operates, how they teach, and how well their students learn (Mills, 2011). Unlike traditional university-based academic research—which seeks broad generalizability and controlled laboratory conditions—action research focuses on localized, practical problem-solving.

The Cyclical Action Research Process

Action research operates as an iterative, continuous cycle comprising four core phases (the Lewin/Kemmis & McTaggart cycle):

                      +---------------------------------------+
                      |           1. PLAN                     |
                      |  * Identify problem of practice       |
                      |  * Formulate researchable question    |
                      |  * Review literature & select strategy|
                      +-------------------+-------------------+
                                          |  
                                          v
                      +---------------------------------------+
                      |           2. ACT                      |
                      |  * Implement targeted intervention    |
                      |  * Maintain instructional fidelity    |
                      |  * Execute over 4-8 weeks             |
                      +-------------------+-------------------+
                                          |  
                                          v
                      +---------------------------------------+
                      |           3. OBSERVE                  |
                      |  * Collect triangulated data          |
                      |  * Quantitative & qualitative evidence|
                      |  * Track student growth & engagement  |
                      +-------------------+-------------------+
                                          |  
                                          v
                      +---------------------------------------+
                      |           4. REFLECT                  |
                      |  * Analyze data against question      |
                      |  * Draw pedagogical conclusions       |
                      |  * Share findings & modify next cycle |
                      +---------------------------------------+

1. Formulating a Researchable Secondary Classroom Question

The foundation of effective action research is a well-crafted question. Researchable questions are specific, actionable, focused on teacher practice within the classroom's locus of control, and linked to measurable adolescent learning outcomes.

• Flawed (Non-Researchable) Question: "Does adolescent poverty decrease high school students' motivation to complete homework?"
  • Why it fails: It examines external socioeconomic variables outside the teacher's immediate control, treats motivation as a fixed deficit, and yields no actionable pedagogical intervention.
• Exemplary (Researchable) Question: "How does the implementation of structured, 5-minute peer-editing protocols using an analytic rubric impact 10th-grade students' ability to synthesize primary source evidence in argumentative history essays?"
  • Why it succeeds: It isolates a clear independent instructional variable (structured peer editing with an analytic rubric), targets a specific secondary cohort (10th-grade history), and evaluates an observable, measurable academic outcome (primary source synthesis in essays).

2. Data Collection and Triangulation

To ensure validity and credibility, action researchers must practice data triangulation—collecting at least three distinct, independent sources of qualitative and quantitative evidence. Relying on a single metric (such as a single quiz score) introduces severe measurement error.

                           [ Triangulated Data Matrix ]
                                         |
         +-------------------------------+-------------------------------+
         |                               |                               |
         v                               v                               v
   Quantitative Data               Qualitative Data              Observational Data
  * Pre- and post-tests           * Student focus group notes    * Teacher field notes
  * CFA mastery percentages       * Open-ended survey responses  * Peer observation tallies
  * Rubric sub-scores             * Student self-reflection logs * Time-on-task matrices

3. Analyzing Evidence to Modify Instruction

Data analysis in action research does not require advanced multivariate statistics. Practitioners utilize inductive thematic coding for qualitative data (identifying recurring themes in student reflections) and descriptive statistics for quantitative data (calculating percentage gains, median scores, and disaggregating by learner sub-populations). The ultimate goal is pedagogical modification: determining whether the intervention produced meaningful cognitive growth and identifying specific adjustments required for the next instructional iteration.

4. Disseminating Findings and Collective Inquiry

Action research achieves maximum institutional power when findings are not kept secret. Sharing outcomes during department PLC meetings, district professional development sessions, or regional educational conferences builds collective teacher efficacy. Even when an intervention fails to produce expected gains, sharing that outcome prevents colleagues from repeating ineffective strategies and initiates a new cycle of collaborative problem-solving.

---

Disciplinary Scenarios in Secondary Classrooms

Secondary Humanities: 11th-Grade ELA Action Research

• The Problem: Ms. Thornton notices that her 11th-grade students participate minimally during full-class discussions of complex literary texts, with conversations dominated by the same three extroverted students.
• Action Research Plan: Ms. Thornton formulates the question: "What is the effect of integrating structured Silent Socratic Discussions (digital collaborative annotation boards) prior to oral seminars on the equity and analytical depth of adolescent discourse?"
• Action & Observation: Over a 6-week unit on American transcendentalism, Ms. Thornton implements 15 minutes of silent digital annotations before every spoken seminar. She triangulates data: (1) a rubric-based tally of oral textual citations, (2) digital platform contribution logs, and (3) an anonymous student perception survey.
• Reflection & Impact: Data reveals oral participation among previously silent students increased by 140%, and textual citations in analytical essays rose by 35%. Students reported that silent written brainstorming lowered their affective filter and gave them the confidence to speak. Ms. Thornton presents these findings at the regional language arts symposium.

---

Common Exam Traps

• Confusing Action Research with Academic Randomized Controlled Trials (RCTs): Action research is conducted by classroom teachers in their own naturalistic environments to solve immediate problems of practice. It does not require withholding instruction from a "control group" of students, which would be ethically unacceptable in a secondary classroom.