3.10 Evaluating Evidence in a Draft
Key Takeaways
- Evaluation of evidence is a Revision sub-topic distinct from source evaluation: it asks whether the evidence in a draft actually supports the sentence it is attached to.
- Relevance, sufficiency, representativeness, and currency are the four tests, and relevance failures are the most common.
- A statistic can be accurate and still fail to support the claim if it measures a different variable than the claim asserts.
- Anecdotes establish that something can happen, not how often; a frequency claim supported by one case is underdetermined.
- When a draft's evidence is weaker than its claim, the credited revision either strengthens the evidence or weakens the claim - both are legitimate fixes.
3.10 Evaluating Evidence in a Draft
The Revision outline lists evaluation of evidence separately from the Source Materials area's evaluation of sources. The distinction matters. Source evaluation asks whether a source is trustworthy. Evidence evaluation asks a narrower question inside the draft: does this particular piece of support establish this particular claim?
A perfectly credible source can supply evidence that does not support the sentence it has been attached to.
Four Tests
| Test | Question | Failure name |
|---|---|---|
| Relevance | Does the evidence bear on the claim as stated? | Off-point support |
| Sufficiency | Is there enough of it to carry a claim this strong? | Hasty generalization |
| Representativeness | Does the sample reflect the population the claim covers? | Biased sample |
| Currency | Is the evidence recent enough for the claim's time frame? | Stale data |
Relevance is the test that decides most CLEP items, and it is the one drafts fail most quietly, because the evidence is usually true.
The Measurement Mismatch
The commonest relevance failure is evidence that measures a different variable than the claim asserts.
| Claim asserts | Evidence measures | Verdict |
|---|---|---|
| The tutoring program improved learning. | Student satisfaction ratings | Mismatch — satisfaction is not learning |
| The policy reduced traffic deaths. | Reported traffic citations | Mismatch — enforcement is not outcome |
| Remote work lowered turnover. | Employee self-reported intent to stay | Mismatch — intent is not turnover |
| The drug reduced mortality. | Reduction in a biomarker | Mismatch — a surrogate endpoint is not mortality |
Procedure: underline the noun the claim is about, then underline the noun the evidence counts. If they are different nouns, the evidence is off point regardless of how strong the numbers are.
Existence Claims vs. Frequency Claims
An anecdote or a single case establishes that something can happen. It cannot establish how often.
- Supported by one case: "Municipal composting can operate at a lower cost per household than curbside recycling." (one city demonstrating it is enough)
- Not supported by one case: "Municipal composting typically operates at a lower cost per household." (typically is a frequency claim; one city cannot establish it)
Watch the quantifier in the claim: can, may, sometimes are existence claims; typically, generally, most, usually, always are frequency claims and require systematic evidence.
Sufficiency and Representativeness
- Hasty generalization: three interviews used to characterize a 40,000-student university.
- Biased sample: an online poll of a newsletter's subscribers used to characterize public opinion; the respondents selected themselves.
- Survivorship problem: surveying only current employees about workplace satisfaction, when the dissatisfied ones already left.
- Unit mismatch: national averages used to support a claim about one district, or one district's figures used to support a national claim.
Currency
Currency is claim-relative, not absolute. A 1998 study is stale for a claim about social media use and perfectly current for a claim about a 19th-century archival holding. Ask what would have to have changed since publication for the finding to no longer hold; if the answer is "the thing being measured," the evidence is stale.
Repairing an Over-Claimed Draft
When the evidence in a draft is weaker than the claim it supports, revision items typically offer both legitimate repairs. Either is correct depending on what the rest of the draft can sustain.
Draft: "Four-day workweeks eliminate burnout. At one Reykjavík firm, self-reported stress fell during a six-month trial."
| Repair | Revision |
|---|---|
| Strengthen the evidence | Replace the single trial with the multi-site results the draft cites later, and report a measured outcome rather than self-report. |
| Weaken the claim | "Four-day workweeks can reduce reported stress, at least over short trials." |
What is not a repair: adding an intensifier ("clearly eliminate"), adding a second anecdote (still not a frequency claim), or attributing the claim to an authority without adding evidence.
Distinguishing Evidence from Assertion
Some sentences in a draft look like support but only repeat the claim in the vocabulary of evidence:
- Assertion dressed as evidence: "Research consistently confirms that the program works." — names no study, no measure, no result.
- Actual evidence: "A randomized trial of 1,200 participants found completion rates 18 points higher in the program group."
On items asking which sentence best supports a claim, eliminate every option that gestures at evidence without supplying a measurable result, a named source, or a described case. Vague appeals to "studies," "experts," and "research" are among the most reliable wrong answers in the Revision area.
A draft claims that a university's new tutoring program improved student learning, and supports the claim with end-of-term surveys in which 88% of participants said they were satisfied with their tutor. What is the problem with this evidence?
Which claim is adequately supported by a single documented case?
A draft asserts that four-day workweeks eliminate burnout, citing self-reported stress reductions at one firm during a six-month trial. Which revision legitimately repairs the mismatch between claim and evidence?