10.3 Choosing Useful Test Cases: Boundaries, Partitions, Coverage, and Regression

What this competency asks

Under debugging and testing, ETS asks you to identify which test cases are most useful for given code. ETS's sample question shows a segment meant to store the largest of three numbers, a, b, and c, in max. It asks which set of values demonstrates that the segment does not work. The correct choice sends execution into the one branch that forgets to assign max. This section gives you a method for such questions.

What makes a test case useful

A test case is an input together with its expected output. A test is useful when it could reveal a defect that other tests would miss. Ten tests that all follow the same path through the code add little over one.

Equivalence partitioning

Divide the possible inputs into partitions, groups the program should treat the same way, and test one representative from each, including invalid partitions.

Example: a ticket price is $5 for ages 0–12, $10 for ages 13–64, and $6 for ages 65 and up. Negative ages are invalid.

Boundary value analysis

Defects cluster at the edges of partitions, where < and ≤ are easy to confuse. For each boundary, test the value at it, just below it, and just above it.

For a rule that accepts 12 ≤ age ≤ 17, the boundary tests are 11, 12, 13, 16, 17, 18. If the code wrongly used age < 17, the test with 17 fails immediately. Middle values like 14 and 15 would pass either version, so they cannot catch this bug.

Finding the input that exposes a bug

Method:

1. Read the code and find the suspect path: a branch that does not assign something, a boundary comparison, or an unusual case.
2. Choose an input that reaches that path.
3. Confirm that the correct answer differs from what the code actually produces.

Example:

// intended: return true if year is a leap year
boolean isLeap ( int year )
    if ( year % 4 == 0 )
        return true
    else
        return false
    end if
end isLeap

The full rule: a year is a leap year if it is divisible by 4, except century years, which must be divisible by 400. Consider these tests:

Only 1900 exposes the missing century rule. A test must both reach the faulty logic and have a different correct answer. The year 2000 reaches the century case, but the code happens to return the correct answer anyway.

Coverage: exercising the code itself

White-box testing chooses inputs by looking at the code. Black-box testing chooses them from the specification alone.

Branch coverage is stronger than statement coverage. An if with no else can have every statement covered by a single true case, while the false case, which might be the buggy one, is never tried.

Special cases worth a test

• Empty input: an empty array or empty string
• One element
• Equal values, such as ties when finding a maximum, or duplicates in a sort
• Extremes: 0, negative numbers, and the largest expected values
• Already sorted and reverse-sorted data for sorting code
• Invalid input, such as text where a number is expected, to test validation (Section 11.2)

Levels of testing

Regression testing matters because fixes can break other things. Automated test suites make it cheap to rerun every test after every change.

Writing expected results first

Decide the expected output before running the test, from the specification or a hand calculation. A test with no expected result can only show that a program runs, not that it is right. Teachers can model this habit by having students write test tables before they write code.