2.5 Confirmation, Regression, and Maintenance Testing

Confirmation Testing Versus Regression Testing

Software changes constantly: defects are fixed, features added, platforms upgraded, data migrated, and old systems retired. Two change-related test activities address the resulting risk, and CTFL v4.0 insists you keep them separate.

Confirmation testing (also called retesting) is performed after a defect is fixed to confirm that the original failure no longer occurs. The syllabus specifies that, in general, confirmation testing should re-run all the test cases that previously failed because of the defect, and where the fix changed code, it may also add new tests to cover the change. Example: a defect report says checkout ignored gift cards; confirmation testing repeats the gift-card scenario and verifies the card is now applied.

Regression testing is performed to detect whether a change has caused unintended side effects in previously working, unchanged parts of the system or its environment. Such changes can include not only code changes but a change to the operating system, database, or a third-party component. The gift-card fix might inadvertently break discounts, tax, invoicing, or refunds — regression testing hunts those side effects.

A leading exam trap is to call every post-fix test "regression testing." If the test re-runs the exact defect scenario, it is confirmation testing; if it checks other, unchanged areas for side effects, it is regression testing. Both are usually needed after a fix.

Maintenance Testing and Its Four Triggers

Maintenance means modifying software after it is operational; maintenance testing evaluates those modifications and their effects on the existing system. CTFL v4.0 groups the triggers for maintenance (and therefore maintenance testing) into four categories:

• Modification — planned enhancements (e.g., a release-based change), corrective and emergency fixes, and changes to the operating environment such as planned operating-system or database upgrades, and changes to commercial off-the-shelf (COTS) components.
• Migration — moving to another platform, which can require operational tests of the new environment plus tests of the changed software, and data conversion testing when data migrates from another application.
• Retirement — when an application reaches end of life; testing may include archiving (for long data-retention periods) and restore/retrieve test procedures after archiving.
• Environmental change — changes to the operating environment such as the OS, database, or a third-party service's interface, even when users see no new feature.

The scope of maintenance testing depends on the degree of risk of the change, the size of the existing system, and the size of the change. A small, isolated fix may need little regression testing; a large change to a large, high-risk system may need extensive maintenance testing.

Impact Analysis and the Role of Automation

Impact analysis evaluates the changes made in a maintenance release to identify the intended consequences and the expected and possible side effects of a change, and to identify the areas in the system that will be affected. It also helps identify the impact on existing tests. Impact analysis drives regression test selection and effort estimation. It is hard when specifications are out of date or missing, and it is far more accurate when traceability links requirements, design, code, tests, defects, and results — so a change can be traced to exactly the tests that should be re-run.

Because regression suites run repeatedly, they are the strongest candidates for test automation, which keeps repeated execution affordable. But automation never decides what matters: a stale automated suite can miss the new risk a change introduces. Testers must still review the change, the affected architecture, and defect history to choose and update the regression set.

For exam questions, anchor on the trigger: a specific defect re-verified → confirmation; unchanged functionality re-checked after a change → regression; a released system modified, migrated, retired, or its environment changed → maintenance; analysis of which areas are affected → impact analysis.

Building and Maintaining a Regression Suite

A practical question CTFL touches on is how a team keeps regression testing affordable as the system grows. A naive approach re-runs everything after every change, which becomes prohibitively slow. Instead, teams use risk-based selection guided by impact analysis to choose which regression tests to run, and they continuously prune and refactor the suite: removing tests for retired features, merging duplicates, and adding new regression tests whenever a change creates a new area worth guarding.

A regression suite is itself an asset that needs maintenance — an unmaintained suite drifts out of date, producing false confidence (it passes but no longer covers current risk) or noisy false failures that erode trust.

Distinguishing the Four Concepts on the Exam

Because these four terms are so close, the exam frequently offers all four as options for one scenario. * If a system is unchanged and still in active development, none of these apply. * Re-running the exact failed case proves the fix → confirmation. Checking other, unchanged areas for side effects → regression. If the change is to an operational system (post-release modification, migration, retirement, or environment change), the umbrella activity is maintenance testing, which itself uses confirmation and regression testing internally. And the analytical step that decides which areas to retest is impact analysis.

Anchoring on "change or not" and then "verify the fix vs. guard the rest" resolves nearly every question in this section.