5.3 Forecast Performance

A forecast that is never measured never improves. CPIM Domain III expects you to compute and interpret core error metrics, recognize when a model is drifting, keep hierarchical forecasts consistent, and understand how actual orders consume the forecast inside master planning.

Forecast Error Basics

For any period t:

e(t) = A(t) − F(t)

Sign convention matters for bias. If actuals consistently exceed forecasts, errors are positive and the forecast is biased low (under-forecasting). If forecasts exceed actuals, errors are negative and the forecast is biased high (over-forecasting). Some texts define error as F - A; on the exam, read the stem carefully and stay consistent with the formula given.

Bias (Mean Error)

Bias (mean forecast error) over n periods:

Bias = (Σ e(t)) / n = (Σ (A(t) − F(t))) / n

Worked example. Four periods:

Sum of errors = 10 + 10 − 10 + 20 = +30. Bias = 30 / 4 = +7.5. On average the forecast is 7.5 units too low. Bias near zero can still hide large misses that cancel (over one month, under the next), so bias alone is incomplete.

MAD — Mean Absolute Deviation

MAD = (Σ |e(t)|) / n

Using the same table, absolute errors are 10, 10, 10, 20. Sum = 50. MAD = 50 / 4 = 12.5. MAD answers “how large are typical errors?” regardless of direction. Safety-stock formulas in many APICS contexts use MAD (or a related σ estimate) as the variability input.

MAPE — Mean Absolute Percentage Error

MAPE = (100% / n) × Σ | (A(t) − F(t)) / A(t) |

(undefined if any actual is zero—another exam trap).

For period 1: |100−90|/100 = 0.10. Period 2: 10/110 ≈ 0.0909. Period 3: 10/95 ≈ 0.1053. Period 4: 20/120 ≈ 0.1667. Average ≈ 0.1157 → MAPE ≈ 11.6%.

MAPE lets you compare a high-volume SKU (errors of 500 units) with a low-volume SKU (errors of 5 units) on a common percentage scale. It can overweight periods with tiny actuals, so interpret cautiously for intermittent demand.

Tracking Signal — Worked Example

The tracking signal (TS) monitors whether bias is accumulating beyond what random error would explain:

TS = (Σ e(t)) / MAD

(Algebraic sum of errors, sometimes called the running sum of forecast errors / RSFE, divided by MAD.)

Continuing the example: RSFE = +30, MAD = 12.5, so TS = 30 / 12.5 = +2.4.

Organizations set control limits such as ±3 or ±4. If TS exceeds the limit, the forecast is out of control—investigate level shifts, missing seasonality, or broken promotions assumptions; do not silently keep using the model. A TS near zero with a high MAD means errors are large but not systematically one-sided.

MSE, CFE, and Demand Filters

The ECM names three more error tools that candidates routinely skip.

Mean squared error (MSE) squares each error before averaging:

MSE = (Σ e(t)²) / n

Using the same four periods (errors +10, +10, −10, +20): 100 + 100 + 100 + 400 = 700, so MSE = 700 / 4 = 175. Taking the square root gives the root mean squared error (RMSE) ≈ 13.2 units, which is back in the original unit of measure. Because squaring inflates large misses, MSE punishes a single big miss far more than several small ones. Compare it with MAD = 12.5 on the identical data: MAD treats a 20-unit error as twice a 10-unit error, while MSE treats it as four times worse. Choose MSE when one catastrophic miss hurts more than steady small noise; choose MAD when all misses hurt proportionally.

Cumulative forecast error (CFE) is the running algebraic sum of errors — the same quantity as the RSFE in the tracking-signal numerator:

CFE = Σ e(t)

Here CFE = +30 units. A CFE that drifts steadily away from zero is the signature of bias; a CFE that oscillates around zero is random noise. This is why TS = CFE / MAD: it scales accumulated bias against typical error size.

Demand filters work in the opposite direction — they screen the incoming actual before it corrupts history. A demand filter sets a threshold, commonly a multiple of MAD (3 × MAD or 4 × MAD is typical), beyond which an actual demand value is held for human review rather than absorbed automatically.

With MAD = 12.5 and a 4 × MAD filter, the tripwire is 50 units from forecast. A period that forecasts 100 and books 165 (error +65) breaks the filter and gets reviewed: was it a genuine market shift, a one-time tender, or a data-entry error of 165 for 16.5? Without the filter, that single record inflates the average, raises the safety stock calculation, and pushes a phantom requirement through MRP.

Pyramid Forecasting

Pyramid forecasting maintains consistency across hierarchy levels (SKU → family → category → total business):

1. Force-down / top-down — management sets a family or business total; the system disaggregates to SKUs using historical mix percentages or planning bills.
2. Bottom-up — SKU forecasts sum to the family total.
3. Middle-out / reconciliation — both directions are prepared, then adjusted so the pyramid adds up.

Without a pyramid discipline, marketing’s family outlook of 10,000 units can coexist with SKU forecasts that sum to 12,500—guaranteeing either phantom shortages or excess. CPIM emphasizes that aggregation reduces noise (family forecasts are often more accurate percentage-wise) while disaggregation is required for execution at SKU-location. Pyramid logic is how you get both.

Forecast Consumption by Actual Orders

In master scheduling, the demand time fence and consumption logic determine how customer orders interact with the forecast:

• As orders book in a period, they consume (reduce) the remaining forecast for that period so total demand is not forecast + orders.
• If orders exceed the forecast, many systems display the overage as additional demand (and may consume adjacent periods depending on rules).
• Inside the demand time fence, management often relies more on orders than on forecast; outside the fence, forecast dominates.

Simple illustration. Week 5 forecast = 100. By Monday, firm orders = 40. Remaining forecast to plan = 60 (unless policy says otherwise), and total projected demand still centers on 100 unless orders exceed it. If firm orders reach 120, projected customer demand becomes 120—not 220.

Consumption rules vary by software, but the principle is universal: never double-count. Planners who add bookings on top of an unconsumed forecast inflate the MPS, overload capacity, and then wonder why inventory explodes when the “extra” never was real incremental demand.

Closing the Performance Loop

Measure bias, MAD, and MAPE by item family on a regular cadence. Watch tracking signals for model failure. Reconcile hierarchies with pyramid forecasting. Configure consumption so actual orders displace forecast cleanly. Together these practices turn forecasting from a monthly argument into a controlled process—and that is exactly the competence Domain III assesses.