Forecast Accuracy Metrics: Which One Should You Use?

Forecast accuracy metrics are the scorecards of demand planning, and choosing the wrong one can send a team in the wrong direction for years. There is no single best measure. Each answers a slightly different question, behaves badly in particular situations and suits particular decisions. The aim of this guide is to explain the common measures in plain language, apply all of them to one small data set so that you can see how they differ, and give a simple rule for choosing. It is written for operations, supply chain and finance leaders at product companies who need a practical standard for their planning team. It is part of demand planning and inventory optimization, follows demand planning KPIs and statistical forecasting, and leads to MAPE and WMAPE. Finance teams measure forecasts in a similar way, as in measuring financial forecast accuracy, and accuracy drives the cash tied up in stock, covered in cash flow management.

The data

Four items, one period (illustrative):

Mean absolute error (MAE)

Add up the absolute errors and divide by the number of items. Here: 45 / 4 = 11.25 units. MAE is easy to understand and in the units of the item. Its weakness is that it is hard to compare across items of different size: an error of 10 units is huge for an item that sells 20 and trivial for one that sells 2,000.

Mean absolute percentage error (MAPE)

Average the absolute error as a percentage of each item's actual. P1: 10 / 100 = 10 percent. P2: 10 / 20 = 50 percent. P3: 20 / 400 = 5 percent. P4 has an actual of zero, so the percentage is undefined and it is usually excluded or handled by a rule. The mean of the three defined values is (10 + 50 + 5) / 3 = 65 / 3 = 21.7 percent. MAPE is popular because it is a percentage, but it gives a small item such as P2 the same weight as a large one such as P3, and it breaks when actuals are zero. See MAPE.

Weighted absolute percentage error (WAPE)

Divide the total absolute error by the total actuals: 45 / 520 = 8.65 percent. WAPE weights items by volume, so a large item matters more than a small one. It handles items with zero actuals as long as the total is not zero. For uneven portfolios it is often the best single accuracy measure. Some people quote accuracy as 100 percent minus WAPE, which would be 91.4 percent here. See WMAPE.

Root mean squared error (RMSE)

Square each error, average the squares and take the square root. Squared errors: 100, 100, 400 and 25, which total 625. The mean is 625 / 4 = 156.25 and its square root is 12.5 units. RMSE punishes large errors more than small ones, which is useful when a big miss is much more costly than several small ones, as in safety stock calculations, where the standard deviation of error matters. It is in units, so it shares MAE's problem with comparing across items.

Bias

Total forecast minus total actual, divided by total actual: (525 - 520) / 520 = +0.96 percent. The forecast is slightly high overall. Bias shows direction, which the other measures do not. Notice that the individual errors were +10, +10, -20 and +5: the over-forecasts on P1, P2 and P4 are partly offset by the under-forecast on P3. A bias near zero can coexist with substantial error, which is why bias must be read alongside WAPE.

Tracking signal

The tracking signal detects persistent bias over time. It equals the cumulative forecast error divided by the mean absolute deviation. Suppose an item's monthly errors over six months are +10, +8, +12, +9, +11 and +10. The cumulative error is 60 and the mean absolute deviation is (10 + 8 + 12 + 9 + 11 + 10) / 6 = 10. The tracking signal is 60 / 10 = 6.0. A common alert threshold is plus or minus 4, so this item is flagged: it has been over-forecast every month and the model needs attention. The tracking signal is a monitoring tool for individual items.

Scale-free alternatives

The mean absolute scaled error (MASE) divides the forecast's mean absolute error by the mean absolute error of a naive forecast on the same data. A value below 1 means the forecast beats the naive benchmark, and a value above 1 means it does not. MASE works with zeros and can be compared across items. It is less intuitive for business audiences, so many companies use it internally and report WAPE externally. The symmetric MAPE tries to fix MAPE's asymmetry by dividing by the average of actual and forecast, but has its own quirks.

How the measures compare on this data

The same forecast looks like 8.65 percent error or 21.7 percent error depending on the measure. Neither is wrong. They answer different questions, and the discipline is to pick one as the standard and understand its behavior.

How to choose

For a company-wide headline and comparisons across categories, use WAPE, because it is volume-weighted, handles zeros and is easy to explain. For direction, always add bias. For setting safety stock, use the standard deviation of error (related to RMSE) at the lead time. For comparing methods fairly across many items, consider MASE or the relative error against a naive forecast. For item-level monitoring, use the tracking signal. Avoid MAPE as a sole measure if your portfolio has many small or intermittent items.

Measure at the right level and lag

An accuracy number is only meaningful with the level (item, family, total), the location and the lag (how far ahead the forecast was made) specified. Measure item-by-location accuracy at the lag of the lead time, because that is what drove the order. Aggregate accuracy will always look better than item accuracy, since errors offset. Use the aggregate for planning conversations with finance, and the detailed measure for operational improvement.

Compare to a benchmark

An accuracy number means little without a benchmark. Compare the forecast with a naive forecast, such as last period or the same period last year. If your WAPE is 8.65 percent and the naive forecast scores 12 percent, the process adds value. If the naive forecast scores 8 percent, it does not. This is the basis of forecast value added.

Common mistakes

1. Using MAPE on items with zeros or small volumes.
2. Reporting accuracy without bias.
3. Averaging percentages across items of different size.
4. Measuring at the wrong lag.
5. Changing the metric definition midstream.
6. No benchmark.

FAQs

Which forecast accuracy metric is best?

There is no single best. WAPE is a strong default for headline accuracy, and bias should always accompany it. Use RMSE-based measures for safety stock.

What is a good forecast accuracy?

It depends on the product, level and lag. Compare with your own history and a naive forecast, and set targets by segment.

Why does MAPE give a different answer from WAPE?

MAPE averages item percentages, giving small items equal weight. WAPE divides total error by total actuals, so large items count more.

Should I include bias?

Yes. Bias shows whether errors lean high or low, which drives chronic excess or shortage.

What is the tracking signal?

The cumulative forecast error divided by the mean absolute deviation. Values beyond about plus or minus 4 suggest a persistent bias.