Choosing the correct error metric: MAPE vs. sMAPE (2024)

FAQs

What is the difference between sMAPE and MAPE? ›

MAPE might reach arbitrary big values, while sMAPE will have an upper bound (either 200 or 100, depending on the implementation). Both metrics are known to assign unequal weights to overshooting and undershooting. MAPE and sMAPE both don't have continuous derivatives and have issues with values being close to 0.

MAPE is the single WORST forecasting KPI. Here's why MAPE rewards under forecasting. It heavily penalizes forecast errors during periods of low demand Each forecast error is divided by the corresponding demand to get a percentage. If demand is low, you get a high percentage error.

sMAPE can take negative values, so the interpretation of an “absolute percentage error” can be misleading. The range of 0% to 200% is not that intuitive to interpret, hence often the division by the 2 in the denominator of the sMAPE formula is omitted.

A smaller value of SMAPE is better, and it is often multiplied by 100% to obtain the percentage error. Best possible score is 0.0, smaller value is better. Range = [0, 1].

Metric: smape

Symmetric Mean Absolute Percentage Error (sMAPE) is the symmetric mean percentage error difference between the predicted and actual values as defined by Chen and Yang (2004), based on the metric by Armstrong (1985) and Makridakis (1993).

Removing negative values from the equation (see chart below) allows the accuracy to be calculated without positive and negative numbers canceling each other out. This makes forecasting reliable and easy to understand, which is why MAPE is the most used method in measuring the accuracy of forecasts.

MAPE is an error metric that is easy to understand because it provides the error in terms of percentages. A lower value of MAPE implies a higher accuracy of the model. A significant disadvantage of MAPE is that it produces undefined values when the actual values are 0, which is a common occurrence in some fields.

Root Mean Squared Error (RMSE) is a widely used metric for evaluating time series forecasting models.

Models with MAPE value less than 10% are very good, between 10% and 20% are good, between 20% and 50% are acceptable, and models with MAPE value more than 50% are classified as falsely predicted. The estimated ANN model is at an acceptable level (Moreno et al., 2013; Shrestha at al., 2021).

WMAPE (weighted mean absolute % error)

Weighted Mean Absolute Percentage Error, as the name suggests, is a measure that weights the errors by product volume, thus overcoming one of the main drawbacks of MAPE. There is a very easy way to calculate WMAPE.

What is the major problem with using MAPE as a way of evaluating forecasts? ›

The mean absolute percentage error (MAPE) is one of the most widely used measures of forecast accuracy, due to its advantages of scale-independency and interpretability. However, MAPE has the significant disadvantage that it produces infinite or undefined values for zero or close-to-zero actual values.

The resulting SMAPE values range from 0% to 100%, with lower values indicating higher accuracy. A SMAPE value of 0% signifies a perfect alignment between the forecasted and actual values, while a value of 100% implies no similarity between the two.

Symmetric mean absolute percentage error (SMAPE or sMAPE) is an accuracy measure based on percentage (or relative) errors.

1 Answer. so MAPE >100% means that the errors are "much greater" then the actual values (e.g. actual is 1, you predict 3, so MAPE is 200%). However beware that MAPE has many pitfalls as error measure, so often it won't be the best choice.

One of the most common metrics of model prediction accuracy, mean absolute percentage error (MAPE) is the percentage equivalent of mean absolute error (MAE). Mean absolute percentage error measures the average magnitude of error produced by a model, or how far off predictions are on average.

The symmetric mean absolute percentage error (SMAPE) is an accuracy measure based on percentage (or relative) errors. Relative error is the absolute error divided by the magnitude of the exact value. In contrast to the mean absolute percentage error, SMAPE has both a lower bound and an upper bound.

Forecast accuracy is the measure of how accurately a given forecast matches actual sales. Forecast bias describes how much the forecast is consistently over or under the actual sales. Common metrics used to evaluate forecast accuracy include Mean Absolute Percentage Error (MAPE) and Mean Absolute Deviation (MAD).