-
Notifications
You must be signed in to change notification settings - Fork 73
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Use MASE by default for PyAF Model Selection #229
Comments
https://otexts.com/fpp3/accuracy.html https://www.sciencedirect.com/science/article/abs/pii/S0169207006000239?via%3Dihub Another look at measures of forecast accuracy |
Also add the same scaling for RMSE, the RMSSE as a new performance measure : |
…sures computations. Allow caching some intermediate results. Added RMSSE.
…et. Model Not OK with MAPE, performs much better with MASE, RMSE., RMSSE
…et. Model Not OK with MAPE, performs much better with MASE, RMSE., RMSSE
…et. Model Not OK with MAPE, performs much better with MASE, RMSE, RMSSE. Updated Makefile
…et. Model Not OK with MAPE, performs much better with MASE, RMSE, RMSSE. Added new references.
Origin : user model on google colab (#PyAF hashtag rocks ;). https://colab.research.google.com/drive/1zaVQuobR8M63qB-UDDX8ZX37ctl98YIT?usp=sharing |
CLOSING. Added to 5.0 |
… Use np.diff to precompute naive error
… Remove some unnecessay abs ops.
When the signal contains zeros, the MAPE values are not defined. MAPE is simply not suitable for such signals ans this can lead to low quality models.
https://stackoverflow.com/questions/41571215/forecasr-accuracy-mape-and-zero-values
https://otexts.com/fpp3/accuracy.html
There is no technical work-around for better using MAPE.
MAPE is not suitable for reporting performance measure values when zeros in the signal, but it is very user-friendly and easy to understand..
The only solution is to use a scaled measure like MASE for model selection by default.
Some benchmarking is needed (#222 ).
It would be nice to have this in PyAF 5.0 (expected on 2023-07-14). So far, so good. DONE.
The text was updated successfully, but these errors were encountered: