Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add Differentiable Variant of SMAPE Performance Measure #221

Closed
antoinecarme opened this issue Feb 25, 2023 · 3 comments
Closed

Add Differentiable Variant of SMAPE Performance Measure #221

antoinecarme opened this issue Feb 25, 2023 · 3 comments
Assignees
@antoinecarme
Labels
class:enhancement priority:high topic:generic topic:modeling_quality

Comments

@antoinecarme
Copy link
Owner

antoinecarme commented Feb 25, 2023 •
edited
Loading

PyAF can be configured to use MAPE, SMAPE and many other performance measures to perform model selection.

MAPE and SMAPE are sensitive to zero values in the signal/forecast.

Differentiable SMAPE (Suilin, 2017) is a recent variant of the SMAPE that makes it not only smooth, differentiable but also more resistant to zero values.

Found in : https://cbergmeir.com/talks/FFDS_ACML2020.pdf (page 104)

image

Original source code :
https://www.kaggle.com/code/asuilin/smape-variants/notebook
https://www.kaggle.com/c/web-traffic-time-series-forecasting/discussion/39529

To be included in the next PyAF release (5.0, expected on 2023-07-14).
@antoinecarme antoinecarme mentioned this issue Feb 25, 2023
Closed
antoinecarme added a commit that referenced this issue Feb 25, 2023
@antoinecarme
Add Differentiable Variant of SMAPE Performance Measure #221. Added t…
7343071 
…wo test scripts.
antoinecarme added a commit that referenced this issue Feb 25, 2023
@antoinecarme
Add Differentiable Variant of SMAPE Performance Measure #221. Added n…
e6dd0e2 
…ew perf measure.
antoinecarme added a commit that referenced this issue Feb 25, 2023
@antoinecarme
Add Differentiable Variant of SMAPE Performance Measure #221. Added n…
d1f2450 
…ew perf measure values in the logs.
@antoinecarme
Copy link
Owner Author

Ozone model with MAPE (default)

image

@antoinecarme
Copy link
Owner Author

Ozone with differentiable SMAPE

image

@antoinecarme antoinecarme self-assigned this Feb 26, 2023
@antoinecarme
Copy link
Owner Author

FIXED.

Closing.

@antoinecarme antoinecarme mentioned this issue Mar 9, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@antoinecarme antoinecarme

Labels
class:enhancement priority:high topic:generic topic:modeling_quality
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@antoinecarme