Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: note on chronos pr #390

Merged
merged 1 commit into from
Jun 10, 2024
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions experiments/foundation-time-series-arena/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

> TL;DR: Foundation models for time series outperform alternatives and are ready to be tested in production. TimeGPT-1 is (so far) the most accurate and fastest model but TimesFM from Google comes very close. Some models are still outperformed by classical alternatives.

Note: The Amazon team responded to the original benchmark with this [PR](https://github.com/Nixtla/nixtla/pull/382) that shows, according to them, that by changing some parameters, Chronos is significantly faster and more accurate. We are currently reviewing the PR.

# Introduction

Expand All @@ -20,6 +21,8 @@ We at Nixtla have provided some [early success stories](https://techcommunity.mi

However, the field [is still divided](https://news.ycombinator.com/item?id=39235983) on how all the different foundation models compare against each other. In the spirit of collaboration, we are starting a new project, `xiuhmolpilli`, in honor of how ancient civilizations celebrated the end of cycles, to build a benchmark to compare all the different foundation models for time series data in a large scale dataset and against classical, ML and Deep Learning Models.



# Empirical Evaluation

This study considers **over 30,000 unique time series** from the Monash Repository, M-Competitions, Wikipedia page views, among others, spanning various time series frequencies: Monthly, Weekly, Daily, and Hourly. Our evaluation compares five foundation models for time series data in terms of accuracy and inference times. We have also included comparisons to a large battery of statistical, machine learning, and deep-learning models, to provide a benchmark against traditional forecasting methods.
Expand Down
Loading