Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FEAT - information about partitions are now logged #251

Closed
wants to merge 17 commits into from

Conversation

marcopeix
Copy link
Contributor

The number of partitions is logged
We also log which partition is being treated

Screenshot 2024-03-14 at 2 42 49 PM

Copy link

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

Copy link
Contributor

Experiment Results

Experiment 1: air-passengers

Description:

variable experiment
h 12
season_length 12
freq MS
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 12.6793 11.0623 47.8333 76
mape 0.027 0.0232 0.0999 0.1425
mse 213.936 199.132 2571.33 10604.2
total_time 3.515 2.9661 0.0106 0.0063

Plot:

Experiment 2: air-passengers

Description:

variable experiment
h 24
season_length 12
freq MS
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 58.1031 58.4587 71.25 115.25
mape 0.1257 0.1267 0.1552 0.2358
mse 4040.21 4110.79 5928.17 18859.2
total_time 9.2055 5.7553 0.0076 0.0059

Plot:

Experiment 3: electricity-multiple-series

Description:

variable experiment
h 24
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 142.394 196.363 269.23 1331.02
mape 0.0203 0.0234 0.0304 0.1692
mse 63464.7 123119 213677 4.68961e+06
total_time 14.0936 7.0186 0.0101 0.0085

Plot:

Experiment 4: electricity-multiple-series

Description:

variable experiment
h 168
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 522.427 353.528 398.956 1119.26
mape 0.069 0.0454 0.0512 0.1583
mse 966294 422332 656723 3.17316e+06
total_time 5.2437 3.8524 0.0097 0.0089

Plot:

Experiment 5: electricity-multiple-series

Description:

variable experiment
h 336
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 478.362 361.033 602.926 1340.95
mape 0.0622 0.046 0.0787 0.17
mse 805039 441118 1.61572e+06 6.04619e+06
total_time 5.6951 4.7243 0.0101 0.0087

Plot:

@marcopeix marcopeix linked an issue Mar 14, 2024 that may be closed by this pull request
Copy link
Contributor

Experiment Results

Experiment 1: air-passengers

Description:

variable experiment
h 12
season_length 12
freq MS
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 12.6793 11.0623 47.8333 76
mape 0.027 0.0232 0.0999 0.1425
mse 213.936 199.132 2571.33 10604.2
total_time 8.0299 8.0084 0.0095 0.0055

Plot:

Experiment 2: air-passengers

Description:

variable experiment
h 24
season_length 12
freq MS
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 58.1031 58.4587 71.25 115.25
mape 0.1257 0.1267 0.1552 0.2358
mse 4040.21 4110.79 5928.17 18859.2
total_time 10.8493 9.4008 0.0066 0.0057

Plot:

Experiment 3: electricity-multiple-series

Description:

variable experiment
h 24
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 142.394 196.363 269.23 1331.02
mape 0.0203 0.0234 0.0304 0.1692
mse 63464.7 123119 213677 4.68961e+06
total_time 14.0182 12.5868 0.0089 0.0097

Plot:

Experiment 4: electricity-multiple-series

Description:

variable experiment
h 168
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 522.427 353.528 398.956 1119.26
mape 0.069 0.0454 0.0512 0.1583
mse 966294 422332 656723 3.17316e+06
total_time 14.2574 11.8714 0.0085 0.0079

Plot:

Experiment 5: electricity-multiple-series

Description:

variable experiment
h 336
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 478.362 361.033 602.926 1340.95
mape 0.0622 0.046 0.0787 0.17
mse 805039 441118 1.61572e+06 6.04619e+06
total_time 13.5598 16.0153 0.0088 0.0081

Plot:

Copy link
Contributor

Experiment Results

Experiment 1: air-passengers

Description:

variable experiment
h 12
season_length 12
freq MS
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 12.6793 11.0623 47.8333 76
mape 0.027 0.0232 0.0999 0.1425
mse 213.936 199.132 2571.33 10604.2
total_time 7.3633 5.7768 0.0096 0.0054

Plot:

Experiment 2: air-passengers

Description:

variable experiment
h 24
season_length 12
freq MS
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 58.1031 58.4587 71.25 115.25
mape 0.1257 0.1267 0.1552 0.2358
mse 4040.21 4110.79 5928.17 18859.2
total_time 6.8443 9.8291 0.0063 0.0056

Plot:

Experiment 3: electricity-multiple-series

Description:

variable experiment
h 24
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 142.394 196.363 269.23 1331.02
mape 0.0203 0.0234 0.0304 0.1692
mse 63464.7 123119 213677 4.68961e+06
total_time 8.2676 12.9297 0.0093 0.008

Plot:

Experiment 4: electricity-multiple-series

Description:

variable experiment
h 168
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 522.427 353.528 398.956 1119.26
mape 0.069 0.0454 0.0512 0.1583
mse 966294 422332 656723 3.17316e+06
total_time 11.5545 11.6361 0.0085 0.0082

Plot:

Experiment 5: electricity-multiple-series

Description:

variable experiment
h 336
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 478.362 361.033 602.926 1340.95
mape 0.0622 0.046 0.0787 0.17
mse 805039 441118 1.61572e+06 6.04619e+06
total_time 13.1415 12.0035 0.0087 0.0081

Plot:

Copy link
Contributor

Experiment Results

Experiment 1: air-passengers

Description:

variable experiment
h 12
season_length 12
freq MS
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 12.6793 11.0623 47.8333 76
mape 0.027 0.0232 0.0999 0.1425
mse 213.936 199.132 2571.33 10604.2
total_time 3.2549 2.802 0.0091 0.0053

Plot:

Experiment 2: air-passengers

Description:

variable experiment
h 24
season_length 12
freq MS
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 58.1031 58.4587 71.25 115.25
mape 0.1257 0.1267 0.1552 0.2358
mse 4040.21 4110.79 5928.17 18859.2
total_time 4.2498 9.2307 0.0062 0.0054

Plot:

Experiment 3: electricity-multiple-series

Description:

variable experiment
h 24
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 142.394 196.363 269.23 1331.02
mape 0.0203 0.0234 0.0304 0.1692
mse 63464.7 123119 213677 4.68961e+06
total_time 5.9432 5.9636 0.0087 0.0074

Plot:

Experiment 4: electricity-multiple-series

Description:

variable experiment
h 168
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 522.427 353.528 398.956 1119.26
mape 0.069 0.0454 0.0512 0.1583
mse 966294 422332 656723 3.17316e+06
total_time 6.915 6.7298 0.008 0.0075

Plot:

Experiment 5: electricity-multiple-series

Description:

variable experiment
h 336
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 478.362 361.033 602.926 1340.95
mape 0.0622 0.046 0.0787 0.17
mse 805039 441118 1.61572e+06 6.04619e+06
total_time 5.9302 5.5822 0.0083 0.0077

Plot:

Copy link
Contributor

github-actions bot commented Apr 3, 2024

Experiment Results

Experiment 1: air-passengers

Description:

variable experiment
h 12
season_length 12
freq MS
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 12.6793 11.0623 47.8333 76
mape 0.027 0.0232 0.0999 0.1425
mse 213.935 199.132 2571.33 10604.2
total_time 18.1344 7.6626 0.0087 0.0052

Plot:

Experiment 2: air-passengers

Description:

variable experiment
h 24
season_length 12
freq MS
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 58.1031 58.4587 71.25 115.25
mape 0.1257 0.1267 0.1552 0.2358
mse 4040.22 4110.79 5928.17 18859.2
total_time 4.049 4.8514 0.0069 0.0055

Plot:

Experiment 3: electricity-multiple-series

Description:

variable experiment
h 24
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 142.394 196.363 269.23 1331.02
mape 0.0203 0.0234 0.0304 0.1692
mse 63464.7 123119 213677 4.68961e+06
total_time 6.2692 7.0321 0.0083 0.0075

Plot:

Experiment 4: electricity-multiple-series

Description:

variable experiment
h 168
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 522.427 353.528 398.956 1119.26
mape 0.069 0.0454 0.0512 0.1583
mse 966295 422332 656723 3.17316e+06
total_time 6.8316 5.6233 0.008 0.0075

Plot:

Experiment 5: electricity-multiple-series

Description:

variable experiment
h 336
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 478.362 361.033 602.926 1340.95
mape 0.0622 0.046 0.0787 0.17
mse 805038 441118 1.61572e+06 6.04619e+06
total_time 8.7468 4.671 0.0084 0.0078

Plot:

Copy link
Contributor

github-actions bot commented Apr 3, 2024

Experiment Results

Experiment 1: air-passengers

Description:

variable experiment
h 12
season_length 12
freq MS
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 12.6793 11.0623 47.8333 76
mape 0.027 0.0232 0.0999 0.1425
mse 213.935 199.132 2571.33 10604.2
total_time 3.4083 3.1566 0.0098 0.0059

Plot:

Experiment 2: air-passengers

Description:

variable experiment
h 24
season_length 12
freq MS
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 58.1031 58.4587 71.25 115.25
mape 0.1257 0.1267 0.1552 0.2358
mse 4040.21 4110.79 5928.17 18859.2
total_time 4.0629 2.9606 0.0065 0.0057

Plot:

Experiment 3: electricity-multiple-series

Description:

variable experiment
h 24
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 142.394 196.363 269.23 1331.02
mape 0.0203 0.0234 0.0304 0.1692
mse 63464.8 123119 213677 4.68961e+06
total_time 3.5848 4.5746 0.0089 0.0081

Plot:

Experiment 4: electricity-multiple-series

Description:

variable experiment
h 168
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 522.427 353.528 398.956 1119.26
mape 0.069 0.0454 0.0512 0.1583
mse 966295 422332 656723 3.17316e+06
total_time 5.4745 4.0266 0.0084 0.0078

Plot:

Experiment 5: electricity-multiple-series

Description:

variable experiment
h 336
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 478.362 361.033 602.926 1340.95
mape 0.0622 0.046 0.0787 0.17
mse 805038 441118 1.61572e+06 6.04619e+06
total_time 8.0434 4.6146 0.0091 0.0082

Plot:

nixtlats/nixtla_client.py Outdated Show resolved Hide resolved
Copy link
Member

@AzulGarza AzulGarza left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

awesome @marcopeix! i left a minor comment. thank you! 🙌

nixtlats/nixtla_client.py Outdated Show resolved Hide resolved
Copy link
Contributor

Experiment Results

Experiment 1: air-passengers

Description:

variable experiment
h 12
season_length 12
freq MS
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 12.6793 11.0623 47.8333 76
mape 0.027 0.0232 0.0999 0.1425
mse 213.936 199.132 2571.33 10604.2
total_time 8.0796 9.6798 0.0091 0.0055

Plot:

Experiment 2: air-passengers

Description:

variable experiment
h 24
season_length 12
freq MS
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 58.1031 58.4587 71.25 115.25
mape 0.1257 0.1267 0.1552 0.2358
mse 4040.21 4110.79 5928.17 18859.2
total_time 7.3228 6.9225 0.006 0.0053

Plot:

Experiment 3: electricity-multiple-series

Description:

variable experiment
h 24
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 142.394 196.363 269.23 1331.02
mape 0.0203 0.0234 0.0304 0.1692
mse 63464.7 123119 213677 4.68961e+06
total_time 7.4321 12.1105 0.0083 0.0075

Plot:

Experiment 4: electricity-multiple-series

Description:

variable experiment
h 168
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 522.427 353.528 398.956 1119.26
mape 0.069 0.0454 0.0512 0.1583
mse 966294 422332 656723 3.17316e+06
total_time 9.5104 9.1816 0.0081 0.0076

Plot:

Experiment 5: electricity-multiple-series

Description:

variable experiment
h 336
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 478.362 361.033 602.926 1340.95
mape 0.0622 0.046 0.0787 0.17
mse 805039 441118 1.61572e+06 6.04619e+06
total_time 10.7586 6.3356 0.0082 0.0077

Plot:

@AzulGarza AzulGarza self-requested a review April 17, 2024 23:25
nixtlats/nixtla_client.py Outdated Show resolved Hide resolved
Copy link
Contributor

Experiment Results

Experiment 1: air-passengers

Description:

variable experiment
h 12
season_length 12
freq MS
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 12.6793 11.0623 47.8333 76
mape 0.027 0.0232 0.0999 0.1425
mse 213.936 199.132 2571.33 10604.2
total_time 5.757 5.8825 0.0106 0.0054

Plot:

Experiment 2: air-passengers

Description:

variable experiment
h 24
season_length 12
freq MS
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 58.1031 58.4587 71.25 115.25
mape 0.1257 0.1267 0.1552 0.2358
mse 4040.22 4110.79 5928.17 18859.2
total_time 7.7928 7.0805 0.0062 0.0053

Plot:

Experiment 3: electricity-multiple-series

Description:

variable experiment
h 24
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 142.394 196.363 269.23 1331.02
mape 0.0203 0.0234 0.0304 0.1692
mse 63464.7 123119 213677 4.68961e+06
total_time 6.985 6.4391 0.0084 0.0074

Plot:

Experiment 4: electricity-multiple-series

Description:

variable experiment
h 168
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 522.427 353.528 398.956 1119.26
mape 0.069 0.0454 0.0512 0.1583
mse 966294 422332 656723 3.17316e+06
total_time 8.2208 6.9104 0.0083 0.0076

Plot:

Experiment 5: electricity-multiple-series

Description:

variable experiment
h 336
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 478.362 361.033 602.926 1340.95
mape 0.0622 0.046 0.0787 0.17
mse 805039 441118 1.61572e+06 6.04619e+06
total_time 8.4482 6.714 0.0085 0.0077

Plot:

Copy link
Contributor

Experiment Results

Experiment 1: air-passengers

Description:

variable experiment
h 12
season_length 12
freq MS
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 12.6793 11.0623 47.8333 76
mape 0.027 0.0232 0.0999 0.1425
mse 213.936 199.132 2571.33 10604.2
total_time 11.1353 8.0454 0.012 0.0089

Plot:

Experiment 2: air-passengers

Description:

variable experiment
h 24
season_length 12
freq MS
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 58.1031 58.4587 71.25 115.25
mape 0.1257 0.1267 0.1552 0.2358
mse 4040.21 4110.79 5928.17 18859.2
total_time 5.6513 4.0269 0.0064 0.0057

Plot:

Experiment 3: electricity-multiple-series

Description:

variable experiment
h 24
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 142.394 196.363 269.23 1331.02
mape 0.0203 0.0234 0.0304 0.1692
mse 63464.7 123119 213677 4.68961e+06
total_time 10.1675 8.4851 0.0093 0.0085

Plot:

Experiment 4: electricity-multiple-series

Description:

variable experiment
h 168
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 522.427 353.528 398.956 1119.26
mape 0.069 0.0454 0.0512 0.1583
mse 966294 422332 656723 3.17316e+06
total_time 8.0019 7.1876 0.0086 0.0081

Plot:

Experiment 5: electricity-multiple-series

Description:

variable experiment
h 336
season_length 24
freq H
level None
n_windows 1

Results:

metric timegpt-1 timegpt-1-long-horizon SeasonalNaive Naive
mae 478.362 361.033 602.926 1340.95
mape 0.0622 0.046 0.0787 0.17
mse 805039 441118 1.61572e+06 6.04619e+06
total_time 9.2113 10.1315 0.0084 0.0077

Plot:

@marcopeix
Copy link
Contributor Author

Is this feature still relevant since partitions are now processed in parallel? The current output looks like this:
Screenshot 2024-04-19 at 10 24 46 AM

@mergenthaler
Copy link
Member

@marcopeix, closing this for grooming. If you feel it should be kept open, please re-open and correct.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[FEAT] Inform users when num_partitions>1
3 participants