Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BugFix] Fix tictactoeenv.py #2417

Merged
merged 1 commit into from
Sep 4, 2024
Merged

[BugFix] Fix tictactoeenv.py #2417

merged 1 commit into from
Sep 4, 2024

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Sep 4, 2024 •
edited
Loading

Stack from ghstack (oldest at bottom):

@vmoens
Update
bd4690a 
[ghstack-poisoned]
@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Sep 4, 2024 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2417

Note: Links to docs will display an error until the docs builds have been completed.

❌ 2 New Failures, 12 Unrelated Failures

As of commit bd4690a with merge base 60cd104 (image):

NEW FAILURES - The following jobs have failed:

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

vmoens added a commit that referenced this pull request Sep 4, 2024
@vmoens
[BugFix] Fix tictactoeenv.py
6c66796 
ghstack-source-id: 99a368cf34cb7a3240ee85e85fb945d39292beb5
Pull Request resolved: #2417
@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Sep 4, 2024
@github-actions GitHub Actions
Copy link

github-actions bot commented Sep 4, 2024

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}7$. Worsened: $\large\color{#d91a1a}7$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 58.7881ms 58.3783ms 17.1297 Ops/s 16.9047 Ops/s $\color{#35bf28}+1.33\%$
test_sync 43.1766ms 33.3708ms 29.9663 Ops/s 31.4726 Ops/s $\color{#d91a1a}-4.79\%$
test_async 55.7770ms 31.1853ms 32.0664 Ops/s 32.8580 Ops/s $\color{#d91a1a}-2.41\%$
test_simple 0.4905s 0.4198s 2.3823 Ops/s 2.4357 Ops/s $\color{#d91a1a}-2.19\%$
test_transformed 0.6438s 0.5764s 1.7348 Ops/s 1.7244 Ops/s $\color{#35bf28}+0.61\%$
test_serial 1.3339s 1.2717s 0.7864 Ops/s 0.7805 Ops/s $\color{#35bf28}+0.76\%$
test_parallel 1.1849s 1.1106s 0.9004 Ops/s 0.8953 Ops/s $\color{#35bf28}+0.57\%$
test_step_mdp_speed[True-True-True-True-True] 0.1847ms 27.4098μs 36.4833 KOps/s 35.9948 KOps/s $\color{#35bf28}+1.36\%$
test_step_mdp_speed[True-True-True-True-False] 44.0520μs 16.0280μs 62.3907 KOps/s 61.7319 KOps/s $\color{#35bf28}+1.07\%$
test_step_mdp_speed[True-True-True-False-True] 49.9540μs 15.7476μs 63.5016 KOps/s 62.9172 KOps/s $\color{#35bf28}+0.93\%$
test_step_mdp_speed[True-True-True-False-False] 30.2770μs 9.2505μs 108.1018 KOps/s 106.4989 KOps/s $\color{#35bf28}+1.51\%$
test_step_mdp_speed[True-True-False-True-True] 61.2050μs 29.0535μs 34.4192 KOps/s 33.2934 KOps/s $\color{#35bf28}+3.38\%$
test_step_mdp_speed[True-True-False-True-False] 43.3310μs 17.5687μs 56.9195 KOps/s 55.8621 KOps/s $\color{#35bf28}+1.89\%$
test_step_mdp_speed[True-True-False-False-True] 44.2130μs 17.3382μs 57.6762 KOps/s 57.0719 KOps/s $\color{#35bf28}+1.06\%$
test_step_mdp_speed[True-True-False-False-False] 37.7110μs 10.7779μs 92.7828 KOps/s 91.4349 KOps/s $\color{#35bf28}+1.47\%$
test_step_mdp_speed[True-False-True-True-True] 83.1860μs 30.8844μs 32.3788 KOps/s 32.0835 KOps/s $\color{#35bf28}+0.92\%$
test_step_mdp_speed[True-False-True-True-False] 57.4480μs 19.3479μs 51.6852 KOps/s 51.4877 KOps/s $\color{#35bf28}+0.38\%$
test_step_mdp_speed[True-False-True-False-True] 48.0800μs 17.4423μs 57.3320 KOps/s 56.6152 KOps/s $\color{#35bf28}+1.27\%$
test_step_mdp_speed[True-False-True-False-False] 40.7570μs 10.8615μs 92.0681 KOps/s 91.2737 KOps/s $\color{#35bf28}+0.87\%$
test_step_mdp_speed[True-False-False-True-True] 78.2070μs 32.4006μs 30.8636 KOps/s 30.2423 KOps/s $\color{#35bf28}+2.05\%$
test_step_mdp_speed[True-False-False-True-False] 49.6030μs 20.8704μs 47.9148 KOps/s 47.4118 KOps/s $\color{#35bf28}+1.06\%$
test_step_mdp_speed[True-False-False-False-True] 50.8350μs 18.8594μs 53.0240 KOps/s 51.6749 KOps/s $\color{#35bf28}+2.61\%$
test_step_mdp_speed[True-False-False-False-False] 56.9370μs 12.1830μs 82.0816 KOps/s 79.1448 KOps/s $\color{#35bf28}+3.71\%$
test_step_mdp_speed[False-True-True-True-True] 78.0560μs 30.7971μs 32.4706 KOps/s 31.3277 KOps/s $\color{#35bf28}+3.65\%$
test_step_mdp_speed[False-True-True-True-False] 52.0970μs 19.0247μs 52.5631 KOps/s 50.9056 KOps/s $\color{#35bf28}+3.26\%$
test_step_mdp_speed[False-True-True-False-True] 53.3710μs 19.9618μs 50.0956 KOps/s 49.6157 KOps/s $\color{#35bf28}+0.97\%$
test_step_mdp_speed[False-True-True-False-False] 56.3560μs 12.0692μs 82.8558 KOps/s 81.3668 KOps/s $\color{#35bf28}+1.83\%$
test_step_mdp_speed[False-True-False-True-True] 70.8620μs 32.2782μs 30.9807 KOps/s 29.9111 KOps/s $\color{#35bf28}+3.58\%$
test_step_mdp_speed[False-True-False-True-False] 44.5540μs 20.5992μs 48.5455 KOps/s 47.1035 KOps/s $\color{#35bf28}+3.06\%$
test_step_mdp_speed[False-True-False-False-True] 2.8340ms 21.3119μs 46.9222 KOps/s 45.3646 KOps/s $\color{#35bf28}+3.43\%$
test_step_mdp_speed[False-True-False-False-False] 39.7950μs 13.5222μs 73.9526 KOps/s 71.1870 KOps/s $\color{#35bf28}+3.89\%$
test_step_mdp_speed[False-False-True-True-True] 67.1160μs 33.9808μs 29.4284 KOps/s 28.7444 KOps/s $\color{#35bf28}+2.38\%$
test_step_mdp_speed[False-False-True-True-False] 53.0900μs 22.2258μs 44.9927 KOps/s 43.6703 KOps/s $\color{#35bf28}+3.03\%$
test_step_mdp_speed[False-False-True-False-True] 53.3400μs 21.3857μs 46.7601 KOps/s 45.2046 KOps/s $\color{#35bf28}+3.44\%$
test_step_mdp_speed[False-False-True-False-False] 42.4290μs 13.6430μs 73.2975 KOps/s 71.5348 KOps/s $\color{#35bf28}+2.46\%$
test_step_mdp_speed[False-False-False-True-True] 65.2630μs 35.1672μs 28.4356 KOps/s 27.6621 KOps/s $\color{#35bf28}+2.80\%$
test_step_mdp_speed[False-False-False-True-False] 56.2650μs 23.7656μs 42.0776 KOps/s 41.0820 KOps/s $\color{#35bf28}+2.42\%$
test_step_mdp_speed[False-False-False-False-True] 55.1030μs 22.3897μs 44.6634 KOps/s 41.9106 KOps/s $\textbf{\color{#35bf28}+6.57\%}$
test_step_mdp_speed[False-False-False-False-False] 41.7480μs 15.0758μs 66.3315 KOps/s 65.4721 KOps/s $\color{#35bf28}+1.31\%$
test_values[generalized_advantage_estimate-True-True] 9.7594ms 9.4507ms 105.8120 Ops/s 101.1964 Ops/s $\color{#35bf28}+4.56\%$
test_values[vec_generalized_advantage_estimate-True-True] 36.2439ms 33.2514ms 30.0739 Ops/s 28.4974 Ops/s $\textbf{\color{#35bf28}+5.53\%}$
test_values[td0_return_estimate-False-False] 0.2260ms 0.1666ms 6.0036 KOps/s 5.7634 KOps/s $\color{#35bf28}+4.17\%$
test_values[td1_return_estimate-False-False] 24.5246ms 23.2471ms 43.0161 Ops/s 40.9758 Ops/s $\color{#35bf28}+4.98\%$
test_values[vec_td1_return_estimate-False-False] 40.6663ms 33.7907ms 29.5940 Ops/s 27.6517 Ops/s $\textbf{\color{#35bf28}+7.02\%}$
test_values[td_lambda_return_estimate-True-False] 36.4626ms 33.7990ms 29.5867 Ops/s 28.3896 Ops/s $\color{#35bf28}+4.22\%$
test_values[vec_td_lambda_return_estimate-True-False] 48.0334ms 38.3658ms 26.0649 Ops/s 27.9976 Ops/s $\textbf{\color{#d91a1a}-6.90\%}$
test_gae_speed[generalized_advantage_estimate-False-1-512] 8.5197ms 8.3419ms 119.8767 Ops/s 118.0624 Ops/s $\color{#35bf28}+1.54\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 2.2967ms 1.9714ms 507.2638 Ops/s 492.7509 Ops/s $\color{#35bf28}+2.95\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.4217ms 0.3516ms 2.8439 KOps/s 2.7789 KOps/s $\color{#35bf28}+2.34\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 42.1522ms 39.3425ms 25.4178 Ops/s 22.1861 Ops/s $\textbf{\color{#35bf28}+14.57\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 4.1820ms 3.0458ms 328.3227 Ops/s 328.1975 Ops/s $\color{#35bf28}+0.04\%$
test_dqn_speed 6.5779ms 1.3477ms 742.0183 Ops/s 749.1861 Ops/s $\color{#d91a1a}-0.96\%$
test_ddpg_speed 3.5598ms 2.7633ms 361.8824 Ops/s 363.5815 Ops/s $\color{#d91a1a}-0.47\%$
test_sac_speed 10.1697ms 8.1392ms 122.8629 Ops/s 124.2501 Ops/s $\color{#d91a1a}-1.12\%$
test_redq_speed 14.3625ms 12.9484ms 77.2294 Ops/s 78.1632 Ops/s $\color{#d91a1a}-1.19\%$
test_redq_deprec_speed 14.0712ms 12.8661ms 77.7237 Ops/s 79.8224 Ops/s $\color{#d91a1a}-2.63\%$
test_td3_speed 8.2701ms 8.0613ms 124.0492 Ops/s 123.5348 Ops/s $\color{#35bf28}+0.42\%$
test_cql_speed 36.8334ms 35.8913ms 27.8619 Ops/s 27.9268 Ops/s $\color{#d91a1a}-0.23\%$
test_a2c_speed 8.6486ms 7.4057ms 135.0311 Ops/s 134.7651 Ops/s $\color{#35bf28}+0.20\%$
test_ppo_speed 9.0805ms 7.7241ms 129.4646 Ops/s 132.7648 Ops/s $\color{#d91a1a}-2.49\%$
test_reinforce_speed 7.4605ms 6.6162ms 151.1449 Ops/s 152.8287 Ops/s $\color{#d91a1a}-1.10\%$
test_iql_speed 40.3742ms 32.3895ms 30.8742 Ops/s 31.3499 Ops/s $\color{#d91a1a}-1.52\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 7.7758ms 4.9675ms 201.3089 Ops/s 208.5258 Ops/s $\color{#d91a1a}-3.46\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.0860ms 0.4700ms 2.1276 KOps/s 2.1273 KOps/s $\color{#35bf28}+0.01\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.9028ms 0.4607ms 2.1704 KOps/s 2.2079 KOps/s $\color{#d91a1a}-1.70\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 7.0726ms 5.0398ms 198.4187 Ops/s 205.1190 Ops/s $\color{#d91a1a}-3.27\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.8476ms 0.4728ms 2.1151 KOps/s 2.1556 KOps/s $\color{#d91a1a}-1.88\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6973ms 0.4532ms 2.2065 KOps/s 2.2737 KOps/s $\color{#d91a1a}-2.95\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 2.2415ms 1.6716ms 598.2190 Ops/s 594.0324 Ops/s $\color{#35bf28}+0.70\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 2.2410ms 1.6002ms 624.9278 Ops/s 631.0048 Ops/s $\color{#d91a1a}-0.96\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.2187ms 5.2949ms 188.8614 Ops/s 197.0953 Ops/s $\color{#d91a1a}-4.18\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.1667s 0.7723ms 1.2948 KOps/s 1.6573 KOps/s $\textbf{\color{#d91a1a}-21.87\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.9360ms 0.6004ms 1.6655 KOps/s 1.7484 KOps/s $\color{#d91a1a}-4.74\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 7.6668ms 5.1689ms 193.4660 Ops/s 209.7706 Ops/s $\textbf{\color{#d91a1a}-7.77\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.6287ms 0.4870ms 2.0535 KOps/s 2.0803 KOps/s $\color{#d91a1a}-1.29\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 7.5379ms 0.4697ms 2.1292 KOps/s 2.2005 KOps/s $\color{#d91a1a}-3.24\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.4478ms 5.0926ms 196.3626 Ops/s 209.5980 Ops/s $\textbf{\color{#d91a1a}-6.31\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.9523ms 0.5020ms 1.9921 KOps/s 2.0970 KOps/s $\textbf{\color{#d91a1a}-5.00\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.8786ms 0.4727ms 2.1156 KOps/s 2.1829 KOps/s $\color{#d91a1a}-3.08\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.7401ms 5.2541ms 190.3265 Ops/s 193.9195 Ops/s $\color{#d91a1a}-1.85\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.4895ms 0.6257ms 1.5982 KOps/s 1.6288 KOps/s $\color{#d91a1a}-1.88\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.7654ms 0.5877ms 1.7016 KOps/s 1.2829 KOps/s $\textbf{\color{#35bf28}+32.64\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1491s 6.8033ms 146.9881 Ops/s 159.7811 Ops/s $\textbf{\color{#d91a1a}-8.01\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 17.1438ms 12.9286ms 77.3478 Ops/s 75.6202 Ops/s $\color{#35bf28}+2.28\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 1.7551ms 1.1981ms 834.6614 Ops/s 819.7496 Ops/s $\color{#35bf28}+1.82\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1266s 8.8130ms 113.4688 Ops/s 157.4047 Ops/s $\textbf{\color{#d91a1a}-27.91\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 17.9377ms 12.9505ms 77.2171 Ops/s 75.5710 Ops/s $\color{#35bf28}+2.18\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 1.9941ms 1.2529ms 798.1747 Ops/s 755.0458 Ops/s $\textbf{\color{#35bf28}+5.71\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1248s 6.5146ms 153.5002 Ops/s 112.7852 Ops/s $\textbf{\color{#35bf28}+36.10\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 18.1858ms 13.1147ms 76.2503 Ops/s 74.5785 Ops/s $\color{#35bf28}+2.24\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 4.9164ms 1.4244ms 702.0710 Ops/s 723.6992 Ops/s $\color{#d91a1a}-2.99\%$

@github-actions GitHub Actions
Copy link

github-actions bot commented Sep 4, 2024

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}4$. Worsened: $\large\color{#d91a1a}12$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1035s 0.1032s 9.6896 Ops/s 9.7319 Ops/s $\color{#d91a1a}-0.43\%$
test_sync 92.7717ms 90.3965ms 11.0624 Ops/s 10.9487 Ops/s $\color{#35bf28}+1.04\%$
test_async 0.2657s 86.8847ms 11.5095 Ops/s 11.7148 Ops/s $\color{#d91a1a}-1.75\%$
test_single_pixels 0.1094s 0.1092s 9.1585 Ops/s 9.1423 Ops/s $\color{#35bf28}+0.18\%$
test_sync_pixels 72.4849ms 71.3286ms 14.0196 Ops/s 14.1435 Ops/s $\color{#d91a1a}-0.88\%$
test_async_pixels 0.1344s 67.9181ms 14.7236 Ops/s 15.0204 Ops/s $\color{#d91a1a}-1.98\%$
test_simple 0.7307s 0.7276s 1.3744 Ops/s 1.3456 Ops/s $\color{#35bf28}+2.14\%$
test_transformed 0.9680s 0.9644s 1.0369 Ops/s 1.0465 Ops/s $\color{#d91a1a}-0.91\%$
test_serial 2.1645s 2.0943s 0.4775 Ops/s 0.4820 Ops/s $\color{#d91a1a}-0.94\%$
test_parallel 1.9285s 1.8660s 0.5359 Ops/s 0.5361 Ops/s $\color{#d91a1a}-0.04\%$
test_step_mdp_speed[True-True-True-True-True] 0.2002ms 36.6017μs 27.3211 KOps/s 27.4569 KOps/s $\color{#d91a1a}-0.49\%$
test_step_mdp_speed[True-True-True-True-False] 0.1010ms 21.3138μs 46.9179 KOps/s 46.5250 KOps/s $\color{#35bf28}+0.84\%$
test_step_mdp_speed[True-True-True-False-True] 0.1677ms 20.7818μs 48.1190 KOps/s 48.1294 KOps/s $\color{#d91a1a}-0.02\%$
test_step_mdp_speed[True-True-True-False-False] 41.6710μs 12.0154μs 83.2266 KOps/s 81.8607 KOps/s $\color{#35bf28}+1.67\%$
test_step_mdp_speed[True-True-False-True-True] 76.0210μs 38.7844μs 25.7836 KOps/s 25.3873 KOps/s $\color{#35bf28}+1.56\%$
test_step_mdp_speed[True-True-False-True-False] 56.2510μs 23.4236μs 42.6920 KOps/s 42.2516 KOps/s $\color{#35bf28}+1.04\%$
test_step_mdp_speed[True-True-False-False-True] 0.1064ms 22.8875μs 43.6920 KOps/s 42.9174 KOps/s $\color{#35bf28}+1.80\%$
test_step_mdp_speed[True-True-False-False-False] 53.6610μs 14.2735μs 70.0599 KOps/s 70.4663 KOps/s $\color{#d91a1a}-0.58\%$
test_step_mdp_speed[True-False-True-True-True] 74.2310μs 41.5534μs 24.0654 KOps/s 23.7433 KOps/s $\color{#35bf28}+1.36\%$
test_step_mdp_speed[True-False-True-True-False] 51.9700μs 25.4189μs 39.3408 KOps/s 38.9426 KOps/s $\color{#35bf28}+1.02\%$
test_step_mdp_speed[True-False-True-False-True] 65.8510μs 23.1192μs 43.2542 KOps/s 43.3562 KOps/s $\color{#d91a1a}-0.24\%$
test_step_mdp_speed[True-False-True-False-False] 44.3210μs 14.1632μs 70.6054 KOps/s 70.8035 KOps/s $\color{#d91a1a}-0.28\%$
test_step_mdp_speed[True-False-False-True-True] 85.5110μs 42.8140μs 23.3568 KOps/s 23.0688 KOps/s $\color{#35bf28}+1.25\%$
test_step_mdp_speed[True-False-False-True-False] 74.9320μs 27.6779μs 36.1299 KOps/s 37.1603 KOps/s $\color{#d91a1a}-2.77\%$
test_step_mdp_speed[True-False-False-False-True] 54.2410μs 24.7801μs 40.3550 KOps/s 40.5696 KOps/s $\color{#d91a1a}-0.53\%$
test_step_mdp_speed[True-False-False-False-False] 58.7210μs 16.0332μs 62.3706 KOps/s 62.4050 KOps/s $\color{#d91a1a}-0.06\%$
test_step_mdp_speed[False-True-True-True-True] 84.1710μs 41.5526μs 24.0659 KOps/s 23.9550 KOps/s $\color{#35bf28}+0.46\%$
test_step_mdp_speed[False-True-True-True-False] 59.2910μs 25.8248μs 38.7224 KOps/s 39.4783 KOps/s $\color{#d91a1a}-1.91\%$
test_step_mdp_speed[False-True-True-False-True] 59.0610μs 26.5917μs 37.6057 KOps/s 36.9575 KOps/s $\color{#35bf28}+1.75\%$
test_step_mdp_speed[False-True-True-False-False] 57.9510μs 15.7918μs 63.3242 KOps/s 62.6537 KOps/s $\color{#35bf28}+1.07\%$
test_step_mdp_speed[False-True-False-True-True] 74.0510μs 42.8560μs 23.3340 KOps/s 23.1891 KOps/s $\color{#35bf28}+0.62\%$
test_step_mdp_speed[False-True-False-True-False] 56.6800μs 27.2352μs 36.7171 KOps/s 36.6003 KOps/s $\color{#35bf28}+0.32\%$
test_step_mdp_speed[False-True-False-False-True] 3.4990ms 28.9445μs 34.5489 KOps/s 34.4522 KOps/s $\color{#35bf28}+0.28\%$
test_step_mdp_speed[False-True-False-False-False] 68.0510μs 18.0963μs 55.2600 KOps/s 55.6072 KOps/s $\color{#d91a1a}-0.62\%$
test_step_mdp_speed[False-False-True-True-True] 79.6110μs 45.3264μs 22.0622 KOps/s 21.9567 KOps/s $\color{#35bf28}+0.48\%$
test_step_mdp_speed[False-False-True-True-False] 57.8000μs 29.5453μs 33.8464 KOps/s 33.9132 KOps/s $\color{#d91a1a}-0.20\%$
test_step_mdp_speed[False-False-True-False-True] 0.1360ms 28.3779μs 35.2387 KOps/s 34.7251 KOps/s $\color{#35bf28}+1.48\%$
test_step_mdp_speed[False-False-True-False-False] 57.3500μs 18.0486μs 55.4059 KOps/s 55.0908 KOps/s $\color{#35bf28}+0.57\%$
test_step_mdp_speed[False-False-False-True-True] 74.4110μs 47.1630μs 21.2031 KOps/s 21.1123 KOps/s $\color{#35bf28}+0.43\%$
test_step_mdp_speed[False-False-False-True-False] 0.2119ms 31.4726μs 31.7737 KOps/s 31.8258 KOps/s $\color{#d91a1a}-0.16\%$
test_step_mdp_speed[False-False-False-False-True] 0.1298ms 30.0861μs 33.2379 KOps/s 33.1349 KOps/s $\color{#35bf28}+0.31\%$
test_step_mdp_speed[False-False-False-False-False] 0.2316ms 19.9907μs 50.0232 KOps/s 49.6800 KOps/s $\color{#35bf28}+0.69\%$
test_values[generalized_advantage_estimate-True-True] 24.2539ms 23.7261ms 42.1477 Ops/s 42.2075 Ops/s $\color{#d91a1a}-0.14\%$
test_values[vec_generalized_advantage_estimate-True-True] 0.1002s 2.8861ms 346.4917 Ops/s 329.3632 Ops/s $\textbf{\color{#35bf28}+5.20\%}$
test_values[td0_return_estimate-False-False] 89.2320μs 66.0789μs 15.1334 KOps/s 15.4069 KOps/s $\color{#d91a1a}-1.77\%$
test_values[td1_return_estimate-False-False] 57.5262ms 55.4706ms 18.0276 Ops/s 18.4869 Ops/s $\color{#d91a1a}-2.48\%$
test_values[vec_td1_return_estimate-False-False] 1.3088ms 1.0560ms 946.9676 Ops/s 943.8164 Ops/s $\color{#35bf28}+0.33\%$
test_values[td_lambda_return_estimate-True-False] 86.0855ms 85.5500ms 11.6891 Ops/s 11.7068 Ops/s $\color{#d91a1a}-0.15\%$
test_values[vec_td_lambda_return_estimate-True-False] 1.3593ms 1.0606ms 942.8928 Ops/s 952.2682 Ops/s $\color{#d91a1a}-0.98\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 24.0393ms 23.7449ms 42.1144 Ops/s 40.7348 Ops/s $\color{#35bf28}+3.39\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 0.9196ms 0.7025ms 1.4234 KOps/s 1.4477 KOps/s $\color{#d91a1a}-1.68\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.7998ms 0.6413ms 1.5594 KOps/s 1.5669 KOps/s $\color{#d91a1a}-0.48\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.7559ms 1.4529ms 688.2580 Ops/s 688.8118 Ops/s $\color{#d91a1a}-0.08\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.8278ms 0.6576ms 1.5206 KOps/s 1.5228 KOps/s $\color{#d91a1a}-0.14\%$
test_dqn_speed 7.1832ms 1.3155ms 760.1786 Ops/s 763.8951 Ops/s $\color{#d91a1a}-0.49\%$
test_ddpg_speed 2.9120ms 2.6477ms 377.6907 Ops/s 370.6903 Ops/s $\color{#35bf28}+1.89\%$
test_sac_speed 7.9930ms 7.6207ms 131.2223 Ops/s 130.7788 Ops/s $\color{#35bf28}+0.34\%$
test_redq_speed 0.1060s 11.0151ms 90.7843 Ops/s 99.2775 Ops/s $\textbf{\color{#d91a1a}-8.56\%}$
test_redq_deprec_speed 11.2233ms 10.6785ms 93.6463 Ops/s 92.5090 Ops/s $\color{#35bf28}+1.23\%$
test_td3_speed 8.0498ms 7.9804ms 125.3077 Ops/s 128.0138 Ops/s $\color{#d91a1a}-2.11\%$
test_cql_speed 27.5212ms 24.9044ms 40.1535 Ops/s 39.4844 Ops/s $\color{#35bf28}+1.69\%$
test_a2c_speed 5.9858ms 5.4790ms 182.5138 Ops/s 183.4650 Ops/s $\color{#d91a1a}-0.52\%$
test_ppo_speed 6.0838ms 5.8143ms 171.9890 Ops/s 175.0556 Ops/s $\color{#d91a1a}-1.75\%$
test_reinforce_speed 4.8763ms 4.5405ms 220.2421 Ops/s 221.2213 Ops/s $\color{#d91a1a}-0.44\%$
test_iql_speed 19.7397ms 19.1785ms 52.1418 Ops/s 53.7634 Ops/s $\color{#d91a1a}-3.02\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 6.7072ms 6.4027ms 156.1837 Ops/s 157.7582 Ops/s $\color{#d91a1a}-1.00\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.2192ms 0.3326ms 3.0069 KOps/s 4.2167 KOps/s $\textbf{\color{#d91a1a}-28.69\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6224ms 0.2136ms 4.6823 KOps/s 4.6665 KOps/s $\color{#35bf28}+0.34\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.7006ms 6.2917ms 158.9400 Ops/s 159.1169 Ops/s $\color{#d91a1a}-0.11\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.7222ms 0.3130ms 3.1952 KOps/s 4.2800 KOps/s $\textbf{\color{#d91a1a}-25.35\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.5981ms 0.2954ms 3.3851 KOps/s 4.6724 KOps/s $\textbf{\color{#d91a1a}-27.55\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.7248ms 1.4009ms 713.8239 Ops/s 750.2821 Ops/s $\color{#d91a1a}-4.86\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 1.5126ms 1.3186ms 758.3914 Ops/s 814.5161 Ops/s $\textbf{\color{#d91a1a}-6.89\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.7644ms 6.4896ms 154.0929 Ops/s 155.3373 Ops/s $\color{#d91a1a}-0.80\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.7993ms 0.4410ms 2.2675 KOps/s 2.6513 KOps/s $\textbf{\color{#d91a1a}-14.48\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.7404ms 0.4224ms 2.3676 KOps/s 2.7734 KOps/s $\textbf{\color{#d91a1a}-14.63\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 6.6461ms 6.3777ms 156.7966 Ops/s 156.8760 Ops/s $\color{#d91a1a}-0.05\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.2874ms 0.3323ms 3.0095 KOps/s 4.1826 KOps/s $\textbf{\color{#d91a1a}-28.05\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.5513ms 0.3129ms 3.1962 KOps/s 4.6167 KOps/s $\textbf{\color{#d91a1a}-30.77\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.7420ms 6.3319ms 157.9308 Ops/s 158.7296 Ops/s $\color{#d91a1a}-0.50\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.1952ms 0.3262ms 3.0657 KOps/s 4.2015 KOps/s $\textbf{\color{#d91a1a}-27.03\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6386ms 0.3081ms 3.2457 KOps/s 4.6746 KOps/s $\textbf{\color{#d91a1a}-30.57\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.7398ms 6.4996ms 153.8546 Ops/s 154.2032 Ops/s $\color{#d91a1a}-0.23\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.7650ms 0.3860ms 2.5909 KOps/s 1.9257 KOps/s $\textbf{\color{#35bf28}+34.54\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.7676ms 0.3568ms 2.8024 KOps/s 2.7927 KOps/s $\color{#35bf28}+0.35\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1474s 7.9489ms 125.8043 Ops/s 125.1088 Ops/s $\color{#35bf28}+0.56\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 21.1635ms 16.0211ms 62.4178 Ops/s 62.0641 Ops/s $\color{#35bf28}+0.57\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 2.0917ms 1.0295ms 971.3663 Ops/s 1.0102 KOps/s $\color{#d91a1a}-3.84\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1289s 7.5491ms 132.4666 Ops/s 130.3030 Ops/s $\color{#35bf28}+1.66\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 21.4438ms 16.2205ms 61.6504 Ops/s 63.1398 Ops/s $\color{#d91a1a}-2.36\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 2.0145ms 1.0176ms 982.7484 Ops/s 873.7568 Ops/s $\textbf{\color{#35bf28}+12.47\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1301s 7.7875ms 128.4101 Ops/s 128.3111 Ops/s $\color{#35bf28}+0.08\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 0.1400s 18.5495ms 53.9098 Ops/s 62.5420 Ops/s $\textbf{\color{#d91a1a}-13.80\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 2.1745ms 1.1405ms 876.7911 Ops/s 779.6596 Ops/s $\textbf{\color{#35bf28}+12.46\%}$

@vmoens vmoens added the bug Something isn't working label Sep 4, 2024
@vmoens vmoens merged commit bd4690a into gh/vmoens/25/base Sep 4, 2024
57 of 71 checks passed
vmoens added a commit that referenced this pull request Sep 4, 2024
@vmoens
[BugFix] Fix tictactoeenv.py
d4b29b5 
ghstack-source-id: 99a368cf34cb7a3240ee85e85fb945d39292beb5
Pull Request resolved: #2417
@vmoens vmoens deleted the gh/vmoens/25/head branch September 4, 2024 14:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
bug Something isn't working CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@vmoens @facebook-github-bot