Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[CI, Minor] Regroup Gen-DGRL CI with other libs #1814

Merged
merged 1 commit into from
Jan 18, 2024
Merged

[CI, Minor] Regroup Gen-DGRL CI with other libs #1814

merged 1 commit into from
Jan 18, 2024

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Jan 18, 2024 •
edited
Loading

No description provided.

@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Jan 18, 2024 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/1814

Note: Links to docs will display an error until the docs builds have been completed.

⏳ 1 Pending, 25 Unrelated Failures

As of commit f33a435 with merge base eb61100 (image):

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jan 18, 2024
@vmoens vmoens added CI Has to do with CI setup (e.g. wheels & builds, tests...) Data Data-related PR, will launch data-related jobs labels Jan 18, 2024
@vmoens vmoens merged commit 2ecb709 into main Jan 18, 2024
25 of 44 checks passed
@github-actions GitHub Actions
Copy link

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 89. Improved: $\large\color{#35bf28}5$. Worsened: $\large\color{#d91a1a}4$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1357s 66.0621ms 15.1373 Ops/s 16.2796 Ops/s $\textbf{\color{#d91a1a}-7.02\%}$
test_sync 41.3595ms 33.4896ms 29.8600 Ops/s 29.9889 Ops/s $\color{#d91a1a}-0.43\%$
test_async 0.1066s 32.5133ms 30.7567 Ops/s 31.9198 Ops/s $\color{#d91a1a}-3.64\%$
test_simple 0.4950s 0.4327s 2.3109 Ops/s 2.2566 Ops/s $\color{#35bf28}+2.41\%$
test_transformed 0.6683s 0.5985s 1.6710 Ops/s 1.6729 Ops/s $\color{#d91a1a}-0.12\%$
test_serial 1.4183s 1.3528s 0.7392 Ops/s 0.7287 Ops/s $\color{#35bf28}+1.45\%$
test_parallel 1.3029s 1.2296s 0.8133 Ops/s 0.8359 Ops/s $\color{#d91a1a}-2.71\%$
test_step_mdp_speed[True-True-True-True-True] 0.1613ms 21.5108μs 46.4882 KOps/s 47.4347 KOps/s $\color{#d91a1a}-2.00\%$
test_step_mdp_speed[True-True-True-True-False] 40.3460μs 13.2268μs 75.6043 KOps/s 77.2524 KOps/s $\color{#d91a1a}-2.13\%$
test_step_mdp_speed[True-True-True-False-True] 48.4210μs 12.5641μs 79.5916 KOps/s 80.7108 KOps/s $\color{#d91a1a}-1.39\%$
test_step_mdp_speed[True-True-True-False-False] 29.1540μs 7.6842μs 130.1374 KOps/s 135.3609 KOps/s $\color{#d91a1a}-3.86\%$
test_step_mdp_speed[True-True-False-True-True] 70.7120μs 22.8465μs 43.7704 KOps/s 44.3043 KOps/s $\color{#d91a1a}-1.21\%$
test_step_mdp_speed[True-True-False-True-False] 42.1180μs 14.4238μs 69.3297 KOps/s 69.7062 KOps/s $\color{#d91a1a}-0.54\%$
test_step_mdp_speed[True-True-False-False-True] 37.7100μs 13.8525μs 72.1891 KOps/s 72.1095 KOps/s $\color{#35bf28}+0.11\%$
test_step_mdp_speed[True-True-False-False-False] 31.9090μs 8.8752μs 112.6741 KOps/s 112.2674 KOps/s $\color{#35bf28}+0.36\%$
test_step_mdp_speed[True-False-True-True-True] 52.9880μs 24.5287μs 40.7685 KOps/s 41.1685 KOps/s $\color{#d91a1a}-0.97\%$
test_step_mdp_speed[True-False-True-True-False] 46.7270μs 15.8523μs 63.0825 KOps/s 63.8022 KOps/s $\color{#d91a1a}-1.13\%$
test_step_mdp_speed[True-False-True-False-True] 41.5870μs 13.9244μs 71.8165 KOps/s 72.0440 KOps/s $\color{#d91a1a}-0.32\%$
test_step_mdp_speed[True-False-True-False-False] 33.2620μs 8.9667μs 111.5238 KOps/s 113.6287 KOps/s $\color{#d91a1a}-1.85\%$
test_step_mdp_speed[True-False-False-True-True] 74.3190μs 25.6140μs 39.0411 KOps/s 39.7351 KOps/s $\color{#d91a1a}-1.75\%$
test_step_mdp_speed[True-False-False-True-False] 45.1140μs 17.0363μs 58.6983 KOps/s 59.9277 KOps/s $\color{#d91a1a}-2.05\%$
test_step_mdp_speed[True-False-False-False-True] 40.1650μs 14.9468μs 66.9041 KOps/s 67.1024 KOps/s $\color{#d91a1a}-0.30\%$
test_step_mdp_speed[True-False-False-False-False] 42.4490μs 10.1741μs 98.2889 KOps/s 102.6721 KOps/s $\color{#d91a1a}-4.27\%$
test_step_mdp_speed[False-True-True-True-True] 63.2080μs 24.6061μs 40.6403 KOps/s 42.4464 KOps/s $\color{#d91a1a}-4.25\%$
test_step_mdp_speed[False-True-True-True-False] 39.4540μs 15.9472μs 62.7070 KOps/s 64.5418 KOps/s $\color{#d91a1a}-2.84\%$
test_step_mdp_speed[False-True-True-False-True] 43.3010μs 16.2101μs 61.6901 KOps/s 62.9638 KOps/s $\color{#d91a1a}-2.02\%$
test_step_mdp_speed[False-True-True-False-False] 30.7670μs 10.2892μs 97.1897 KOps/s 100.3833 KOps/s $\color{#d91a1a}-3.18\%$
test_step_mdp_speed[False-True-False-True-True] 61.8650μs 25.7104μs 38.8947 KOps/s 40.2453 KOps/s $\color{#d91a1a}-3.36\%$
test_step_mdp_speed[False-True-False-True-False] 55.0530μs 17.0492μs 58.6537 KOps/s 59.7857 KOps/s $\color{#d91a1a}-1.89\%$
test_step_mdp_speed[False-True-False-False-True] 48.2200μs 17.2981μs 57.8097 KOps/s 59.1255 KOps/s $\color{#d91a1a}-2.23\%$
test_step_mdp_speed[False-True-False-False-False] 30.5770μs 11.3726μs 87.9305 KOps/s 89.8614 KOps/s $\color{#d91a1a}-2.15\%$
test_step_mdp_speed[False-False-True-True-True] 68.5280μs 26.7767μs 37.3459 KOps/s 38.0025 KOps/s $\color{#d91a1a}-1.73\%$
test_step_mdp_speed[False-False-True-True-False] 61.5750μs 18.3017μs 54.6397 KOps/s 55.3287 KOps/s $\color{#d91a1a}-1.25\%$
test_step_mdp_speed[False-False-True-False-True] 48.7710μs 17.3803μs 57.5365 KOps/s 57.8430 KOps/s $\color{#d91a1a}-0.53\%$
test_step_mdp_speed[False-False-True-False-False] 50.7150μs 11.5004μs 86.9533 KOps/s 89.2828 KOps/s $\color{#d91a1a}-2.61\%$
test_step_mdp_speed[False-False-False-True-True] 64.3800μs 27.8294μs 35.9333 KOps/s 36.6338 KOps/s $\color{#d91a1a}-1.91\%$
test_step_mdp_speed[False-False-False-True-False] 57.2370μs 19.3166μs 51.7689 KOps/s 52.6923 KOps/s $\color{#d91a1a}-1.75\%$
test_step_mdp_speed[False-False-False-False-True] 50.0930μs 18.2982μs 54.6502 KOps/s 55.1482 KOps/s $\color{#d91a1a}-0.90\%$
test_step_mdp_speed[False-False-False-False-False] 39.1730μs 12.5343μs 79.7811 KOps/s 80.9379 KOps/s $\color{#d91a1a}-1.43\%$
test_values[generalized_advantage_estimate-True-True] 12.2320ms 11.9928ms 83.3832 Ops/s 83.5352 Ops/s $\color{#d91a1a}-0.18\%$
test_values[vec_generalized_advantage_estimate-True-True] 36.9398ms 27.9034ms 35.8379 Ops/s 36.1641 Ops/s $\color{#d91a1a}-0.90\%$
test_values[td0_return_estimate-False-False] 0.2223ms 0.1768ms 5.6556 KOps/s 5.5862 KOps/s $\color{#35bf28}+1.24\%$
test_values[td1_return_estimate-False-False] 27.2230ms 25.6393ms 39.0026 Ops/s 39.3554 Ops/s $\color{#d91a1a}-0.90\%$
test_values[vec_td1_return_estimate-False-False] 36.1237ms 28.2297ms 35.4237 Ops/s 36.1288 Ops/s $\color{#d91a1a}-1.95\%$
test_values[td_lambda_return_estimate-True-False] 39.0131ms 36.0096ms 27.7703 Ops/s 28.0290 Ops/s $\color{#d91a1a}-0.92\%$
test_values[vec_td_lambda_return_estimate-True-False] 29.7170ms 27.8276ms 35.9356 Ops/s 35.6196 Ops/s $\color{#35bf28}+0.89\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 12.2809ms 8.2638ms 121.0095 Ops/s 125.8414 Ops/s $\color{#d91a1a}-3.84\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 10.3440ms 1.9064ms 524.5459 Ops/s 510.8753 Ops/s $\color{#35bf28}+2.68\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.6585ms 0.4204ms 2.3784 KOps/s 2.3359 KOps/s $\color{#35bf28}+1.82\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 45.1858ms 40.0761ms 24.9525 Ops/s 28.2444 Ops/s $\textbf{\color{#d91a1a}-11.65\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 11.3663ms 2.6461ms 377.9156 Ops/s 381.0922 Ops/s $\color{#d91a1a}-0.83\%$
test_dqn_speed 85.7044ms 8.1406ms 122.8414 Ops/s 132.6317 Ops/s $\textbf{\color{#d91a1a}-7.38\%}$
test_ddpg_speed 24.4458ms 14.5935ms 68.5238 Ops/s 68.2180 Ops/s $\color{#35bf28}+0.45\%$
test_sac_speed 54.1991ms 30.5505ms 32.7327 Ops/s 31.0479 Ops/s $\textbf{\color{#35bf28}+5.43\%}$
test_redq_speed 52.9675ms 45.1864ms 22.1306 Ops/s 21.4862 Ops/s $\color{#35bf28}+3.00\%$
test_redq_deprec_speed 33.7999ms 25.4134ms 39.3493 Ops/s 38.4020 Ops/s $\color{#35bf28}+2.47\%$
test_td3_speed 28.2231ms 19.9596ms 50.1012 Ops/s 48.6296 Ops/s $\color{#35bf28}+3.03\%$
test_cql_speed 90.8130ms 88.4615ms 11.3044 Ops/s 11.4079 Ops/s $\color{#d91a1a}-0.91\%$
test_a2c_speed 31.2199ms 26.5108ms 37.7204 Ops/s 37.6721 Ops/s $\color{#35bf28}+0.13\%$
test_ppo_speed 30.3205ms 27.0984ms 36.9025 Ops/s 37.1273 Ops/s $\color{#d91a1a}-0.61\%$
test_reinforce_speed 28.1094ms 25.6105ms 39.0465 Ops/s 38.9921 Ops/s $\color{#35bf28}+0.14\%$
test_iql_speed 71.7767ms 62.9055ms 15.8969 Ops/s 15.7295 Ops/s $\color{#35bf28}+1.06\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 2.6170ms 1.3960ms 716.3439 Ops/s 712.0069 Ops/s $\color{#35bf28}+0.61\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 8.7595ms 0.5231ms 1.9116 KOps/s 1.9237 KOps/s $\color{#d91a1a}-0.63\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 8.8047ms 0.4922ms 2.0319 KOps/s 2.0094 KOps/s $\color{#35bf28}+1.12\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 1.7246ms 1.3233ms 755.6928 Ops/s 694.3006 Ops/s $\textbf{\color{#35bf28}+8.84\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 8.8550ms 0.5166ms 1.9359 KOps/s 1.9051 KOps/s $\color{#35bf28}+1.62\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6715ms 0.4810ms 2.0788 KOps/s 1.9821 KOps/s $\color{#35bf28}+4.88\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 2.3246ms 1.5279ms 654.5142 Ops/s 610.4729 Ops/s $\textbf{\color{#35bf28}+7.21\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.4340ms 0.6474ms 1.5445 KOps/s 1.5302 KOps/s $\color{#35bf28}+0.94\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 9.8382ms 0.6239ms 1.6027 KOps/s 1.4136 KOps/s $\textbf{\color{#35bf28}+13.38\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 2.7586ms 1.4012ms 713.6610 Ops/s 723.7052 Ops/s $\color{#d91a1a}-1.39\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 9.7040ms 0.5262ms 1.9006 KOps/s 1.8905 KOps/s $\color{#35bf28}+0.53\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 1.7418ms 0.4895ms 2.0430 KOps/s 2.0401 KOps/s $\color{#35bf28}+0.14\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 1.7250ms 1.3590ms 735.8440 Ops/s 737.2867 Ops/s $\color{#d91a1a}-0.20\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 8.8669ms 0.5252ms 1.9042 KOps/s 1.8983 KOps/s $\color{#35bf28}+0.31\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6348ms 0.4889ms 2.0455 KOps/s 2.0513 KOps/s $\color{#d91a1a}-0.28\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 2.5186ms 1.6167ms 618.5489 Ops/s 621.1058 Ops/s $\color{#d91a1a}-0.41\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 8.9752ms 0.6613ms 1.5122 KOps/s 1.5091 KOps/s $\color{#35bf28}+0.20\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.7451ms 0.6195ms 1.6143 KOps/s 1.5805 KOps/s $\color{#35bf28}+2.14\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1085s 12.1305ms 82.4371 Ops/s 83.3398 Ops/s $\color{#d91a1a}-1.08\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 15.8203ms 13.5502ms 73.7998 Ops/s 73.8081 Ops/s $\color{#d91a1a}-0.01\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 10.5185ms 3.4421ms 290.5170 Ops/s 287.1523 Ops/s $\color{#35bf28}+1.17\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 99.3898ms 10.0016ms 99.9838 Ops/s 100.8278 Ops/s $\color{#d91a1a}-0.84\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 20.1446ms 13.6579ms 73.2179 Ops/s 64.3677 Ops/s $\textbf{\color{#35bf28}+13.75\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 6.7392ms 3.2966ms 303.3385 Ops/s 304.7485 Ops/s $\color{#d91a1a}-0.46\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1157s 13.1312ms 76.1546 Ops/s 94.4721 Ops/s $\textbf{\color{#d91a1a}-19.39\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 16.2365ms 13.9719ms 71.5722 Ops/s 71.5081 Ops/s $\color{#35bf28}+0.09\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 12.4901ms 3.6854ms 271.3440 Ops/s 269.5553 Ops/s $\color{#35bf28}+0.66\%$

@github-actions GitHub Actions
Copy link

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 92. Improved: $\large\color{#35bf28}2$. Worsened: $\large\color{#d91a1a}3$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1174s 0.1160s 8.6208 Ops/s 8.7845 Ops/s $\color{#d91a1a}-1.86\%$
test_sync 0.1713s 0.1028s 9.7229 Ops/s 9.7422 Ops/s $\color{#d91a1a}-0.20\%$
test_async 0.1819s 91.8437ms 10.8881 Ops/s 10.8653 Ops/s $\color{#35bf28}+0.21\%$
test_single_pixels 0.1265s 0.1260s 7.9375 Ops/s 7.1186 Ops/s $\textbf{\color{#35bf28}+11.50\%}$
test_sync_pixels 84.6645ms 79.2230ms 12.6226 Ops/s 13.3194 Ops/s $\textbf{\color{#d91a1a}-5.23\%}$
test_async_pixels 0.2042s 71.8193ms 13.9238 Ops/s 13.9765 Ops/s $\color{#d91a1a}-0.38\%$
test_simple 0.9100s 0.8406s 1.1896 Ops/s 1.1788 Ops/s $\color{#35bf28}+0.91\%$
test_transformed 1.1411s 1.0798s 0.9261 Ops/s 0.9303 Ops/s $\color{#d91a1a}-0.46\%$
test_serial 2.3976s 2.3315s 0.4289 Ops/s 0.4331 Ops/s $\color{#d91a1a}-0.97\%$
test_parallel 2.0450s 1.8833s 0.5310 Ops/s 0.5354 Ops/s $\color{#d91a1a}-0.82\%$
test_step_mdp_speed[True-True-True-True-True] 75.5300μs 32.3503μs 30.9117 KOps/s 30.7561 KOps/s $\color{#35bf28}+0.51\%$
test_step_mdp_speed[True-True-True-True-False] 44.5200μs 19.5123μs 51.2497 KOps/s 51.5659 KOps/s $\color{#d91a1a}-0.61\%$
test_step_mdp_speed[True-True-True-False-True] 42.2200μs 18.5112μs 54.0212 KOps/s 54.8984 KOps/s $\color{#d91a1a}-1.60\%$
test_step_mdp_speed[True-True-True-False-False] 30.3310μs 11.2208μs 89.1205 KOps/s 92.8051 KOps/s $\color{#d91a1a}-3.97\%$
test_step_mdp_speed[True-True-False-True-True] 52.0210μs 33.9160μs 29.4846 KOps/s 29.8440 KOps/s $\color{#d91a1a}-1.20\%$
test_step_mdp_speed[True-True-False-True-False] 42.7110μs 21.5415μs 46.4221 KOps/s 47.4337 KOps/s $\color{#d91a1a}-2.13\%$
test_step_mdp_speed[True-True-False-False-True] 38.0200μs 20.3030μs 49.2538 KOps/s 50.8286 KOps/s $\color{#d91a1a}-3.10\%$
test_step_mdp_speed[True-True-False-False-False] 37.6600μs 13.0837μs 76.4312 KOps/s 78.8469 KOps/s $\color{#d91a1a}-3.06\%$
test_step_mdp_speed[True-False-True-True-True] 59.6700μs 36.1660μs 27.6502 KOps/s 27.8547 KOps/s $\color{#d91a1a}-0.73\%$
test_step_mdp_speed[True-False-True-True-False] 43.2520μs 23.4578μs 42.6298 KOps/s 43.4567 KOps/s $\color{#d91a1a}-1.90\%$
test_step_mdp_speed[True-False-True-False-True] 36.6210μs 20.3518μs 49.1356 KOps/s 50.2485 KOps/s $\color{#d91a1a}-2.21\%$
test_step_mdp_speed[True-False-True-False-False] 27.5510μs 13.1384μs 76.1125 KOps/s 78.7932 KOps/s $\color{#d91a1a}-3.40\%$
test_step_mdp_speed[True-False-False-True-True] 64.0500μs 37.5068μs 26.6619 KOps/s 26.9688 KOps/s $\color{#d91a1a}-1.14\%$
test_step_mdp_speed[True-False-False-True-False] 41.9600μs 25.3055μs 39.5172 KOps/s 40.3437 KOps/s $\color{#d91a1a}-2.05\%$
test_step_mdp_speed[True-False-False-False-True] 38.4210μs 21.9445μs 45.5694 KOps/s 46.3051 KOps/s $\color{#d91a1a}-1.59\%$
test_step_mdp_speed[True-False-False-False-False] 32.8100μs 14.9519μs 66.8809 KOps/s 69.5416 KOps/s $\color{#d91a1a}-3.83\%$
test_step_mdp_speed[False-True-True-True-True] 55.9910μs 36.1131μs 27.6908 KOps/s 28.2992 KOps/s $\color{#d91a1a}-2.15\%$
test_step_mdp_speed[False-True-True-True-False] 40.5100μs 23.6458μs 42.2908 KOps/s 43.1043 KOps/s $\color{#d91a1a}-1.89\%$
test_step_mdp_speed[False-True-True-False-True] 48.4300μs 23.6171μs 42.3423 KOps/s 40.9101 KOps/s $\color{#35bf28}+3.50\%$
test_step_mdp_speed[False-True-True-False-False] 29.1300μs 14.8508μs 67.3367 KOps/s 68.9185 KOps/s $\color{#d91a1a}-2.30\%$
test_step_mdp_speed[False-True-False-True-True] 65.7200μs 38.0020μs 26.3144 KOps/s 26.4466 KOps/s $\color{#d91a1a}-0.50\%$
test_step_mdp_speed[False-True-False-True-False] 44.1820μs 25.3111μs 39.5084 KOps/s 39.5509 KOps/s $\color{#d91a1a}-0.11\%$
test_step_mdp_speed[False-True-False-False-True] 42.6300μs 25.6494μs 38.9872 KOps/s 38.2209 KOps/s $\color{#35bf28}+2.01\%$
test_step_mdp_speed[False-True-False-False-False] 36.3900μs 16.6269μs 60.1435 KOps/s 61.6617 KOps/s $\color{#d91a1a}-2.46\%$
test_step_mdp_speed[False-False-True-True-True] 64.8700μs 39.7739μs 25.1421 KOps/s 25.5399 KOps/s $\color{#d91a1a}-1.56\%$
test_step_mdp_speed[False-False-True-True-False] 46.1110μs 27.3173μs 36.6069 KOps/s 36.9019 KOps/s $\color{#d91a1a}-0.80\%$
test_step_mdp_speed[False-False-True-False-True] 41.1800μs 25.5371μs 39.1587 KOps/s 38.8192 KOps/s $\color{#35bf28}+0.87\%$
test_step_mdp_speed[False-False-True-False-False] 39.2400μs 16.6767μs 59.9639 KOps/s 60.9983 KOps/s $\color{#d91a1a}-1.70\%$
test_step_mdp_speed[False-False-False-True-True] 57.7000μs 41.6847μs 23.9896 KOps/s 24.6592 KOps/s $\color{#d91a1a}-2.72\%$
test_step_mdp_speed[False-False-False-True-False] 92.9300μs 28.8709μs 34.6370 KOps/s 35.0115 KOps/s $\color{#d91a1a}-1.07\%$
test_step_mdp_speed[False-False-False-False-True] 57.8700μs 27.0516μs 36.9664 KOps/s 36.9829 KOps/s $\color{#d91a1a}-0.04\%$
test_step_mdp_speed[False-False-False-False-False] 31.6610μs 18.4407μs 54.2278 KOps/s 55.8147 KOps/s $\color{#d91a1a}-2.84\%$
test_values[generalized_advantage_estimate-True-True] 26.5870ms 26.0893ms 38.3298 Ops/s 41.3480 Ops/s $\textbf{\color{#d91a1a}-7.30\%}$
test_values[vec_generalized_advantage_estimate-True-True] 85.4343ms 3.2683ms 305.9677 Ops/s 314.1450 Ops/s $\color{#d91a1a}-2.60\%$
test_values[td0_return_estimate-False-False] 95.9210μs 62.6894μs 15.9516 KOps/s 16.4817 KOps/s $\color{#d91a1a}-3.22\%$
test_values[td1_return_estimate-False-False] 56.5434ms 56.0201ms 17.8507 Ops/s 19.2167 Ops/s $\textbf{\color{#d91a1a}-7.11\%}$
test_values[vec_td1_return_estimate-False-False] 2.0824ms 1.7593ms 568.4171 Ops/s 570.2066 Ops/s $\color{#d91a1a}-0.31\%$
test_values[td_lambda_return_estimate-True-False] 86.1704ms 83.7717ms 11.9372 Ops/s 12.0178 Ops/s $\color{#d91a1a}-0.67\%$
test_values[vec_td_lambda_return_estimate-True-False] 2.0812ms 1.7549ms 569.8489 Ops/s 573.5132 Ops/s $\color{#d91a1a}-0.64\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 24.2901ms 23.3223ms 42.8773 Ops/s 42.1040 Ops/s $\color{#35bf28}+1.84\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 0.8353ms 0.6930ms 1.4431 KOps/s 1.4754 KOps/s $\color{#d91a1a}-2.19\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.7030ms 0.6474ms 1.5446 KOps/s 1.5200 KOps/s $\color{#35bf28}+1.62\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.5969ms 1.4533ms 688.1024 Ops/s 694.6549 Ops/s $\color{#d91a1a}-0.94\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.9294ms 0.6658ms 1.5018 KOps/s 1.5269 KOps/s $\color{#d91a1a}-1.64\%$
test_dqn_speed 7.9451ms 7.2422ms 138.0788 Ops/s 127.5102 Ops/s $\textbf{\color{#35bf28}+8.29\%}$
test_ddpg_speed 14.9276ms 14.1558ms 70.6426 Ops/s 70.9951 Ops/s $\color{#d91a1a}-0.50\%$
test_sac_speed 29.9652ms 28.8930ms 34.6104 Ops/s 34.8530 Ops/s $\color{#d91a1a}-0.70\%$
test_redq_speed 48.3216ms 47.4557ms 21.0723 Ops/s 21.3919 Ops/s $\color{#d91a1a}-1.49\%$
test_redq_deprec_speed 24.6487ms 23.7959ms 42.0241 Ops/s 42.3376 Ops/s $\color{#d91a1a}-0.74\%$
test_td3_speed 29.1614ms 19.7747ms 50.5698 Ops/s 50.9732 Ops/s $\color{#d91a1a}-0.79\%$
test_cql_speed 83.2911ms 82.4854ms 12.1234 Ops/s 12.2032 Ops/s $\color{#d91a1a}-0.65\%$
test_a2c_speed 27.3320ms 26.5015ms 37.7337 Ops/s 38.0935 Ops/s $\color{#d91a1a}-0.94\%$
test_ppo_speed 27.8864ms 26.7272ms 37.4151 Ops/s 37.7923 Ops/s $\color{#d91a1a}-1.00\%$
test_reinforce_speed 26.1287ms 25.5626ms 39.1196 Ops/s 39.4894 Ops/s $\color{#d91a1a}-0.94\%$
test_iql_speed 58.8566ms 57.1630ms 17.4938 Ops/s 17.6628 Ops/s $\color{#d91a1a}-0.96\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 2.1807ms 1.8156ms 550.7965 Ops/s 554.8572 Ops/s $\color{#d91a1a}-0.73\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.9549ms 0.8442ms 1.1845 KOps/s 1.1933 KOps/s $\color{#d91a1a}-0.73\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.9298ms 0.8205ms 1.2188 KOps/s 1.2251 KOps/s $\color{#d91a1a}-0.52\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 2.5613ms 1.7912ms 558.2888 Ops/s 568.9966 Ops/s $\color{#d91a1a}-1.88\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.0021ms 0.8353ms 1.1972 KOps/s 1.2085 KOps/s $\color{#d91a1a}-0.93\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.9454ms 0.8118ms 1.2318 KOps/s 1.2426 KOps/s $\color{#d91a1a}-0.87\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 2.9013ms 2.0531ms 487.0747 Ops/s 486.8649 Ops/s $\color{#35bf28}+0.04\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.1016ms 0.9654ms 1.0358 KOps/s 1.0461 KOps/s $\color{#d91a1a}-0.98\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 1.0692ms 0.9418ms 1.0618 KOps/s 1.0687 KOps/s $\color{#d91a1a}-0.65\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 2.6161ms 1.8286ms 546.8654 Ops/s 553.0109 Ops/s $\color{#d91a1a}-1.11\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.9587ms 0.8451ms 1.1833 KOps/s 1.1919 KOps/s $\color{#d91a1a}-0.72\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.9425ms 0.8235ms 1.2143 KOps/s 1.2241 KOps/s $\color{#d91a1a}-0.80\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 2.6480ms 1.7919ms 558.0524 Ops/s 563.5481 Ops/s $\color{#d91a1a}-0.98\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.9481ms 0.8341ms 1.1989 KOps/s 1.2080 KOps/s $\color{#d91a1a}-0.75\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.9433ms 0.8129ms 1.2301 KOps/s 1.2414 KOps/s $\color{#d91a1a}-0.91\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.0426ms 2.0632ms 484.6744 Ops/s 487.3571 Ops/s $\color{#d91a1a}-0.55\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.0868ms 0.9658ms 1.0354 KOps/s 1.0440 KOps/s $\color{#d91a1a}-0.82\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 1.0573ms 0.9452ms 1.0580 KOps/s 1.0678 KOps/s $\color{#d91a1a}-0.92\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1428s 10.4544ms 95.6539 Ops/s 99.7527 Ops/s $\color{#d91a1a}-4.11\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 16.2184ms 14.0845ms 70.9999 Ops/s 73.8940 Ops/s $\color{#d91a1a}-3.92\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 5.1916ms 3.3265ms 300.6125 Ops/s 303.0239 Ops/s $\color{#d91a1a}-0.80\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1166s 9.8772ms 101.2431 Ops/s 100.8934 Ops/s $\color{#35bf28}+0.35\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 16.2836ms 14.0281ms 71.2853 Ops/s 73.9323 Ops/s $\color{#d91a1a}-3.58\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 5.6440ms 3.3342ms 299.9224 Ops/s 301.1877 Ops/s $\color{#d91a1a}-0.42\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1164s 12.3066ms 81.2571 Ops/s 81.0953 Ops/s $\color{#35bf28}+0.20\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 16.5389ms 14.2102ms 70.3719 Ops/s 72.4031 Ops/s $\color{#d91a1a}-2.81\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 5.9222ms 3.5174ms 284.3021 Ops/s 289.0584 Ops/s $\color{#d91a1a}-1.65\%$

@vmoens vmoens deleted the group-gendgrl-ci branch February 27, 2024 00:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
CI Has to do with CI setup (e.g. wheels & builds, tests...) CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. Data Data-related PR, will launch data-related jobs
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@vmoens @facebook-github-bot