Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature] Deterministic sample for Masked one-hot #2440

Merged
merged 1 commit into from
Sep 17, 2024
Merged

[Feature] Deterministic sample for Masked one-hot #2440

merged 1 commit into from
Sep 17, 2024

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Sep 17, 2024 •
edited
Loading

@vmoens
Update
3d7a5c4 
[ghstack-poisoned]
@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Sep 17, 2024 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2440

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

❌ 2 New Failures, 19 Unrelated Failures

As of commit 3d7a5c4 with merge base 0a410ff (image):

NEW FAILURES - The following jobs have failed:

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

vmoens added a commit that referenced this pull request Sep 17, 2024
@vmoens
[Feature] Deterministic sample for Masked one-hot
a6d7545 
ghstack-source-id: 27787eab47324c5af152f706d81687e71b5b9803
Pull Request resolved: #2440
@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Sep 17, 2024
@vmoens vmoens linked an issue Sep 17, 2024 that may be closed by this pull request
[BUG] MaskedOneHotCategorical missing mode and deterministic_sample properties. #2439
Closed
3 tasks
@vmoens vmoens merged commit 3d7a5c4 into gh/vmoens/26/base Sep 17, 2024
42 of 59 checks passed
vmoens added a commit that referenced this pull request Sep 17, 2024
@vmoens
[Feature] Deterministic sample for Masked one-hot
e294c68 
ghstack-source-id: 27787eab47324c5af152f706d81687e71b5b9803
Pull Request resolved: #2440
@vmoens vmoens deleted the gh/vmoens/26/head branch September 17, 2024 21:39
@vmoens vmoens added the enhancement New feature or request label Sep 17, 2024
@github-actions GitHub Actions
Copy link

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 146. Improved: $\large\color{#35bf28}16$. Worsened: $\large\color{#d91a1a}8$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 60.2391ms 59.6229ms 16.7721 Ops/s 16.1562 Ops/s $\color{#35bf28}+3.81\%$
test_sync 43.8213ms 33.2538ms 30.0718 Ops/s 26.8689 Ops/s $\textbf{\color{#35bf28}+11.92\%}$
test_async 86.3212ms 31.3953ms 31.8519 Ops/s 31.5230 Ops/s $\color{#35bf28}+1.04\%$
test_simple 0.5086s 0.4308s 2.3213 Ops/s 2.4199 Ops/s $\color{#d91a1a}-4.07\%$
test_transformed 0.5686s 0.5677s 1.7615 Ops/s 1.7210 Ops/s $\color{#35bf28}+2.35\%$
test_serial 1.2712s 1.2671s 0.7892 Ops/s 0.7634 Ops/s $\color{#35bf28}+3.37\%$
test_parallel 1.2354s 1.1463s 0.8724 Ops/s 0.8678 Ops/s $\color{#35bf28}+0.52\%$
test_step_mdp_speed[True-True-True-True-True] 0.4229ms 27.2676μs 36.6736 KOps/s 35.7630 KOps/s $\color{#35bf28}+2.55\%$
test_step_mdp_speed[True-True-True-True-False] 48.2810μs 16.1426μs 61.9480 KOps/s 60.9553 KOps/s $\color{#35bf28}+1.63\%$
test_step_mdp_speed[True-True-True-False-True] 49.6530μs 15.9816μs 62.5721 KOps/s 61.4665 KOps/s $\color{#35bf28}+1.80\%$
test_step_mdp_speed[True-True-True-False-False] 39.9650μs 9.3336μs 107.1402 KOps/s 104.9587 KOps/s $\color{#35bf28}+2.08\%$
test_step_mdp_speed[True-True-False-True-True] 0.1074ms 29.7066μs 33.6626 KOps/s 33.5054 KOps/s $\color{#35bf28}+0.47\%$
test_step_mdp_speed[True-True-False-True-False] 67.7070μs 17.7872μs 56.2201 KOps/s 55.0677 KOps/s $\color{#35bf28}+2.09\%$
test_step_mdp_speed[True-True-False-False-True] 57.3070μs 17.5098μs 57.1108 KOps/s 55.3818 KOps/s $\color{#35bf28}+3.12\%$
test_step_mdp_speed[True-True-False-False-False] 51.9670μs 10.7690μs 92.8591 KOps/s 89.0491 KOps/s $\color{#35bf28}+4.28\%$
test_step_mdp_speed[True-False-True-True-True] 74.6690μs 30.7509μs 32.5194 KOps/s 31.0822 KOps/s $\color{#35bf28}+4.62\%$
test_step_mdp_speed[True-False-True-True-False] 57.5380μs 19.3016μs 51.8091 KOps/s 50.0766 KOps/s $\color{#35bf28}+3.46\%$
test_step_mdp_speed[True-False-True-False-True] 74.7400μs 17.4800μs 57.2082 KOps/s 55.6899 KOps/s $\color{#35bf28}+2.73\%$
test_step_mdp_speed[True-False-True-False-False] 55.0230μs 10.8898μs 91.8291 KOps/s 89.3785 KOps/s $\color{#35bf28}+2.74\%$
test_step_mdp_speed[True-False-False-True-True] 63.6590μs 32.6396μs 30.6376 KOps/s 30.0119 KOps/s $\color{#35bf28}+2.08\%$
test_step_mdp_speed[True-False-False-True-False] 55.8640μs 21.1786μs 47.2174 KOps/s 46.8919 KOps/s $\color{#35bf28}+0.69\%$
test_step_mdp_speed[True-False-False-False-True] 51.5760μs 19.1833μs 52.1288 KOps/s 52.1481 KOps/s $\color{#d91a1a}-0.04\%$
test_step_mdp_speed[True-False-False-False-False] 38.8020μs 12.4573μs 80.2743 KOps/s 79.3085 KOps/s $\color{#35bf28}+1.22\%$
test_step_mdp_speed[False-True-True-True-True] 69.3200μs 31.2080μs 32.0431 KOps/s 31.6993 KOps/s $\color{#35bf28}+1.08\%$
test_step_mdp_speed[False-True-True-True-False] 55.3740μs 19.6236μs 50.9592 KOps/s 50.5195 KOps/s $\color{#35bf28}+0.87\%$
test_step_mdp_speed[False-True-True-False-True] 66.7950μs 20.3568μs 49.1236 KOps/s 48.5662 KOps/s $\color{#35bf28}+1.15\%$
test_step_mdp_speed[False-True-True-False-False] 38.7930μs 12.1559μs 82.2648 KOps/s 81.3139 KOps/s $\color{#35bf28}+1.17\%$
test_step_mdp_speed[False-True-False-True-True] 73.2270μs 32.5941μs 30.6804 KOps/s 30.1799 KOps/s $\color{#35bf28}+1.66\%$
test_step_mdp_speed[False-True-False-True-False] 60.3030μs 21.1556μs 47.2688 KOps/s 46.5076 KOps/s $\color{#35bf28}+1.64\%$
test_step_mdp_speed[False-True-False-False-True] 3.0368ms 21.6207μs 46.2521 KOps/s 44.2124 KOps/s $\color{#35bf28}+4.61\%$
test_step_mdp_speed[False-True-False-False-False] 60.3120μs 13.6410μs 73.3083 KOps/s 72.2169 KOps/s $\color{#35bf28}+1.51\%$
test_step_mdp_speed[False-False-True-True-True] 99.3750μs 34.3934μs 29.0753 KOps/s 28.6976 KOps/s $\color{#35bf28}+1.32\%$
test_step_mdp_speed[False-False-True-True-False] 61.9550μs 22.6330μs 44.1833 KOps/s 43.7551 KOps/s $\color{#35bf28}+0.98\%$
test_step_mdp_speed[False-False-True-False-True] 56.4050μs 21.9647μs 45.5276 KOps/s 44.9827 KOps/s $\color{#35bf28}+1.21\%$
test_step_mdp_speed[False-False-True-False-False] 47.0480μs 13.6837μs 73.0796 KOps/s 72.1788 KOps/s $\color{#35bf28}+1.25\%$
test_step_mdp_speed[False-False-False-True-True] 94.5770μs 35.5837μs 28.1028 KOps/s 27.5073 KOps/s $\color{#35bf28}+2.16\%$
test_step_mdp_speed[False-False-False-True-False] 60.1620μs 24.1677μs 41.3775 KOps/s 40.8620 KOps/s $\color{#35bf28}+1.26\%$
test_step_mdp_speed[False-False-False-False-True] 57.3770μs 22.9545μs 43.5644 KOps/s 42.4410 KOps/s $\color{#35bf28}+2.65\%$
test_step_mdp_speed[False-False-False-False-False] 40.4350μs 15.1374μs 66.0616 KOps/s 65.0777 KOps/s $\color{#35bf28}+1.51\%$
test_values[generalized_advantage_estimate-True-True] 11.8142ms 9.7435ms 102.6330 Ops/s 104.8069 Ops/s $\color{#d91a1a}-2.07\%$
test_values[vec_generalized_advantage_estimate-True-True] 41.1429ms 36.6730ms 27.2680 Ops/s 29.5853 Ops/s $\textbf{\color{#d91a1a}-7.83\%}$
test_values[td0_return_estimate-False-False] 0.3491ms 0.1908ms 5.2413 KOps/s 5.5722 KOps/s $\textbf{\color{#d91a1a}-5.94\%}$
test_values[td1_return_estimate-False-False] 28.0582ms 24.4505ms 40.8990 Ops/s 41.8724 Ops/s $\color{#d91a1a}-2.32\%$
test_values[vec_td1_return_estimate-False-False] 39.7145ms 36.9743ms 27.0458 Ops/s 29.7052 Ops/s $\textbf{\color{#d91a1a}-8.95\%}$
test_values[td_lambda_return_estimate-True-False] 37.8953ms 34.6995ms 28.8188 Ops/s 29.2493 Ops/s $\color{#d91a1a}-1.47\%$
test_values[vec_td_lambda_return_estimate-True-False] 38.7656ms 36.7910ms 27.1805 Ops/s 29.5203 Ops/s $\textbf{\color{#d91a1a}-7.93\%}$
test_gae_speed[generalized_advantage_estimate-False-1-512] 8.4261ms 8.2766ms 120.8220 Ops/s 120.3809 Ops/s $\color{#35bf28}+0.37\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 2.3885ms 1.9840ms 504.0337 Ops/s 558.0304 Ops/s $\textbf{\color{#d91a1a}-9.68\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.4800ms 0.3578ms 2.7952 KOps/s 2.7711 KOps/s $\color{#35bf28}+0.87\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 45.4745ms 44.7018ms 22.3705 Ops/s 24.6612 Ops/s $\textbf{\color{#d91a1a}-9.29\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 4.0036ms 3.0523ms 327.6171 Ops/s 329.0221 Ops/s $\color{#d91a1a}-0.43\%$
test_dqn_speed[False-None] 6.1182ms 1.3144ms 760.8210 Ops/s 733.5709 Ops/s $\color{#35bf28}+3.71\%$
test_dqn_speed[False-backward] 1.8799ms 1.8024ms 554.8192 Ops/s 544.5373 Ops/s $\color{#35bf28}+1.89\%$
test_dqn_speed[True-None] 0.5741ms 0.4543ms 2.2013 KOps/s 2.1527 KOps/s $\color{#35bf28}+2.26\%$
test_dqn_speed[True-backward] 0.9707ms 0.8866ms 1.1280 KOps/s 1.0972 KOps/s $\color{#35bf28}+2.80\%$
test_dqn_speed[reduce-overhead-None] 0.6279ms 0.4667ms 2.1426 KOps/s 2.1254 KOps/s $\color{#35bf28}+0.81\%$
test_dqn_speed[reduce-overhead-backward] 1.0090ms 0.9010ms 1.1099 KOps/s 1.1276 KOps/s $\color{#d91a1a}-1.57\%$
test_ddpg_speed[False-None] 4.1592ms 2.8151ms 355.2213 Ops/s 342.8958 Ops/s $\color{#35bf28}+3.59\%$
test_ddpg_speed[False-backward] 4.4155ms 4.0298ms 248.1535 Ops/s 252.3998 Ops/s $\color{#d91a1a}-1.68\%$
test_ddpg_speed[True-None] 1.4049ms 1.0084ms 991.6996 Ops/s 977.8387 Ops/s $\color{#35bf28}+1.42\%$
test_ddpg_speed[True-backward] 1.9452ms 1.8830ms 531.0790 Ops/s 460.2391 Ops/s $\textbf{\color{#35bf28}+15.39\%}$
test_ddpg_speed[reduce-overhead-None] 1.6607ms 1.0074ms 992.6882 Ops/s 982.3153 Ops/s $\color{#35bf28}+1.06\%$
test_ddpg_speed[reduce-overhead-backward] 1.9375ms 1.8824ms 531.2488 Ops/s 519.8863 Ops/s $\color{#35bf28}+2.19\%$
test_sac_speed[False-None] 10.1727ms 8.0658ms 123.9799 Ops/s 122.3589 Ops/s $\color{#35bf28}+1.32\%$
test_sac_speed[False-backward] 12.2997ms 10.9969ms 90.9351 Ops/s 90.6710 Ops/s $\color{#35bf28}+0.29\%$
test_sac_speed[True-None] 2.2008ms 1.8476ms 541.2410 Ops/s 532.9716 Ops/s $\color{#35bf28}+1.55\%$
test_sac_speed[True-backward] 3.8018ms 3.6059ms 277.3248 Ops/s 281.0401 Ops/s $\color{#d91a1a}-1.32\%$
test_sac_speed[reduce-overhead-None] 2.3657ms 1.8478ms 541.1732 Ops/s 532.1569 Ops/s $\color{#35bf28}+1.69\%$
test_sac_speed[reduce-overhead-backward] 4.0016ms 3.6622ms 273.0624 Ops/s 277.3137 Ops/s $\color{#d91a1a}-1.53\%$
test_redq_speed[False-None] 15.3670ms 13.2853ms 75.2712 Ops/s 74.4324 Ops/s $\color{#35bf28}+1.13\%$
test_redq_speed[False-backward] 26.3833ms 23.3817ms 42.7685 Ops/s 43.4901 Ops/s $\color{#d91a1a}-1.66\%$
test_redq_speed[True-None] 6.2994ms 5.0720ms 197.1614 Ops/s 182.3094 Ops/s $\textbf{\color{#35bf28}+8.15\%}$
test_redq_speed[True-backward] 13.8042ms 12.8017ms 78.1148 Ops/s 76.4635 Ops/s $\color{#35bf28}+2.16\%$
test_redq_speed[reduce-overhead-None] 6.0977ms 4.8506ms 206.1610 Ops/s 182.8045 Ops/s $\textbf{\color{#35bf28}+12.78\%}$
test_redq_speed[reduce-overhead-backward] 13.8331ms 12.6533ms 79.0308 Ops/s 78.3616 Ops/s $\color{#35bf28}+0.85\%$
test_redq_deprec_speed[False-None] 14.3576ms 13.1484ms 76.0547 Ops/s 51.7946 Ops/s $\textbf{\color{#35bf28}+46.84\%}$
test_redq_deprec_speed[False-backward] 20.8283ms 19.0380ms 52.5266 Ops/s 49.5054 Ops/s $\textbf{\color{#35bf28}+6.10\%}$
test_redq_deprec_speed[True-None] 4.4812ms 3.7424ms 267.2080 Ops/s 262.3155 Ops/s $\color{#35bf28}+1.87\%$
test_redq_deprec_speed[True-backward] 9.0412ms 8.1948ms 122.0293 Ops/s 112.0077 Ops/s $\textbf{\color{#35bf28}+8.95\%}$
test_redq_deprec_speed[reduce-overhead-None] 5.2575ms 4.0657ms 245.9571 Ops/s 255.5489 Ops/s $\color{#d91a1a}-3.75\%$
test_redq_deprec_speed[reduce-overhead-backward] 9.1381ms 8.4377ms 118.5154 Ops/s 116.3842 Ops/s $\color{#35bf28}+1.83\%$
test_td3_speed[False-None] 8.5071ms 7.9289ms 126.1209 Ops/s 122.0193 Ops/s $\color{#35bf28}+3.36\%$
test_td3_speed[False-backward] 14.4818ms 10.8280ms 92.3531 Ops/s 85.7540 Ops/s $\textbf{\color{#35bf28}+7.70\%}$
test_td3_speed[True-None] 2.1370ms 1.9228ms 520.0848 Ops/s 498.9120 Ops/s $\color{#35bf28}+4.24\%$
test_td3_speed[True-backward] 3.5879ms 3.4931ms 286.2761 Ops/s 276.7175 Ops/s $\color{#35bf28}+3.45\%$
test_td3_speed[reduce-overhead-None] 2.0746ms 1.9118ms 523.0714 Ops/s 497.2634 Ops/s $\textbf{\color{#35bf28}+5.19\%}$
test_td3_speed[reduce-overhead-backward] 4.1247ms 3.6429ms 274.5096 Ops/s 268.4126 Ops/s $\color{#35bf28}+2.27\%$
test_cql_speed[False-None] 40.2967ms 36.1161ms 27.6885 Ops/s 27.3303 Ops/s $\color{#35bf28}+1.31\%$
test_cql_speed[False-backward] 51.1114ms 47.6680ms 20.9784 Ops/s 21.3016 Ops/s $\color{#d91a1a}-1.52\%$
test_cql_speed[True-None] 17.0622ms 15.9211ms 62.8096 Ops/s 61.5944 Ops/s $\color{#35bf28}+1.97\%$
test_cql_speed[True-backward] 24.4346ms 22.7615ms 43.9338 Ops/s 43.3031 Ops/s $\color{#35bf28}+1.46\%$
test_cql_speed[reduce-overhead-None] 17.2518ms 16.0089ms 62.4651 Ops/s 61.6583 Ops/s $\color{#35bf28}+1.31\%$
test_cql_speed[reduce-overhead-backward] 24.1008ms 22.7417ms 43.9722 Ops/s 43.1405 Ops/s $\color{#35bf28}+1.93\%$
test_a2c_speed[False-None] 8.8639ms 7.4036ms 135.0697 Ops/s 135.3435 Ops/s $\color{#d91a1a}-0.20\%$
test_a2c_speed[False-backward] 16.3321ms 15.2351ms 65.6377 Ops/s 67.0797 Ops/s $\color{#d91a1a}-2.15\%$
test_a2c_speed[True-None] 3.9579ms 3.3671ms 296.9896 Ops/s 293.7267 Ops/s $\color{#35bf28}+1.11\%$
test_a2c_speed[True-backward] 11.7105ms 10.4183ms 95.9852 Ops/s 89.7088 Ops/s $\textbf{\color{#35bf28}+7.00\%}$
test_a2c_speed[reduce-overhead-None] 3.9628ms 3.3361ms 299.7521 Ops/s 291.8175 Ops/s $\color{#35bf28}+2.72\%$
test_a2c_speed[reduce-overhead-backward] 11.1611ms 10.5779ms 94.5371 Ops/s 93.2749 Ops/s $\color{#35bf28}+1.35\%$
test_ppo_speed[False-None] 9.4023ms 7.7772ms 128.5815 Ops/s 129.3179 Ops/s $\color{#d91a1a}-0.57\%$
test_ppo_speed[False-backward] 16.8030ms 15.5485ms 64.3147 Ops/s 60.8094 Ops/s $\textbf{\color{#35bf28}+5.76\%}$
test_ppo_speed[True-None] 4.9610ms 3.9402ms 253.7935 Ops/s 261.1279 Ops/s $\color{#d91a1a}-2.81\%$
test_ppo_speed[True-backward] 11.0958ms 10.1304ms 98.7130 Ops/s 96.8803 Ops/s $\color{#35bf28}+1.89\%$
test_ppo_speed[reduce-overhead-None] 5.9306ms 3.7708ms 265.1929 Ops/s 254.9564 Ops/s $\color{#35bf28}+4.02\%$
test_ppo_speed[reduce-overhead-backward] 12.3269ms 10.7364ms 93.1407 Ops/s 95.9393 Ops/s $\color{#d91a1a}-2.92\%$
test_reinforce_speed[False-None] 8.9099ms 6.6222ms 151.0079 Ops/s 149.7228 Ops/s $\color{#35bf28}+0.86\%$
test_reinforce_speed[False-backward] 10.9246ms 10.2762ms 97.3123 Ops/s 99.9683 Ops/s $\color{#d91a1a}-2.66\%$
test_reinforce_speed[True-None] 3.2298ms 2.7045ms 369.7573 Ops/s 365.1241 Ops/s $\color{#35bf28}+1.27\%$
test_reinforce_speed[True-backward] 10.2582ms 9.1220ms 109.6246 Ops/s 104.2035 Ops/s $\textbf{\color{#35bf28}+5.20\%}$
test_reinforce_speed[reduce-overhead-None] 3.2611ms 2.7224ms 367.3182 Ops/s 367.8513 Ops/s $\color{#d91a1a}-0.14\%$
test_reinforce_speed[reduce-overhead-backward] 9.8288ms 9.0591ms 110.3858 Ops/s 110.6962 Ops/s $\color{#d91a1a}-0.28\%$
test_iql_speed[False-None] 33.9304ms 32.3003ms 30.9594 Ops/s 30.2557 Ops/s $\color{#35bf28}+2.33\%$
test_iql_speed[False-backward] 47.5375ms 45.4607ms 21.9970 Ops/s 21.7593 Ops/s $\color{#35bf28}+1.09\%$
test_iql_speed[True-None] 14.7251ms 13.7327ms 72.8191 Ops/s 70.4937 Ops/s $\color{#35bf28}+3.30\%$
test_iql_speed[True-backward] 27.1005ms 25.7887ms 38.7766 Ops/s 37.4460 Ops/s $\color{#35bf28}+3.55\%$
test_iql_speed[reduce-overhead-None] 15.4490ms 14.2860ms 69.9985 Ops/s 70.7223 Ops/s $\color{#d91a1a}-1.02\%$
test_iql_speed[reduce-overhead-backward] 27.6167ms 25.6143ms 39.0407 Ops/s 39.0143 Ops/s $\color{#35bf28}+0.07\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 5.7016ms 5.3365ms 187.3884 Ops/s 183.5819 Ops/s $\color{#35bf28}+2.07\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.7794ms 0.4921ms 2.0321 KOps/s 2.0293 KOps/s $\color{#35bf28}+0.14\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.8703ms 0.4648ms 2.1516 KOps/s 2.1523 KOps/s $\color{#d91a1a}-0.04\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 7.5598ms 5.3112ms 188.2800 Ops/s 188.8679 Ops/s $\color{#d91a1a}-0.31\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.3274ms 0.4864ms 2.0560 KOps/s 2.0052 KOps/s $\color{#35bf28}+2.54\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7392ms 0.4615ms 2.1666 KOps/s 2.1319 KOps/s $\color{#35bf28}+1.63\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 2.2198ms 1.6104ms 620.9706 Ops/s 603.5689 Ops/s $\color{#35bf28}+2.88\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 2.0207ms 1.5190ms 658.3469 Ops/s 637.5827 Ops/s $\color{#35bf28}+3.26\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 5.9667ms 5.4664ms 182.9356 Ops/s 173.6810 Ops/s $\textbf{\color{#35bf28}+5.33\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 2.6226ms 0.6256ms 1.5985 KOps/s 1.5675 KOps/s $\color{#35bf28}+1.98\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8616ms 0.5995ms 1.6681 KOps/s 1.6530 KOps/s $\color{#35bf28}+0.91\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 8.2181ms 5.3730ms 186.1161 Ops/s 183.9574 Ops/s $\color{#35bf28}+1.17\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.6260ms 0.4848ms 2.0626 KOps/s 2.0193 KOps/s $\color{#35bf28}+2.14\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6242ms 0.4746ms 2.1070 KOps/s 2.0435 KOps/s $\color{#35bf28}+3.11\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 7.6266ms 5.3461ms 187.0508 Ops/s 177.8817 Ops/s $\textbf{\color{#35bf28}+5.15\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.8370ms 0.4817ms 2.0759 KOps/s 2.0303 KOps/s $\color{#35bf28}+2.25\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6812ms 0.4701ms 2.1274 KOps/s 2.1154 KOps/s $\color{#35bf28}+0.57\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 8.4189ms 5.5535ms 180.0678 Ops/s 179.6208 Ops/s $\color{#35bf28}+0.25\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 3.2119ms 0.6155ms 1.6248 KOps/s 1.5481 KOps/s $\color{#35bf28}+4.95\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8214ms 0.5910ms 1.6921 KOps/s 1.6200 KOps/s $\color{#35bf28}+4.45\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 5.6367ms 4.2920ms 232.9927 Ops/s 224.6502 Ops/s $\color{#35bf28}+3.71\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 0.4487s 22.4636ms 44.5165 Ops/s 73.4356 Ops/s $\textbf{\color{#d91a1a}-39.38\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 2.0307ms 1.2368ms 808.5660 Ops/s 672.7895 Ops/s $\textbf{\color{#35bf28}+20.18\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 5.7746ms 4.2561ms 234.9560 Ops/s 228.9375 Ops/s $\color{#35bf28}+2.63\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 18.7386ms 13.7254ms 72.8574 Ops/s 73.8091 Ops/s $\color{#d91a1a}-1.29\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 4.9511ms 1.4058ms 711.3489 Ops/s 710.4578 Ops/s $\color{#35bf28}+0.13\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.4259s 13.0879ms 76.4066 Ops/s 230.2064 Ops/s $\textbf{\color{#d91a1a}-66.81\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 19.6205ms 14.1085ms 70.8793 Ops/s 25.0974 Ops/s $\textbf{\color{#35bf28}+182.42\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 2.3632ms 1.5438ms 647.7625 Ops/s 629.3624 Ops/s $\color{#35bf28}+2.92\%$

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[BUG] MaskedOneHotCategorical missing mode and deterministic_sample properties.
2 participants
@vmoens @facebook-github-bot