Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Versioning] Allow any torch version for local builds #764

Merged
merged 1 commit into from
Apr 29, 2024
Merged

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Apr 29, 2024

Closing pytorch/rl#2124

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Apr 29, 2024
Copy link

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 127. Improved: $\large\color{#35bf28}10$. Worsened: $\large\color{#d91a1a}15$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_plain_set_nested 35.3860μs 17.5559μs 56.9609 KOps/s 57.5380 KOps/s $\color{#d91a1a}-1.00\%$
test_plain_set_stack_nested 43.3310μs 17.5053μs 57.1257 KOps/s 57.1334 KOps/s $\color{#d91a1a}-0.01\%$
test_plain_set_nested_inplace 64.8840μs 19.9954μs 50.0115 KOps/s 50.1690 KOps/s $\color{#d91a1a}-0.31\%$
test_plain_set_stack_nested_inplace 62.6680μs 19.9133μs 50.2178 KOps/s 50.6054 KOps/s $\color{#d91a1a}-0.77\%$
test_items 21.8810μs 2.5431μs 393.2182 KOps/s 391.0765 KOps/s $\color{#35bf28}+0.55\%$
test_items_nested 1.0449ms 0.2647ms 3.7782 KOps/s 3.6443 KOps/s $\color{#35bf28}+3.67\%$
test_items_nested_locked 0.4878ms 0.2638ms 3.7905 KOps/s 3.5546 KOps/s $\textbf{\color{#35bf28}+6.64\%}$
test_items_nested_leaf 0.1564ms 76.5511μs 13.0632 KOps/s 12.8787 KOps/s $\color{#35bf28}+1.43\%$
test_items_stack_nested 0.4931ms 0.2636ms 3.7942 KOps/s 3.6068 KOps/s $\textbf{\color{#35bf28}+5.20\%}$
test_items_stack_nested_leaf 0.1869ms 80.9586μs 12.3520 KOps/s 12.3686 KOps/s $\color{#d91a1a}-0.13\%$
test_items_stack_nested_locked 0.4984ms 0.2638ms 3.7902 KOps/s 3.5808 KOps/s $\textbf{\color{#35bf28}+5.85\%}$
test_keys 27.1610μs 3.8859μs 257.3439 KOps/s 263.1797 KOps/s $\color{#d91a1a}-2.22\%$
test_keys_nested 0.1868ms 0.1384ms 7.2241 KOps/s 6.9198 KOps/s $\color{#35bf28}+4.40\%$
test_keys_nested_locked 1.8908ms 0.1447ms 6.9087 KOps/s 6.7713 KOps/s $\color{#35bf28}+2.03\%$
test_keys_nested_leaf 0.1728ms 0.1187ms 8.4263 KOps/s 8.2925 KOps/s $\color{#35bf28}+1.61\%$
test_keys_stack_nested 0.2469ms 0.1389ms 7.2017 KOps/s 7.0271 KOps/s $\color{#35bf28}+2.49\%$
test_keys_stack_nested_leaf 0.1947ms 0.1187ms 8.4237 KOps/s 8.2329 KOps/s $\color{#35bf28}+2.32\%$
test_keys_stack_nested_locked 0.2661ms 0.1435ms 6.9685 KOps/s 6.7967 KOps/s $\color{#35bf28}+2.53\%$
test_values 10.3717μs 1.1914μs 839.3617 KOps/s 857.7249 KOps/s $\color{#d91a1a}-2.14\%$
test_values_nested 93.6150μs 51.1345μs 19.5563 KOps/s 18.8874 KOps/s $\color{#35bf28}+3.54\%$
test_values_nested_locked 0.1021ms 51.6156μs 19.3740 KOps/s 18.8520 KOps/s $\color{#35bf28}+2.77\%$
test_values_nested_leaf 92.2960μs 45.9304μs 21.7721 KOps/s 21.0691 KOps/s $\color{#35bf28}+3.34\%$
test_values_stack_nested 0.1014ms 51.6770μs 19.3510 KOps/s 18.6929 KOps/s $\color{#35bf28}+3.52\%$
test_values_stack_nested_leaf 95.3590μs 45.9864μs 21.7455 KOps/s 21.1304 KOps/s $\color{#35bf28}+2.91\%$
test_values_stack_nested_locked 0.1018ms 51.4391μs 19.4405 KOps/s 18.6289 KOps/s $\color{#35bf28}+4.36\%$
test_membership 27.7620μs 1.3509μs 740.2264 KOps/s 753.3868 KOps/s $\color{#d91a1a}-1.75\%$
test_membership_nested 37.2500μs 3.4823μs 287.1634 KOps/s 294.7402 KOps/s $\color{#d91a1a}-2.57\%$
test_membership_nested_leaf 26.2690μs 3.4896μs 286.5645 KOps/s 291.3894 KOps/s $\color{#d91a1a}-1.66\%$
test_membership_stacked_nested 37.4100μs 3.4638μs 288.6997 KOps/s 295.3525 KOps/s $\color{#d91a1a}-2.25\%$
test_membership_stacked_nested_leaf 30.2670μs 3.4775μs 287.5608 KOps/s 296.2299 KOps/s $\color{#d91a1a}-2.93\%$
test_membership_nested_last 29.6160μs 4.1924μs 238.5268 KOps/s 211.0784 KOps/s $\textbf{\color{#35bf28}+13.00\%}$
test_membership_nested_leaf_last 42.5800μs 4.2716μs 234.1064 KOps/s 233.8688 KOps/s $\color{#35bf28}+0.10\%$
test_membership_stacked_nested_last 29.1150μs 4.1976μs 238.2340 KOps/s 205.2481 KOps/s $\textbf{\color{#35bf28}+16.07\%}$
test_membership_stacked_nested_leaf_last 19.1660μs 4.2160μs 237.1944 KOps/s 207.3990 KOps/s $\textbf{\color{#35bf28}+14.37\%}$
test_nested_getleaf 41.7590μs 10.7759μs 92.7993 KOps/s 94.0015 KOps/s $\color{#d91a1a}-1.28\%$
test_nested_get 33.7130μs 10.2082μs 97.9604 KOps/s 98.3236 KOps/s $\color{#d91a1a}-0.37\%$
test_stacked_getleaf 50.5040μs 10.7949μs 92.6360 KOps/s 94.0149 KOps/s $\color{#d91a1a}-1.47\%$
test_stacked_get 54.7430μs 10.1574μs 98.4508 KOps/s 99.2398 KOps/s $\color{#d91a1a}-0.79\%$
test_nested_getitemleaf 34.9560μs 11.4094μs 87.6467 KOps/s 89.9559 KOps/s $\color{#d91a1a}-2.57\%$
test_nested_getitem 47.4590μs 10.6165μs 94.1931 KOps/s 97.2993 KOps/s $\color{#d91a1a}-3.19\%$
test_stacked_getitemleaf 50.6940μs 11.3602μs 88.0265 KOps/s 90.4618 KOps/s $\color{#d91a1a}-2.69\%$
test_stacked_getitem 30.6170μs 10.5939μs 94.3936 KOps/s 98.1521 KOps/s $\color{#d91a1a}-3.83\%$
test_lock_nested 46.9829ms 0.3941ms 2.5374 KOps/s 2.8311 KOps/s $\textbf{\color{#d91a1a}-10.38\%}$
test_lock_stack_nested 0.5895ms 0.3111ms 3.2140 KOps/s 3.2146 KOps/s $\color{#d91a1a}-0.02\%$
test_unlock_nested 0.6967ms 0.3480ms 2.8736 KOps/s 2.5078 KOps/s $\textbf{\color{#35bf28}+14.59\%}$
test_unlock_stack_nested 0.4434ms 0.3179ms 3.1453 KOps/s 3.1445 KOps/s $\color{#35bf28}+0.02\%$
test_flatten_speed 0.2045ms 96.8253μs 10.3279 KOps/s 10.3395 KOps/s $\color{#d91a1a}-0.11\%$
test_unflatten_speed 0.7228ms 0.4095ms 2.4420 KOps/s 2.3767 KOps/s $\color{#35bf28}+2.75\%$
test_common_ops 1.3597ms 0.7244ms 1.3804 KOps/s 1.4541 KOps/s $\textbf{\color{#d91a1a}-5.07\%}$
test_creation 24.2460μs 1.9532μs 511.9917 KOps/s 517.0694 KOps/s $\color{#d91a1a}-0.98\%$
test_creation_empty 59.5710μs 11.4381μs 87.4271 KOps/s 102.3369 KOps/s $\textbf{\color{#d91a1a}-14.57\%}$
test_creation_nested_1 37.5000μs 14.1026μs 70.9089 KOps/s 80.8220 KOps/s $\textbf{\color{#d91a1a}-12.27\%}$
test_creation_nested_2 55.1230μs 17.5972μs 56.8271 KOps/s 63.2273 KOps/s $\textbf{\color{#d91a1a}-10.12\%}$
test_clone 0.1044ms 13.6120μs 73.4647 KOps/s 74.7080 KOps/s $\color{#d91a1a}-1.66\%$
test_getitem[int] 29.9760μs 11.6650μs 85.7268 KOps/s 87.4367 KOps/s $\color{#d91a1a}-1.96\%$
test_getitem[slice_int] 81.1220μs 23.0460μs 43.3915 KOps/s 43.7986 KOps/s $\color{#d91a1a}-0.93\%$
test_getitem[range] 81.0110μs 59.2528μs 16.8768 KOps/s 16.8478 KOps/s $\color{#35bf28}+0.17\%$
test_getitem[tuple] 47.1780μs 19.2171μs 52.0369 KOps/s 52.9815 KOps/s $\color{#d91a1a}-1.78\%$
test_getitem[list] 0.1071ms 41.1842μs 24.2812 KOps/s 24.2935 KOps/s $\color{#d91a1a}-0.05\%$
test_setitem_dim[int] 79.4280μs 35.5773μs 28.1078 KOps/s 29.4842 KOps/s $\color{#d91a1a}-4.67\%$
test_setitem_dim[slice_int] 0.1303ms 62.5643μs 15.9835 KOps/s 16.7790 KOps/s $\color{#d91a1a}-4.74\%$
test_setitem_dim[range] 0.1420ms 84.5494μs 11.8274 KOps/s 11.8690 KOps/s $\color{#d91a1a}-0.35\%$
test_setitem_dim[tuple] 0.1224ms 51.4592μs 19.4329 KOps/s 20.8153 KOps/s $\textbf{\color{#d91a1a}-6.64\%}$
test_setitem 54.6320μs 20.9772μs 47.6709 KOps/s 51.2153 KOps/s $\textbf{\color{#d91a1a}-6.92\%}$
test_set 70.0010μs 20.4927μs 48.7979 KOps/s 52.6164 KOps/s $\textbf{\color{#d91a1a}-7.26\%}$
test_set_shared 1.6824ms 0.1401ms 7.1354 KOps/s 7.0059 KOps/s $\color{#35bf28}+1.85\%$
test_update 83.4060μs 22.8664μs 43.7323 KOps/s 48.5684 KOps/s $\textbf{\color{#d91a1a}-9.96\%}$
test_update_nested 77.3850μs 31.1519μs 32.1007 KOps/s 34.4629 KOps/s $\textbf{\color{#d91a1a}-6.85\%}$
test_update__nested 96.2900μs 25.4645μs 39.2703 KOps/s 39.9596 KOps/s $\color{#d91a1a}-1.72\%$
test_set_nested 55.9750μs 22.3266μs 44.7897 KOps/s 48.4252 KOps/s $\textbf{\color{#d91a1a}-7.51\%}$
test_set_nested_new 85.7100μs 26.6959μs 37.4589 KOps/s 40.2996 KOps/s $\textbf{\color{#d91a1a}-7.05\%}$
test_select 0.1044ms 41.5677μs 24.0572 KOps/s 25.2364 KOps/s $\color{#d91a1a}-4.67\%$
test_select_nested 0.1318ms 60.0514μs 16.6524 KOps/s 16.2503 KOps/s $\color{#35bf28}+2.47\%$
test_exclude_nested 0.2388ms 0.1199ms 8.3407 KOps/s 8.1929 KOps/s $\color{#35bf28}+1.80\%$
test_empty[True] 0.6024ms 0.3938ms 2.5397 KOps/s 2.4650 KOps/s $\color{#35bf28}+3.03\%$
test_empty[False] 8.7444μs 1.0904μs 917.1298 KOps/s 929.1446 KOps/s $\color{#d91a1a}-1.29\%$
test_unbind_speed 1.5314ms 0.2620ms 3.8163 KOps/s 3.8380 KOps/s $\color{#d91a1a}-0.57\%$
test_unbind_speed_stack0 0.5704ms 0.2555ms 3.9133 KOps/s 3.9174 KOps/s $\color{#d91a1a}-0.10\%$
test_unbind_speed_stack1 61.7180ms 0.7263ms 1.3768 KOps/s 1.3225 KOps/s $\color{#35bf28}+4.10\%$
test_split 60.2834ms 1.6180ms 618.0402 Ops/s 622.1635 Ops/s $\color{#d91a1a}-0.66\%$
test_chunk 59.4299ms 1.6190ms 617.6554 Ops/s 623.4096 Ops/s $\color{#d91a1a}-0.92\%$
test_creation[device0] 0.1865ms 0.1063ms 9.4107 KOps/s 9.3399 KOps/s $\color{#35bf28}+0.76\%$
test_creation_from_tensor 3.3740ms 84.2424μs 11.8705 KOps/s 11.9970 KOps/s $\color{#d91a1a}-1.05\%$
test_add_one[memmap_tensor0] 88.7260μs 5.6075μs 178.3312 KOps/s 183.0417 KOps/s $\color{#d91a1a}-2.57\%$
test_contiguous[memmap_tensor0] 20.4380μs 0.6600μs 1.5152 MOps/s 1.5865 MOps/s $\color{#d91a1a}-4.50\%$
test_stack[memmap_tensor0] 26.6900μs 3.4790μs 287.4375 KOps/s 271.6093 KOps/s $\textbf{\color{#35bf28}+5.83\%}$
test_memmaptd_index 1.0307ms 0.2411ms 4.1483 KOps/s 4.0951 KOps/s $\color{#35bf28}+1.30\%$
test_memmaptd_index_astensor 0.6290ms 0.3202ms 3.1229 KOps/s 3.1400 KOps/s $\color{#d91a1a}-0.54\%$
test_memmaptd_index_op 1.1646ms 0.6289ms 1.5902 KOps/s 1.6755 KOps/s $\textbf{\color{#d91a1a}-5.09\%}$
test_serialize_model 0.1060s 0.1009s 9.9071 Ops/s 9.2497 Ops/s $\textbf{\color{#35bf28}+7.11\%}$
test_serialize_model_pickle 0.4481s 0.3806s 2.6276 Ops/s 2.6326 Ops/s $\color{#d91a1a}-0.19\%$
test_serialize_weights 0.1625s 0.1058s 9.4489 Ops/s 9.4955 Ops/s $\color{#d91a1a}-0.49\%$
test_serialize_weights_returnearly 0.1850s 0.1295s 7.7193 Ops/s 8.2433 Ops/s $\textbf{\color{#d91a1a}-6.36\%}$
test_serialize_weights_pickle 0.6601s 0.4845s 2.0640 Ops/s 1.5498 Ops/s $\textbf{\color{#35bf28}+33.18\%}$
test_serialize_weights_filesystem 0.1512s 96.7805ms 10.3327 Ops/s 11.1592 Ops/s $\textbf{\color{#d91a1a}-7.41\%}$
test_serialize_model_filesystem 0.1032s 93.4887ms 10.6965 Ops/s 10.3153 Ops/s $\color{#35bf28}+3.70\%$
test_reshape_pytree 69.0790μs 25.4155μs 39.3461 KOps/s 37.7800 KOps/s $\color{#35bf28}+4.15\%$
test_reshape_td 85.7000μs 32.8513μs 30.4402 KOps/s 29.5824 KOps/s $\color{#35bf28}+2.90\%$
test_view_pytree 58.7600μs 25.3945μs 39.3787 KOps/s 38.4618 KOps/s $\color{#35bf28}+2.38\%$
test_view_td 95.3680μs 36.7577μs 27.2052 KOps/s 27.0719 KOps/s $\color{#35bf28}+0.49\%$
test_unbind_pytree 85.8310μs 29.1712μs 34.2804 KOps/s 33.2607 KOps/s $\color{#35bf28}+3.07\%$
test_unbind_td 0.4123ms 37.6861μs 26.5350 KOps/s 26.3933 KOps/s $\color{#35bf28}+0.54\%$
test_split_pytree 83.3590μs 29.4283μs 33.9809 KOps/s 32.7810 KOps/s $\color{#35bf28}+3.66\%$
test_split_td 0.4586ms 40.7545μs 24.5372 KOps/s 24.1629 KOps/s $\color{#35bf28}+1.55\%$
test_add_pytree 84.4080μs 35.3423μs 28.2947 KOps/s 28.4625 KOps/s $\color{#d91a1a}-0.59\%$
test_add_td 0.1370ms 55.9402μs 17.8762 KOps/s 17.3867 KOps/s $\color{#35bf28}+2.82\%$
test_distributed 0.1776ms 98.4482μs 10.1576 KOps/s 9.9358 KOps/s $\color{#35bf28}+2.23\%$
test_tdmodule 30.1070μs 17.5171μs 57.0870 KOps/s 59.2456 KOps/s $\color{#d91a1a}-3.64\%$
test_tdmodule_dispatch 74.3800μs 35.3379μs 28.2982 KOps/s 29.6767 KOps/s $\color{#d91a1a}-4.65\%$
test_tdseq 44.3730μs 20.3500μs 49.1400 KOps/s 51.7029 KOps/s $\color{#d91a1a}-4.96\%$
test_tdseq_dispatch 59.1810μs 39.8167μs 25.1151 KOps/s 26.4121 KOps/s $\color{#d91a1a}-4.91\%$
test_instantiation_functorch 1.4002ms 1.3072ms 764.9950 Ops/s 745.1221 Ops/s $\color{#35bf28}+2.67\%$
test_instantiation_td 1.5567ms 1.0115ms 988.6018 Ops/s 973.3089 Ops/s $\color{#35bf28}+1.57\%$
test_exec_functorch 0.2136ms 0.1616ms 6.1877 KOps/s 6.0606 KOps/s $\color{#35bf28}+2.10\%$
test_exec_functional_call 0.3136ms 0.1493ms 6.6993 KOps/s 6.5562 KOps/s $\color{#35bf28}+2.18\%$
test_exec_td 0.2180ms 0.1443ms 6.9287 KOps/s 6.7758 KOps/s $\color{#35bf28}+2.26\%$
test_exec_td_decorator 0.7123ms 0.2198ms 4.5499 KOps/s 4.5172 KOps/s $\color{#35bf28}+0.72\%$
test_vmap_mlp_speed[True-True] 0.6028ms 0.4772ms 2.0954 KOps/s 2.0689 KOps/s $\color{#35bf28}+1.28\%$
test_vmap_mlp_speed[True-False] 0.7432ms 0.4760ms 2.1009 KOps/s 2.0871 KOps/s $\color{#35bf28}+0.66\%$
test_vmap_mlp_speed[False-True] 0.5766ms 0.3881ms 2.5766 KOps/s 2.5287 KOps/s $\color{#35bf28}+1.89\%$
test_vmap_mlp_speed[False-False] 0.6336ms 0.4047ms 2.4709 KOps/s 2.5263 KOps/s $\color{#d91a1a}-2.19\%$
test_vmap_mlp_speed_decorator[True-True] 0.9931ms 0.5498ms 1.8189 KOps/s 1.8059 KOps/s $\color{#35bf28}+0.72\%$
test_vmap_mlp_speed_decorator[True-False] 0.8312ms 0.5514ms 1.8136 KOps/s 1.8145 KOps/s $\color{#d91a1a}-0.05\%$
test_vmap_mlp_speed_decorator[False-True] 0.6813ms 0.4524ms 2.2103 KOps/s 2.1844 KOps/s $\color{#35bf28}+1.19\%$
test_vmap_mlp_speed_decorator[False-False] 1.6092ms 0.4515ms 2.2148 KOps/s 2.1862 KOps/s $\color{#35bf28}+1.31\%$
test_to_module_speed[True] 2.5286ms 1.6691ms 599.1289 Ops/s 575.1770 Ops/s $\color{#35bf28}+4.16\%$
test_to_module_speed[False] 1.7692ms 1.6399ms 609.7764 Ops/s 588.3608 Ops/s $\color{#35bf28}+3.64\%$

@vmoens vmoens merged commit 4ad00db into main Apr 29, 2024
33 of 38 checks passed
@vmoens vmoens deleted the allow-any-torch branch April 29, 2024 07:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. versioning
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants