Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CI] Remove OSX x86 jobs #753

Merged
merged 8 commits into from
Apr 25, 2024
Merged

[CI] Remove OSX x86 jobs #753

merged 8 commits into from
Apr 25, 2024

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Apr 25, 2024

No description provided.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Apr 25, 2024
@vmoens vmoens added the CI label Apr 25, 2024
Copy link

github-actions bot commented Apr 25, 2024

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 127. Improved: $\large\color{#35bf28}24$. Worsened: $\large\color{#d91a1a}9$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_plain_set_nested 31.4490μs 16.2230μs 61.6407 KOps/s 55.8085 KOps/s $\textbf{\color{#35bf28}+10.45\%}$
test_plain_set_stack_nested 42.8900μs 16.6981μs 59.8871 KOps/s 56.8236 KOps/s $\textbf{\color{#35bf28}+5.39\%}$
test_plain_set_nested_inplace 0.1883ms 19.0423μs 52.5146 KOps/s 50.1712 KOps/s $\color{#35bf28}+4.67\%$
test_plain_set_stack_nested_inplace 44.1030μs 18.8933μs 52.9288 KOps/s 49.9078 KOps/s $\textbf{\color{#35bf28}+6.05\%}$
test_items 19.3360μs 2.7010μs 370.2310 KOps/s 395.9386 KOps/s $\textbf{\color{#d91a1a}-6.49\%}$
test_items_nested 1.0305ms 0.2726ms 3.6684 KOps/s 3.5932 KOps/s $\color{#35bf28}+2.09\%$
test_items_nested_locked 0.4085ms 0.2662ms 3.7566 KOps/s 3.7108 KOps/s $\color{#35bf28}+1.23\%$
test_items_nested_leaf 0.1412ms 77.0167μs 12.9842 KOps/s 13.0328 KOps/s $\color{#d91a1a}-0.37\%$
test_items_stack_nested 0.5005ms 0.2701ms 3.7023 KOps/s 3.7076 KOps/s $\color{#d91a1a}-0.14\%$
test_items_stack_nested_leaf 0.2621ms 80.9398μs 12.3549 KOps/s 12.7609 KOps/s $\color{#d91a1a}-3.18\%$
test_items_stack_nested_locked 0.4745ms 0.2681ms 3.7306 KOps/s 3.7410 KOps/s $\color{#d91a1a}-0.28\%$
test_keys 18.0640μs 4.1688μs 239.8776 KOps/s 257.0233 KOps/s $\textbf{\color{#d91a1a}-6.67\%}$
test_keys_nested 0.2468ms 0.1389ms 7.1972 KOps/s 7.2711 KOps/s $\color{#d91a1a}-1.02\%$
test_keys_nested_locked 1.9599ms 0.1452ms 6.8850 KOps/s 7.0395 KOps/s $\color{#d91a1a}-2.20\%$
test_keys_nested_leaf 0.2083ms 0.1178ms 8.4868 KOps/s 8.5992 KOps/s $\color{#d91a1a}-1.31\%$
test_keys_stack_nested 0.2537ms 0.1414ms 7.0714 KOps/s 7.2355 KOps/s $\color{#d91a1a}-2.27\%$
test_keys_stack_nested_leaf 0.2525ms 0.1197ms 8.3566 KOps/s 8.6036 KOps/s $\color{#d91a1a}-2.87\%$
test_keys_stack_nested_locked 0.2389ms 0.1447ms 6.9105 KOps/s 7.0396 KOps/s $\color{#d91a1a}-1.83\%$
test_values 8.3582μs 1.1965μs 835.7487 KOps/s 869.4705 KOps/s $\color{#d91a1a}-3.88\%$
test_values_nested 96.1790μs 50.6276μs 19.7521 KOps/s 19.7481 KOps/s $\color{#35bf28}+0.02\%$
test_values_nested_locked 0.1166ms 50.8536μs 19.6643 KOps/s 19.7643 KOps/s $\color{#d91a1a}-0.51\%$
test_values_nested_leaf 86.3910μs 46.2391μs 21.6267 KOps/s 21.9527 KOps/s $\color{#d91a1a}-1.48\%$
test_values_stack_nested 0.1078ms 51.7038μs 19.3409 KOps/s 19.2697 KOps/s $\color{#35bf28}+0.37\%$
test_values_stack_nested_leaf 88.8860μs 46.3118μs 21.5928 KOps/s 21.8339 KOps/s $\color{#d91a1a}-1.10\%$
test_values_stack_nested_locked 0.1060ms 51.5075μs 19.4146 KOps/s 19.6003 KOps/s $\color{#d91a1a}-0.95\%$
test_membership 20.0580μs 1.3879μs 720.5228 KOps/s 734.9348 KOps/s $\color{#d91a1a}-1.96\%$
test_membership_nested 31.2090μs 3.4298μs 291.5642 KOps/s 293.8443 KOps/s $\color{#d91a1a}-0.78\%$
test_membership_nested_leaf 29.6550μs 3.4401μs 290.6891 KOps/s 294.6846 KOps/s $\color{#d91a1a}-1.36\%$
test_membership_stacked_nested 21.2900μs 3.4277μs 291.7394 KOps/s 296.5821 KOps/s $\color{#d91a1a}-1.63\%$
test_membership_stacked_nested_leaf 18.6450μs 3.4282μs 291.7017 KOps/s 283.3794 KOps/s $\color{#35bf28}+2.94\%$
test_membership_nested_last 23.1130μs 4.2182μs 237.0657 KOps/s 232.2551 KOps/s $\color{#35bf28}+2.07\%$
test_membership_nested_leaf_last 23.9550μs 4.2692μs 234.2371 KOps/s 234.6544 KOps/s $\color{#d91a1a}-0.18\%$
test_membership_stacked_nested_last 26.4390μs 4.8076μs 208.0055 KOps/s 238.7854 KOps/s $\textbf{\color{#d91a1a}-12.89\%}$
test_membership_stacked_nested_leaf_last 26.4590μs 4.8226μs 207.3555 KOps/s 236.2462 KOps/s $\textbf{\color{#d91a1a}-12.23\%}$
test_nested_getleaf 26.9810μs 10.6696μs 93.7242 KOps/s 93.4754 KOps/s $\color{#35bf28}+0.27\%$
test_nested_get 36.4390μs 10.2063μs 97.9783 KOps/s 97.8202 KOps/s $\color{#35bf28}+0.16\%$
test_stacked_getleaf 35.0360μs 10.5579μs 94.7154 KOps/s 95.1931 KOps/s $\color{#d91a1a}-0.50\%$
test_stacked_get 37.6900μs 10.1569μs 98.4557 KOps/s 100.0609 KOps/s $\color{#d91a1a}-1.60\%$
test_nested_getitemleaf 60.7710μs 11.2618μs 88.7960 KOps/s 91.1469 KOps/s $\color{#d91a1a}-2.58\%$
test_nested_getitem 36.2880μs 10.5201μs 95.0563 KOps/s 95.2267 KOps/s $\color{#d91a1a}-0.18\%$
test_stacked_getitemleaf 34.4250μs 11.2698μs 88.7324 KOps/s 90.5481 KOps/s $\color{#d91a1a}-2.01\%$
test_stacked_getitem 31.7990μs 10.3543μs 96.5783 KOps/s 99.9651 KOps/s $\color{#d91a1a}-3.39\%$
test_lock_nested 50.3472ms 0.3954ms 2.5290 KOps/s 2.8760 KOps/s $\textbf{\color{#d91a1a}-12.07\%}$
test_lock_stack_nested 0.4047ms 0.3063ms 3.2643 KOps/s 3.2020 KOps/s $\color{#35bf28}+1.95\%$
test_unlock_nested 0.6878ms 0.3458ms 2.8920 KOps/s 2.5187 KOps/s $\textbf{\color{#35bf28}+14.82\%}$
test_unlock_stack_nested 0.6120ms 0.3154ms 3.1709 KOps/s 3.1390 KOps/s $\color{#35bf28}+1.02\%$
test_flatten_speed 0.2059ms 94.3824μs 10.5952 KOps/s 10.4698 KOps/s $\color{#35bf28}+1.20\%$
test_unflatten_speed 0.6390ms 0.4152ms 2.4085 KOps/s 2.4481 KOps/s $\color{#d91a1a}-1.62\%$
test_common_ops 4.5992ms 0.6980ms 1.4327 KOps/s 1.3469 KOps/s $\textbf{\color{#35bf28}+6.37\%}$
test_creation 15.7700μs 1.9349μs 516.8105 KOps/s 496.1066 KOps/s $\color{#35bf28}+4.17\%$
test_creation_empty 25.5270μs 9.7501μs 102.5626 KOps/s 83.1135 KOps/s $\textbf{\color{#35bf28}+23.40\%}$
test_creation_nested_1 36.9290μs 12.3406μs 81.0335 KOps/s 67.9197 KOps/s $\textbf{\color{#35bf28}+19.31\%}$
test_creation_nested_2 47.1480μs 15.8080μs 63.2590 KOps/s 54.7453 KOps/s $\textbf{\color{#35bf28}+15.55\%}$
test_clone 0.1019ms 13.5820μs 73.6269 KOps/s 74.9540 KOps/s $\color{#d91a1a}-1.77\%$
test_getitem[int] 33.5020μs 11.6281μs 85.9986 KOps/s 86.2049 KOps/s $\color{#d91a1a}-0.24\%$
test_getitem[slice_int] 90.8890μs 23.2541μs 43.0031 KOps/s 44.8591 KOps/s $\color{#d91a1a}-4.14\%$
test_getitem[range] 78.9070μs 62.0156μs 16.1250 KOps/s 17.2854 KOps/s $\textbf{\color{#d91a1a}-6.71\%}$
test_getitem[tuple] 49.7130μs 18.6290μs 53.6798 KOps/s 52.7767 KOps/s $\color{#35bf28}+1.71\%$
test_getitem[list] 0.1129ms 40.6981μs 24.5711 KOps/s 25.2848 KOps/s $\color{#d91a1a}-2.82\%$
test_setitem_dim[int] 56.8860μs 32.2679μs 30.9905 KOps/s 28.0502 KOps/s $\textbf{\color{#35bf28}+10.48\%}$
test_setitem_dim[slice_int] 0.1092ms 59.4536μs 16.8198 KOps/s 15.7933 KOps/s $\textbf{\color{#35bf28}+6.50\%}$
test_setitem_dim[range] 0.1525ms 82.3424μs 12.1444 KOps/s 11.5792 KOps/s $\color{#35bf28}+4.88\%$
test_setitem_dim[tuple] 81.5220μs 48.6748μs 20.5445 KOps/s 19.2688 KOps/s $\textbf{\color{#35bf28}+6.62\%}$
test_setitem 61.7750μs 20.0036μs 49.9910 KOps/s 48.3043 KOps/s $\color{#35bf28}+3.49\%$
test_set 64.7910μs 19.5314μs 51.1996 KOps/s 45.3023 KOps/s $\textbf{\color{#35bf28}+13.02\%}$
test_set_shared 1.5535ms 0.1409ms 7.0971 KOps/s 6.8669 KOps/s $\color{#35bf28}+3.35\%$
test_update 0.1314ms 21.0490μs 47.5083 KOps/s 43.2046 KOps/s $\textbf{\color{#35bf28}+9.96\%}$
test_update_nested 74.4790μs 29.4180μs 33.9928 KOps/s 31.8674 KOps/s $\textbf{\color{#35bf28}+6.67\%}$
test_update__nested 81.8830μs 25.6780μs 38.9438 KOps/s 40.5631 KOps/s $\color{#d91a1a}-3.99\%$
test_set_nested 96.6100μs 21.8194μs 45.8309 KOps/s 44.8697 KOps/s $\color{#35bf28}+2.14\%$
test_set_nested_new 68.9690μs 25.2956μs 39.5326 KOps/s 37.5000 KOps/s $\textbf{\color{#35bf28}+5.42\%}$
test_select 0.1064ms 40.5407μs 24.6665 KOps/s 23.9913 KOps/s $\color{#35bf28}+2.81\%$
test_select_nested 0.1322ms 62.7286μs 15.9417 KOps/s 16.2722 KOps/s $\color{#d91a1a}-2.03\%$
test_exclude_nested 0.2296ms 0.1225ms 8.1617 KOps/s 8.2420 KOps/s $\color{#d91a1a}-0.97\%$
test_empty[True] 0.8319ms 0.4030ms 2.4811 KOps/s 2.5529 KOps/s $\color{#d91a1a}-2.81\%$
test_empty[False] 14.3548μs 1.1212μs 891.8671 KOps/s 918.2017 KOps/s $\color{#d91a1a}-2.87\%$
test_unbind_speed 1.6170ms 0.2601ms 3.8441 KOps/s 3.8508 KOps/s $\color{#d91a1a}-0.17\%$
test_unbind_speed_stack0 0.4285ms 0.2585ms 3.8688 KOps/s 3.9248 KOps/s $\color{#d91a1a}-1.43\%$
test_unbind_speed_stack1 67.1890ms 0.7300ms 1.3699 KOps/s 1.2799 KOps/s $\textbf{\color{#35bf28}+7.03\%}$
test_split 66.3048ms 1.6922ms 590.9561 Ops/s 622.3454 Ops/s $\textbf{\color{#d91a1a}-5.04\%}$
test_chunk 69.7499ms 1.6155ms 618.9921 Ops/s 620.6437 Ops/s $\color{#d91a1a}-0.27\%$
test_creation[device0] 0.1928ms 0.1028ms 9.7310 KOps/s 9.6582 KOps/s $\color{#35bf28}+0.75\%$
test_creation_from_tensor 3.5713ms 82.4231μs 12.1325 KOps/s 12.0332 KOps/s $\color{#35bf28}+0.83\%$
test_add_one[memmap_tensor0] 70.8630μs 5.2002μs 192.2994 KOps/s 187.5799 KOps/s $\color{#35bf28}+2.52\%$
test_contiguous[memmap_tensor0] 13.6550μs 0.6310μs 1.5849 MOps/s 1.5255 MOps/s $\color{#35bf28}+3.89\%$
test_stack[memmap_tensor0] 19.8570μs 3.6034μs 277.5141 KOps/s 289.3206 KOps/s $\color{#d91a1a}-4.08\%$
test_memmaptd_index 0.9566ms 0.2297ms 4.3535 KOps/s 4.2774 KOps/s $\color{#35bf28}+1.78\%$
test_memmaptd_index_astensor 0.8295ms 0.3046ms 3.2827 KOps/s 3.2528 KOps/s $\color{#35bf28}+0.92\%$
test_memmaptd_index_op 1.1138ms 0.5685ms 1.7590 KOps/s 1.6388 KOps/s $\textbf{\color{#35bf28}+7.33\%}$
test_serialize_model 0.1761s 0.1105s 9.0508 Ops/s 9.0031 Ops/s $\color{#35bf28}+0.53\%$
test_serialize_model_pickle 0.4524s 0.3792s 2.6371 Ops/s 2.6399 Ops/s $\color{#d91a1a}-0.11\%$
test_serialize_weights 0.1007s 98.3954ms 10.1631 Ops/s 8.9406 Ops/s $\textbf{\color{#35bf28}+13.67\%}$
test_serialize_weights_returnearly 0.1886s 0.1320s 7.5730 Ops/s 7.2901 Ops/s $\color{#35bf28}+3.88\%$
test_serialize_weights_pickle 0.8515s 0.5686s 1.7586 Ops/s 1.5730 Ops/s $\textbf{\color{#35bf28}+11.80\%}$
test_serialize_weights_filesystem 0.1595s 97.1868ms 10.2895 Ops/s 10.8368 Ops/s $\textbf{\color{#d91a1a}-5.05\%}$
test_serialize_model_filesystem 98.9853ms 92.5028ms 10.8105 Ops/s 10.1589 Ops/s $\textbf{\color{#35bf28}+6.41\%}$
test_reshape_pytree 70.4920μs 25.4556μs 39.2841 KOps/s 39.6400 KOps/s $\color{#d91a1a}-0.90\%$
test_reshape_td 75.9520μs 33.7889μs 29.5956 KOps/s 29.9212 KOps/s $\color{#d91a1a}-1.09\%$
test_view_pytree 59.5910μs 24.9993μs 40.0012 KOps/s 39.8468 KOps/s $\color{#35bf28}+0.39\%$
test_view_td 85.0190μs 37.9213μs 26.3704 KOps/s 26.7727 KOps/s $\color{#d91a1a}-1.50\%$
test_unbind_pytree 77.7260μs 28.8246μs 34.6925 KOps/s 34.5690 KOps/s $\color{#35bf28}+0.36\%$
test_unbind_td 0.3747ms 38.0973μs 26.2486 KOps/s 26.3174 KOps/s $\color{#d91a1a}-0.26\%$
test_split_pytree 77.4450μs 29.1061μs 34.3571 KOps/s 34.6319 KOps/s $\color{#d91a1a}-0.79\%$
test_split_td 0.1231ms 41.0388μs 24.3672 KOps/s 24.6429 KOps/s $\color{#d91a1a}-1.12\%$
test_add_pytree 86.5620μs 34.5697μs 28.9270 KOps/s 29.0313 KOps/s $\color{#d91a1a}-0.36\%$
test_add_td 0.1159ms 54.6963μs 18.2828 KOps/s 17.4573 KOps/s $\color{#35bf28}+4.73\%$
test_distributed 0.1806ms 99.6839μs 10.0317 KOps/s 9.8987 KOps/s $\color{#35bf28}+1.34\%$
test_tdmodule 44.5530μs 17.3040μs 57.7901 KOps/s 52.8880 KOps/s $\textbf{\color{#35bf28}+9.27\%}$
test_tdmodule_dispatch 61.8950μs 34.0366μs 29.3802 KOps/s 26.6707 KOps/s $\textbf{\color{#35bf28}+10.16\%}$
test_tdseq 47.3590μs 20.6589μs 48.4053 KOps/s 45.3288 KOps/s $\textbf{\color{#35bf28}+6.79\%}$
test_tdseq_dispatch 65.2420μs 38.7445μs 25.8101 KOps/s 23.0873 KOps/s $\textbf{\color{#35bf28}+11.79\%}$
test_instantiation_functorch 2.0932ms 1.3442ms 743.9402 Ops/s 757.1143 Ops/s $\color{#d91a1a}-1.74\%$
test_instantiation_td 66.0909ms 1.1032ms 906.4299 Ops/s 991.6790 Ops/s $\textbf{\color{#d91a1a}-8.60\%}$
test_exec_functorch 0.2537ms 0.1601ms 6.2465 KOps/s 6.3746 KOps/s $\color{#d91a1a}-2.01\%$
test_exec_functional_call 0.2266ms 0.1483ms 6.7428 KOps/s 6.9003 KOps/s $\color{#d91a1a}-2.28\%$
test_exec_td 0.2944ms 0.1437ms 6.9566 KOps/s 7.1715 KOps/s $\color{#d91a1a}-3.00\%$
test_exec_td_decorator 0.8240ms 0.2238ms 4.4688 KOps/s 4.6393 KOps/s $\color{#d91a1a}-3.67\%$
test_vmap_mlp_speed[True-True] 0.9658ms 0.4832ms 2.0697 KOps/s 2.0434 KOps/s $\color{#35bf28}+1.29\%$
test_vmap_mlp_speed[True-False] 0.7903ms 0.4808ms 2.0797 KOps/s 2.0092 KOps/s $\color{#35bf28}+3.51\%$
test_vmap_mlp_speed[False-True] 0.5587ms 0.3929ms 2.5452 KOps/s 2.5477 KOps/s $\color{#d91a1a}-0.10\%$
test_vmap_mlp_speed[False-False] 0.6142ms 0.3939ms 2.5384 KOps/s 2.5423 KOps/s $\color{#d91a1a}-0.15\%$
test_vmap_mlp_speed_decorator[True-True] 1.1539ms 0.5493ms 1.8206 KOps/s 1.7665 KOps/s $\color{#35bf28}+3.07\%$
test_vmap_mlp_speed_decorator[True-False] 0.7306ms 0.5513ms 1.8138 KOps/s 1.7692 KOps/s $\color{#35bf28}+2.52\%$
test_vmap_mlp_speed_decorator[False-True] 0.7792ms 0.4572ms 2.1871 KOps/s 2.1788 KOps/s $\color{#35bf28}+0.38\%$
test_vmap_mlp_speed_decorator[False-False] 0.8093ms 0.4563ms 2.1914 KOps/s 2.1244 KOps/s $\color{#35bf28}+3.15\%$
test_to_module_speed[True] 1.8164ms 1.7016ms 587.6666 Ops/s 588.6558 Ops/s $\color{#d91a1a}-0.17\%$
test_to_module_speed[False] 2.3339ms 1.6897ms 591.8299 Ops/s 600.8658 Ops/s $\color{#d91a1a}-1.50\%$

@vmoens vmoens merged commit 4332210 into main Apr 25, 2024
21 of 24 checks passed
@vmoens vmoens deleted the remove-osx-x86 branch April 25, 2024 11:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CI CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants