Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CI] nightly test #420

Merged
merged 5 commits into from
Jun 14, 2023
Merged

[CI] nightly test #420

merged 5 commits into from
Jun 14, 2023

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Jun 14, 2023

No description provided.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jun 14, 2023
@vmoens vmoens changed the title [NOMERG] nightly test [CI] nightly test Jun 14, 2023
@vmoens vmoens merged commit 112adfa into main Jun 14, 2023
@vmoens vmoens deleted the nightly branch June 14, 2023 12:39
@github-actions
Copy link

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 47. Improved: $\large\color{#35bf28}1$. Worsened: $\large\color{#d91a1a}16$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_common_ops 1.2895ms 1.2436ms 804.1233 Ops/s 872.0629 Ops/s $\textbf{\color{#d91a1a}-7.79\%}$
test_creation 5.6672μs 5.0067μs 199.7338 KOps/s 210.5497 KOps/s $\textbf{\color{#d91a1a}-5.14\%}$
test_creation_empty 14.5656μs 13.0472μs 76.6445 KOps/s 79.2922 KOps/s $\color{#d91a1a}-3.34\%$
test_creation_nested_1 26.1901μs 24.2996μs 41.1529 KOps/s 42.9515 KOps/s $\color{#d91a1a}-4.19\%$
test_creation_nested_2 26.8561μs 25.0088μs 39.9859 KOps/s 42.5823 KOps/s $\textbf{\color{#d91a1a}-6.10\%}$
test_clone 33.3054μs 30.8283μs 32.4377 KOps/s 34.0948 KOps/s $\color{#d91a1a}-4.86\%$
test_getitem[int] 40.8723μs 38.0437μs 26.2855 KOps/s 28.7894 KOps/s $\textbf{\color{#d91a1a}-8.70\%}$
test_getitem[slice_int] 91.7420μs 78.6956μs 12.7072 KOps/s 12.5524 KOps/s $\color{#35bf28}+1.23\%$
test_getitem[range] 84.3388μs 76.5904μs 13.0565 KOps/s 12.9890 KOps/s $\color{#35bf28}+0.52\%$
test_getitem[tuple] 79.1741μs 73.9781μs 13.5175 KOps/s 12.9816 KOps/s $\color{#35bf28}+4.13\%$
test_getitem[list] 81.6741μs 69.7841μs 14.3299 KOps/s 14.8802 KOps/s $\color{#d91a1a}-3.70\%$
test_setitem_dim[int] 0.1084ms 54.9025μs 18.2141 KOps/s 20.3558 KOps/s $\textbf{\color{#d91a1a}-10.52\%}$
test_setitem_dim[slice_int] 0.2212ms 97.7572μs 10.2294 KOps/s 10.5400 KOps/s $\color{#d91a1a}-2.95\%$
test_setitem_dim[range] 0.1700ms 89.2235μs 11.2078 KOps/s 11.3748 KOps/s $\color{#d91a1a}-1.47\%$
test_setitem_dim[tuple] 0.1408ms 87.3274μs 11.4512 KOps/s 11.3594 KOps/s $\color{#35bf28}+0.81\%$
test_setitem 40.7016μs 35.9598μs 27.8089 KOps/s 27.7042 KOps/s $\color{#35bf28}+0.38\%$
test_set 40.4946μs 36.1695μs 27.6476 KOps/s 28.1497 KOps/s $\color{#d91a1a}-1.78\%$
test_set_shared 0.1928ms 0.1804ms 5.5428 KOps/s 5.6039 KOps/s $\color{#d91a1a}-1.09\%$
test_update 43.3287μs 38.9800μs 25.6542 KOps/s 24.7132 KOps/s $\color{#35bf28}+3.81\%$
test_update_nested 67.1336μs 62.1215μs 16.0975 KOps/s 16.5100 KOps/s $\color{#d91a1a}-2.50\%$
test_set_nested 53.6021μs 50.4340μs 19.8279 KOps/s 20.9956 KOps/s $\textbf{\color{#d91a1a}-5.56\%}$
test_set_nested_new 76.5790μs 70.4744μs 14.1896 KOps/s 14.9114 KOps/s $\color{#d91a1a}-4.84\%$
test_select 0.1208ms 0.1158ms 8.6335 KOps/s 9.1550 KOps/s $\textbf{\color{#d91a1a}-5.70\%}$
test_creation[device0] 1.3768ms 0.5864ms 1.7052 KOps/s 1.5832 KOps/s $\textbf{\color{#35bf28}+7.71\%}$
test_creation_from_tensor 0.6942ms 0.5578ms 1.7928 KOps/s 1.8201 KOps/s $\color{#d91a1a}-1.50\%$
test_add_one[memmap_tensor0] 53.9321μs 36.2884μs 27.5570 KOps/s 30.6468 KOps/s $\textbf{\color{#d91a1a}-10.08\%}$
test_contiguous[memmap_tensor0] 10.6364μs 9.9061μs 100.9481 KOps/s 112.0303 KOps/s $\textbf{\color{#d91a1a}-9.89\%}$
test_stack[memmap_tensor0] 0.2107ms 51.3020μs 19.4924 KOps/s 20.9723 KOps/s $\textbf{\color{#d91a1a}-7.06\%}$
test_reshape_pytree 45.6018μs 42.3201μs 23.6295 KOps/s 26.0943 KOps/s $\textbf{\color{#d91a1a}-9.45\%}$
test_reshape_td 60.9613μs 58.8129μs 17.0031 KOps/s 18.4303 KOps/s $\textbf{\color{#d91a1a}-7.74\%}$
test_view_pytree 41.9236μs 39.7562μs 25.1533 KOps/s 26.4281 KOps/s $\color{#d91a1a}-4.82\%$
test_view_td 12.1645μs 10.8855μs 91.8650 KOps/s 96.1516 KOps/s $\color{#d91a1a}-4.46\%$
test_unbind_pytree 45.0957μs 42.5172μs 23.5199 KOps/s 24.4639 KOps/s $\color{#d91a1a}-3.86\%$
test_unbind_td 0.1905ms 0.1811ms 5.5216 KOps/s 5.9257 KOps/s $\textbf{\color{#d91a1a}-6.82\%}$
test_split_pytree 52.2220μs 48.6419μs 20.5584 KOps/s 21.5280 KOps/s $\color{#d91a1a}-4.50\%$
test_split_td 0.1415ms 0.1355ms 7.3808 KOps/s 7.6314 KOps/s $\color{#d91a1a}-3.28\%$
test_add_pytree 53.6821μs 50.7844μs 19.6911 KOps/s 19.8986 KOps/s $\color{#d91a1a}-1.04\%$
test_add_td 77.0400μs 69.9824μs 14.2893 KOps/s 14.0241 KOps/s $\color{#35bf28}+1.89\%$
test_distributed 74.9030μs 74.9030μs 13.3506 KOps/s 13.6979 KOps/s $\color{#d91a1a}-2.54\%$
test_tdmodule 72.3030μs 29.4010μs 34.0125 KOps/s 35.1659 KOps/s $\color{#d91a1a}-3.28\%$
test_tdmodule_dispatch 55.5801ms 69.3429μs 14.4211 KOps/s 16.0555 KOps/s $\textbf{\color{#d91a1a}-10.18\%}$
test_tdseq 0.2507ms 41.5470μs 24.0691 KOps/s 24.2869 KOps/s $\color{#d91a1a}-0.90\%$
test_tdseq_dispatch 0.1402ms 72.1719μs 13.8558 KOps/s 13.8404 KOps/s $\color{#35bf28}+0.11\%$
test_instantiation_functorch 1.9418ms 1.8420ms 542.8867 Ops/s 569.1155 Ops/s $\color{#d91a1a}-4.61\%$
test_instantiation_td 8.4306ms 1.4713ms 679.6781 Ops/s 740.0592 Ops/s $\textbf{\color{#d91a1a}-8.16\%}$
test_exec_functorch 0.2296ms 0.2099ms 4.7639 KOps/s 4.8989 KOps/s $\color{#d91a1a}-2.75\%$
test_exec_td 0.2769ms 0.2611ms 3.8296 KOps/s 4.0804 KOps/s $\textbf{\color{#d91a1a}-6.15\%}$

@github-actions
Copy link

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 47. Improved: $\large\color{#35bf28}2$. Worsened: $\large\color{#d91a1a}1$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_common_ops 1.0886ms 1.0610ms 942.5080 Ops/s 921.7321 Ops/s $\color{#35bf28}+2.25\%$
test_creation 4.3543μs 4.0083μs 249.4828 KOps/s 264.3155 KOps/s $\textbf{\color{#d91a1a}-5.61\%}$
test_creation_empty 11.6678μs 10.9753μs 91.1135 KOps/s 92.3628 KOps/s $\color{#d91a1a}-1.35\%$
test_creation_nested_1 22.6625μs 19.6896μs 50.7882 KOps/s 50.8494 KOps/s $\color{#d91a1a}-0.12\%$
test_creation_nested_2 21.9214μs 20.8327μs 48.0014 KOps/s 48.3659 KOps/s $\color{#d91a1a}-0.75\%$
test_clone 26.8938μs 25.9367μs 38.5555 KOps/s 40.3740 KOps/s $\color{#d91a1a}-4.50\%$
test_getitem[int] 31.8539μs 30.2314μs 33.0782 KOps/s 33.2619 KOps/s $\color{#d91a1a}-0.55\%$
test_getitem[slice_int] 68.6498μs 64.1669μs 15.5844 KOps/s 15.9113 KOps/s $\color{#d91a1a}-2.05\%$
test_getitem[range] 72.8659μs 68.0752μs 14.6896 KOps/s 14.6147 KOps/s $\color{#35bf28}+0.51\%$
test_getitem[tuple] 61.2667μs 59.3722μs 16.8429 KOps/s 17.0262 KOps/s $\color{#d91a1a}-1.08\%$
test_getitem[list] 65.3246μs 61.4111μs 16.2837 KOps/s 16.4563 KOps/s $\color{#d91a1a}-1.05\%$
test_setitem_dim[int] 82.0050μs 46.5624μs 21.4766 KOps/s 21.8234 KOps/s $\color{#d91a1a}-1.59\%$
test_setitem_dim[slice_int] 0.1415ms 84.7230μs 11.8032 KOps/s 12.1256 KOps/s $\color{#d91a1a}-2.66\%$
test_setitem_dim[range] 0.1974ms 84.0227μs 11.9015 KOps/s 12.2548 KOps/s $\color{#d91a1a}-2.88\%$
test_setitem_dim[tuple] 0.1175ms 76.5552μs 13.0625 KOps/s 13.4469 KOps/s $\color{#d91a1a}-2.86\%$
test_setitem 33.0941μs 31.0053μs 32.2526 KOps/s 32.8783 KOps/s $\color{#d91a1a}-1.90\%$
test_set 32.3560μs 30.7689μs 32.5003 KOps/s 33.1101 KOps/s $\color{#d91a1a}-1.84\%$
test_set_shared 0.1842ms 0.1795ms 5.5724 KOps/s 5.5860 KOps/s $\color{#d91a1a}-0.24\%$
test_update 36.1153μs 33.2089μs 30.1124 KOps/s 30.7304 KOps/s $\color{#d91a1a}-2.01\%$
test_update_nested 53.2424μs 50.2553μs 19.8984 KOps/s 20.2268 KOps/s $\color{#d91a1a}-1.62\%$
test_set_nested 41.9956μs 39.8756μs 25.0780 KOps/s 25.4784 KOps/s $\color{#d91a1a}-1.57\%$
test_set_nested_new 57.0926μs 55.2013μs 18.1155 KOps/s 18.2004 KOps/s $\color{#d91a1a}-0.47\%$
test_select 99.4493μs 90.6888μs 11.0267 KOps/s 11.1152 KOps/s $\color{#d91a1a}-0.80\%$
test_creation[device0] 1.3823ms 0.5777ms 1.7310 KOps/s 1.7268 KOps/s $\color{#35bf28}+0.25\%$
test_creation_from_tensor 0.6799ms 0.5503ms 1.8173 KOps/s 1.8063 KOps/s $\color{#35bf28}+0.61\%$
test_add_one[memmap_tensor0] 37.5064μs 33.4089μs 29.9322 KOps/s 28.9386 KOps/s $\color{#35bf28}+3.43\%$
test_contiguous[memmap_tensor0] 9.7286μs 9.0337μs 110.6963 KOps/s 106.5893 KOps/s $\color{#35bf28}+3.85\%$
test_stack[memmap_tensor0] 0.2282ms 52.2832μs 19.1266 KOps/s 16.3842 KOps/s $\textbf{\color{#35bf28}+16.74\%}$
test_reshape_pytree 35.9513μs 32.9553μs 30.3441 KOps/s 30.2510 KOps/s $\color{#35bf28}+0.31\%$
test_reshape_td 50.8632μs 47.1668μs 21.2014 KOps/s 21.4441 KOps/s $\color{#d91a1a}-1.13\%$
test_view_pytree 31.9790μs 30.2690μs 33.0371 KOps/s 33.0983 KOps/s $\color{#d91a1a}-0.18\%$
test_view_td 9.3176μs 8.1553μs 122.6193 KOps/s 126.9971 KOps/s $\color{#d91a1a}-3.45\%$
test_unbind_pytree 36.5873μs 34.9551μs 28.6081 KOps/s 29.0295 KOps/s $\color{#d91a1a}-1.45\%$
test_unbind_td 0.1454ms 0.1424ms 7.0211 KOps/s 7.0982 KOps/s $\color{#d91a1a}-1.09\%$
test_split_pytree 41.9486μs 39.6437μs 25.2247 KOps/s 25.3821 KOps/s $\color{#d91a1a}-0.62\%$
test_split_td 0.1141ms 0.1106ms 9.0453 KOps/s 9.0792 KOps/s $\color{#d91a1a}-0.37\%$
test_add_pytree 45.1668μs 43.2338μs 23.1300 KOps/s 23.4825 KOps/s $\color{#d91a1a}-1.50\%$
test_add_td 63.6029μs 59.9479μs 16.6812 KOps/s 16.9297 KOps/s $\color{#d91a1a}-1.47\%$
test_distributed 86.5050μs 86.5050μs 11.5600 KOps/s 10.5034 KOps/s $\textbf{\color{#35bf28}+10.06\%}$
test_tdmodule 0.1276ms 25.6799μs 38.9409 KOps/s 39.9978 KOps/s $\color{#d91a1a}-2.64\%$
test_tdmodule_dispatch 0.2570ms 56.3618μs 17.7425 KOps/s 18.3139 KOps/s $\color{#d91a1a}-3.12\%$
test_tdseq 0.1138ms 36.0118μs 27.7687 KOps/s 29.0831 KOps/s $\color{#d91a1a}-4.52\%$
test_tdseq_dispatch 0.1485ms 67.5965μs 14.7937 KOps/s 15.2672 KOps/s $\color{#d91a1a}-3.10\%$
test_instantiation_functorch 1.6103ms 1.5275ms 654.6570 Ops/s 673.6577 Ops/s $\color{#d91a1a}-2.82\%$
test_instantiation_td 1.2463ms 1.1810ms 846.7223 Ops/s 863.8417 Ops/s $\color{#d91a1a}-1.98\%$
test_exec_functorch 0.2219ms 0.1827ms 5.4744 KOps/s 5.4441 KOps/s $\color{#35bf28}+0.56\%$
test_exec_td 0.2304ms 0.2252ms 4.4396 KOps/s 4.4359 KOps/s $\color{#35bf28}+0.08\%$

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants