Test performance of Enso via WSL

[wdanilo]

Summary

As Python and other "C-heavy" languages are not supported by GraalVM on Windows, we might want to run Enso via WSL. It is important to understand the performance implications of that, including file loading or any runtime performance overhead. If we come to the conclusion that this is the proper solution, then the WSL support would need to be added on Engine side, as it needs to work both in GUI as well as in batch (cmd) mode.

Specification

• Create a test to load a big data file (1 Gb) and check the loading speed on Windows native and under WSL.
• Run our benchmarks on Windows native and WSL and compare the results.

Acceptance Criteria & Test Cases

• Results of tests

[mwu-tow]

Results for WSL v1. Results for v2 will come on the next day.

WSL v1

Reading 1GB file

I've generated a 1GB file filled with ascii character.

Tried measuring Enso's reading functions, however they are too slow to provide any meaningful results. See #1724.
So I've run Python's read_text on the file and compared between WSL and native Windows.

	Time [s]	relative to Win
Win	0.963
WSL1	2.85	+196%
WSL2-ext4	0.655	-32%
WSL2-9p	3.00	+212%

(median over 10 repeated runs)

Whether the WSL1 accessed file on mounted NTFS drive or read it from its local filesystem did not make significant difference.
It should be noted that WSL2-ext4 is particularly good when file is repeatedly loaded. Cold file load seems to be comparable to Win.

Our benchmarks

Native Windows

	citations		listings
WIN	4,002	0,308	1,127	0,128	0,779	0,045		27,325
WIN	4,13	0,276	1,147	0,128	0,729	0,045		26,707
WIN	4,222	0,237	1,154	0,133	0,837	0,042		26,85
WIN	4,452	0,185	1,279	0,132	0,803	0,045		27,917
WIN	4,225	0,254	1,348	0,116	0,778	0,033		27,527
WIN	4,062	0,186	1,264	0,138	0,802	0,075		26,807
WIN	3,953	0,158	1,238	0,12	0,748	0,034		26,201
WIN	3,82	0,195	1,05	0,121	0,672	0,028		24,227
MEDIAN	4,096	0,216	1,196	0,128	0,7785	0,0435		26,8285

WSL v1

	citations		listings
	read	processing	read	processing	makes processing	final processing		command time
WSL	6,034	0,218	1,869	0,138	0,696	0,047		29,743
WSL	5,524	0,204	1,565	0,117	0,642	0,04		28,289
WSL	5,66	0,218	1,481	0,114	0,699	0,049		28,915
WSL	5,896	0,184	1,685	0,148	0,969	0,049		29,939
WSL	5,236	0,204	1,862	0,125	0,765	0,114		28,54
WSL	6,178	0,277	1,746	0,138	0,83	0,029		29,743
WSL	5,272	0,31	1,495	0,357	0,709	0,036		29,485
WSL	5,432	0,185	1,531	0,118	0,788	0,052		27,898
MEDIAN	5,592	0,211	1,625	0,1315	0,737	0,048		29,2
DIFF	+37%	-2%	+36%	+3%	-5%	+10%		+9%

WSL v2 (linux internal filesystem)

	citations		listings
	read	processing	read	processing	makes processing	final processing		command time
WSL2-ext4	4,313	0,217	1,261	0,138	1,056	0,029		32,431
WSL2-ext4	3,998	0,195	1,213	0,129	0,908	0,029		30,438
WSL2-ext4	4,422	0,21	1,323	0,111	0,707	4%		30,858
WSL2-ext4	4,124	0,203	1,204	0,143	0,801	0,05		30,163
WSL2-ext4	3,94	0,221	1,17	0,117	0,789	0,054		30,436
WSL2-ext4	4,6	0,214	1,269	0,149	0,951	0,053		31,902
WSL2-ext4	4,654	0,274	1,363	0,156	1,072	0,043		42,752
WSL2-ext4	4,593	0,322	1,293	0,143	0,827	0,06		35,611
WSL2-ext4	4,422	0,29	1,209	0,127	0,963	0,051		35,537
WSL2-ext4	4,222	0,224	1,212	0,121	0,924	0,057		32,653
MEDIAN	4,3675	0,219	1,237	0,1335	0,916	0,0505		32,1665
DIFF	7%	1%	3%	4%	18%	16%		20%

WSL v2 (mounted Win NTFS drive)

	citations		listings
	read	processing	read	processing	makes processing	final processing		command time
WSL2-9p	11,13	0,211	3,447	0,119	0,821	0,04		40,524
WSL2-9p	8,987	0,242	2,882	0,108	0,737	0,031		38,081
WSL2-9p	9,019	0,241	2,916	0,111	0,836	0,042		38,823
WSL2-9p	9,127	0,219	2,834	0,116	0,812	0,036		38,262
WSL2-9p	9,105	0,203	2,925	0,121	0,929	0,051		38,571
WSL2-9p	9,853	0,302	2,944	0,114	0,809	0,047		39,163
WSL2-9p	9,261	0,266	2,877	0,116	0,854	0,042		39,092
WSL2-9p	10,494	0,249	3,028	0,123	0,826	0,027		39,922
WSL2-9p	9,157	0,262	2,935	0,125	1,086	0,051		40,833
WSL2-9p	9,557	0,209	2,92	0,114	0,828	0,039		41,158
MEDIAN	9,209	0,2415	2,9225	0,116	0,827	0,041		39,1275
DIFF	125%	12%	144%	-9%	6%	-6%		46%

Summary

The processing performance is comparable across WSL and native Windows.
Reading CSV is about 35% slower.
Whether files and engine were located in WSL1 local filesystem on invoked through mounted NTFS disk made no difference.
WSL2 suffers heavy penalty when accessing NTFS drives (over 9

[mwu-tow]

Updated with WSL v2 results.

[iamrecursion]

It looks like you've got an unfinished sentence in the big update. Can you clarify what you were trying to say?