Optimizing AST Traversal with Adaptive Iteration

[JoshuaBatty]

Description

Implements a new adaptive_iter method for use when traversing the ASTs. If the size of the collection is less than 8, then regular sequential iteration is used, otherwise a rayon parallel iterator is used. I tested all values between 0-30 and found that 8 to be the sweet spot. Less than this we don't see any benefit as the overhead for spawning rayon threads isn't worth it.

This also fixes a stackoverlow error I noticed below. With this change, we aren't overloading the rayon thread stack size when the recursion level is quite deep.

thread '<unknown>' has overflowed its stack
fatal runtime error: stack overflow
error: test failed, to rerun pass `--test lib`

Caused by:
  process didn't exit successfully: `/Users/josh/Documents/rust/fuel/sway/target/debug/deps/lib-6ef1da655cf2d895 pedro --nocapture` (signal: 6, SIGABRT: process abort signal)

[github-actions]

Benchmark for 6be504b

Click to view benchmark

Test	Base	PR	%
code_action	5.3±0.10ms	5.3±0.09ms	0.00%
code_lens	286.2±10.79ns	293.6±12.60ns	+2.59%
compile	5.8±0.05s	5.8±0.09s	0.00%
completion	4.9±0.16ms	4.8±0.01ms	-2.04%
did_change_with_caching	5.3±0.02s	5.3±0.03s	0.00%
document_symbol	950.0±11.33µs	980.5±21.52µs	+3.21%
format	68.8±1.16ms	73.2±0.90ms	+6.40%
goto_definition	368.1±9.47µs	367.6±6.21µs	-0.14%
highlight	8.8±0.32ms	8.8±0.19ms	0.00%
hover	607.8±23.96µs	607.6±20.09µs	-0.03%
idents_at_position	124.7±0.50µs	122.7±1.65µs	-1.60%
inlay_hints	662.4±23.68µs	666.7±68.08µs	+0.65%
on_enter	485.0±16.90ns	474.7±16.58ns	-2.12%
parent_decl_at_position	3.6±0.02ms	3.6±0.10ms	0.00%
prepare_rename	363.9±7.94µs	366.5±12.91µs	+0.71%
rename	9.2±0.31ms	9.2±0.15ms	0.00%
semantic_tokens	1040.1±28.27µs	1039.0±14.26µs	-0.11%
token_at_position	370.5±5.54µs	365.0±3.70µs	-1.48%
tokens_at_position	3.6±0.03ms	3.6±0.03ms	0.00%
tokens_for_file	410.7±2.87µs	419.3±4.36µs	+2.09%
traverse	43.5±1.69ms	43.0±1.99ms	-1.15%

[github-actions]

Benchmark for 00327e3

Click to view benchmark

Test	Base	PR	%
code_action	5.8±0.24ms	6.3±0.06ms	+8.62%
code_lens	332.0±11.39ns	287.7±7.94ns	-13.34%
compile	6.5±0.12s	6.7±0.10s	+3.08%
completion	5.5±0.11ms	6.8±0.32ms	+23.64%
did_change_with_caching	6.0±0.09s	6.0±0.17s	0.00%
document_symbol	982.7±20.69µs	1008.1±67.22µs	+2.58%
format	72.6±1.18ms	72.2±1.25ms	-0.55%
goto_definition	359.8±5.76µs	372.2±7.53µs	+3.45%
highlight	9.4±0.08ms	10.1±0.39ms	+7.45%
hover	602.8±24.10µs	615.2±17.49µs	+2.06%
idents_at_position	121.9±0.46µs	123.0±1.46µs	+0.90%
inlay_hints	667.7±11.89µs	674.9±32.86µs	+1.08%
on_enter	488.5±17.52ns	484.0±13.42ns	-0.92%
parent_decl_at_position	3.8±0.05ms	4.5±0.06ms	+18.42%
prepare_rename	360.3±4.15µs	360.8±8.79µs	+0.14%
rename	10.1±0.09ms	10.4±0.05ms	+2.97%
semantic_tokens	1088.8±15.20µs	1091.7±32.12µs	+0.27%
token_at_position	361.2±2.51µs	363.5±1.93µs	+0.64%
tokens_at_position	3.9±0.14ms	4.0±0.14ms	+2.56%
tokens_for_file	420.8±2.21µs	412.3±4.84µs	-2.02%
traverse	47.8±1.66ms	49.2±1.37ms	+2.93%

[github-actions]

Benchmark for b76609d

Click to view benchmark

Test	Base	PR	%
code_action	5.4±0.02ms	5.4±0.16ms	0.00%
code_lens	286.3±7.38ns	294.4±9.50ns	+2.83%
compile	5.9±0.05s	5.9±0.08s	0.00%
completion	4.9±0.13ms	5.0±0.05ms	+2.04%
did_change_with_caching	5.4±0.02s	5.3±0.04s	-1.85%
document_symbol	1018.8±52.95µs	988.9±28.37µs	-2.93%
format	70.1±1.32ms	69.5±1.32ms	-0.86%
goto_definition	360.3±4.70µs	361.1±6.47µs	+0.22%
highlight	9.0±0.15ms	9.1±0.24ms	+1.11%
hover	600.8±22.60µs	600.7±17.59µs	-0.02%
idents_at_position	122.1±0.52µs	121.5±1.99µs	-0.49%
inlay_hints	652.6±8.90µs	659.9±22.57µs	+1.12%
on_enter	488.9±17.80ns	480.0±18.43ns	-1.82%
parent_decl_at_position	3.7±0.04ms	3.7±0.03ms	0.00%
prepare_rename	362.3±4.64µs	363.8±5.75µs	+0.41%
rename	9.5±0.06ms	9.5±0.22ms	0.00%
semantic_tokens	1031.0±13.87µs	1051.4±15.73µs	+1.98%
token_at_position	363.0±2.68µs	355.2±2.97µs	-2.15%
tokens_at_position	3.7±0.03ms	3.7±0.04ms	0.00%
tokens_for_file	409.5±1.94µs	414.8±2.25µs	+1.29%
traverse	44.4±1.71ms	43.5±0.94ms	-2.03%