Investigate startup performance of LS+engine

[wdanilo]

This task is automatically imported from the old Task Issue Board and it was originally created by jaroslavtulach.
Original issue is here.

---

The engine startup is extremely slow.

We need to investigate it:

project-manager --profiling-events-log-path=start.log --profiling-path=start.npss --profiling-time=20

try some experiments, remeasure and schedule a plan of further actions.

Blockers:

#184370589 blocked

Comments:

These two files contain one recorded startup of the engine. They were generated using the PR-4110 logging support. To analyze what is happening during the start, download both files, put them next to each other and open the .npss file in VisualVM. (jaroslavtulach - Feb 1, 2023)

---

1164ms of 17s wasted in `org.enso.languageserver.search.SuggestionsHandler.org$enso$languageserver$search$SuggestionsHandler$$generateDocumentation` (jaroslavtulach - Feb 1, 2023)

---

`org.enso.compiler.Compiler.runImportsAndExportsResolution` takes 3s - also because of slow `SerializationManager.deserialize`. (jaroslavtulach - Feb 1, 2023)

---

1.5s initializing the language context including 600ms in initialization of `PackageManager`:

(jaroslavtulach - Feb 1, 2023)

---

**Jaroslav Tulach** reports a new **STANDUP** for yesterday (2023-02-02):

Progress: - measuring suggestions

• proposing benchmark: https://discord.com/channels/401396655599124480/1070399718649237514/1070708726757064714
• measuring: generateDocumentation: https://www.pivotaltracker.com/story/show/184117587/comments/235359646
• book club - issues with API incompatibilities, filter vs. filter_by_expression, https://discord.com/channels/401396655599124480/1068485595984318504/1070667696502476810
• review of type-node: https://github.com/enso-org/enso/pull/- catching up with discord It should be finished by 2023-02-06.

Next Day: Analyzing performance (Enso Bot - Feb 3, 2023)

---

**Jaroslav Tulach** reports a new **STANDUP** for yesterday (2023-02-03):

Progress: - caching suggestions

• raw impl for review: 8fd217d
• caching Suggestions per set of modules, doesn't work: 28064ee
• review and idea: JGit should run its bookkeeping in the background #4116 It should be finished by 2023-02-06.

Next Day: Analyzing performance (Enso Bot - Feb 4, 2023)

---

**Jaroslav Tulach** reports a new **STANDUP** for yesterday (2023-02-04):

Progress: - Pre-compute suggestion db prototype & plan: https://www.pivotaltracker.com/n/projects/2539304/stories/184350816

• fixing XML startup logging: Write the log in XML format suitable for VisualVM 'UI Actions' #4110
• documentation: https://github.com/enso-org/enso/blob/27fcd06c600898f356c1ac555051f735dc91453c/docs/profiler/engine-startup.md
• reviews It should be finished by 2023-02-06.

Next Day: Prepare presentation about analyzing performance (Enso Bot - Feb 5, 2023)

---

[JaroslavTulach]

The Startup investigation by Feb 6 has been finished. Time to report individual issues:

• Pre-compute Suggestion DB during build time #5068
• Investigate More Effective IR Caching format #5567
• Process only Requested Modules when Resolving Imports #5568

Low priority tasks:

• Automating Startup Measurement #5569
• Investigate transfer of Suggestion DB to IDE as binary not as JSON #5570

[JaroslavTulach]

Results of startup improvements as of Mar 9 are promising. After integration of #5068 as well as #5568 and #5791 we managed to speed startup (according to our methodology - #5569) from 11s to 8s. Great work!