Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Load files into the VFS in parallel on startup #17373

Open
Veykril opened this issue Jun 9, 2024 · 2 comments
Open

Load files into the VFS in parallel on startup #17373

Veykril opened this issue Jun 9, 2024 · 2 comments
Labels
A-perf performance issues C-enhancement Category: enhancement

Comments

@Veykril
Copy link
Member

Veykril commented Jun 9, 2024

We currently load them from a single thread which can make startup very slow for bigger projects as we re-query the project layout over and over again until all files are loaded (this is a separate issue that we should fix, that is wait with that until all files are loaded into memory).

@Veykril Veykril added A-perf performance issues C-enhancement Category: enhancement labels Jun 9, 2024
@davidbarsky
Copy link
Contributor

For some additional context from Zulip, the current single-threaded loading behavior is especially problematic on EdenFS, which really wants tooling to send as many IO requests as possible in parallel.

@davidbarsky
Copy link
Contributor

this is a separate issue that we should fix, that is wait with that until all files are loaded into memory.

I did some rough, back-of-napkin benchmarking on this: rust-analyzer spends 5 seconds waiting on rustc for every 100 crates it loads (just printed Instant::elapsed::as_millis...). Given that total crate dependency graphs of over 1000 are not uncommon for larger Rust projects in the wild, it's likely that people are unnecessarily waiting for over a minute.

What makes this fix a little more complicated is that during project loading, there isn't a clear end-state: rust-analyzer just sorta repeatedly calls GlobalState::fetch_workspaces.

bors added a commit that referenced this issue Aug 5, 2024
internal: Load VFS config changes in parallel

Simple attempt to make some progress f or #17373
No clue if those atomic orderings are right, though I don't think they are really too relevant either.

A more complete fix would probably need to replace our `ProjectFolders` handling a bit.
lnicola pushed a commit to lnicola/rust that referenced this issue Aug 13, 2024
internal: Load VFS config changes in parallel

Simple attempt to make some progress f or rust-lang/rust-analyzer#17373
No clue if those atomic orderings are right, though I don't think they are really too relevant either.

A more complete fix would probably need to replace our `ProjectFolders` handling a bit.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-perf performance issues C-enhancement Category: enhancement
Projects
None yet
Development

No branches or pull requests

2 participants