rust-analyzer tries to check all crates in vscode workspace in parallel

[jbg]

After installing this extension for the first time, and shortly after opening a VSCode workspace with around 40-50 crates in it, my laptop becomes completely unresponsive.

Investigation shows around 200 rustc processes running. rust-analyzer seems to be running cargo check on every one of the crates in my workspace at the same time, regardless of how many crates there are and how many CPUs I have.

After killall rustc I am able to use my computer again, but when working in the workspace the problem periodically recurs.

Is there a way to limit how many cargo check processes will be run at a time? Could this default to something based on the number of CPUs the system has?

[flodiebold]

Related to #8622, we should maybe run the initial cargo check lazily (maybe except for a top-level workspace).

Assuming this is just a folder with a bunch of crates in it, and not a Cargo workspace, I would probably recommend opening the specific crate folders instead of the top-level folders, at least for now.

[jonas-schievink]

This sounds like we might want to provide a shared jobserver to all Cargo processes we spawn

[jbg]

It's a VSCode "workspace" containing around 40-50 individual Cargo crates, but not a Cargo workspace. I've turned off cargo check for now as I can't find a way to open them separately without having multiple windows open, which makes simultaneously working on multiple crates unpleasant.

[flodiebold]

@jonas-schievink that's a good idea, but I don't think it would really solve the usability problem with such a large workspace. I.e. presumably you want to work on one or two crates out of the 50, but you still have to wait for all 50 to be built for everything to work.

[jbg]

In my specific case that'd be fine, but in general I agree it would make sense for projects with opened files to be built first.

[flodiebold]

Although actually the initial check shouldn't take that long, since we don't actually compile the crates. But flycheck (check on save) should really be done lazily only for crates that are actually opened, if we're not doing that already.

[flodiebold]

Still, I would tend to say that probably support for such huge workspaces isn't really a goal right now (but maybe @matklad and @jonas-schievink disagree).

[matklad]

I don't think we can make wins here with laziness. As soon as the user invokes "workspace symbols", we'd have to process the world anyways. It feels like another case where our automated discovery of the project structure falls down, and where an explicit configuration in terms of linkedProjects is warranted.

For the time being, I suggest not fixing this issue.

Long term, I think the right fix would be to, if our auto-discovery found > 5 crates, show a message to the user about the project load.

[flodiebold]

As soon as the user invokes "workspace symbols", we'd have to process the world anyways

... unless we don't do that for crates that aren't loaded currently (that was what I was imagining). Of course that would make it very non-obvious what will be covered by workspace symbols / find references etc., which I guess is kind of what you refer to in #8622. It seems to me what we really want would be a nice way for the user to dynamically control which crates are loaded and which aren't; you're probably right that doing it completely lazily/implicitly is also not a good user experience. Maybe this is best left to the client.

[matklad]

Yeah, IntelliJ has unloaded modules thing for that: https://www.jetbrains.com/help/idea/unloading-modules.html#7a53c16f.

IIRC, it was added because IJ struggled with JetBrains own monrepo :D

[jbg]

FWIW, every other feature of the extension that I've tried works snappily in my large workspace, including find references, workspace symbols, etc. It's only the spawning of 200 rustc processes that have caused me a problem.

(Maybe find references etc only have a partial view of the "world" though since I disabled cargo check after attempting to open the workspace a few times?)

[matklad]

Ahh, wait, I think the problem is not the initial cargo check (we do that sequentially) but the checkOnSave cargo check. That one is fully parallel, and probably should be made sequential as well.

[jbg]

That aligns with my experience. My laptop didn't start to melt as soon as I opened the workspace, but a while later — probably shortly after I saved a file.

[crepererum]

I think sequential / less parallel cargo check on save can easily be archived by setting Rust Analyzer > Check On Save: Extra Args to something like -j1 (sequential) or -j2 (less parallel, or use any other number).

However, another thing that could help would be instead of checking the entire workspace to only check the current crate on save and offer some extra command or a low-prio background task to check the whole workspace.

[jbg]

This would only work if it's a Cargo workspace, right? Because rust-analyzer is actually running a separate cargo check for each crate in the folder (that I have opened as a VSCode workspace), isn't it?

[crepererum]

Because rust-analyzer is actually running a separate cargo check for each crate in the folder (that I have opened as a VSCode workspace), isn't it?

Ahh, I didn't know that. 💡

[wycats]

Is there a good reason not to run cargo check --all?

I sometimes encounter situations where cargo check --all has errors in it, but rust-analyzer is silent. I'm not sure if it's related to this exact issue, but it seems at least tangentially related to the delta between what r-a does and cargo check --all.

[jbg]

That's not related to this issue - this issue is about r-a running cargo check on every crate within a vscode workspace (which is not necessarily a cargo workspace) at the same time instead of having some parallelism limit. The issue would be the same whether --all was passed or not.

[pitaj]

I'm thinking of working on this issue, but I'm wondering:

If the issue is that we're rechecking every cargo-workspace in the vscode-workspace on each save, shouldn't we change that so that we detect which cargo-workspace the given saved file was in, and only recheck that cargo-workspace?

[bjorn3]

If another cargo workspace depends on the changed crate as deoendency, that other cargo workspace needs to be recompiled anyway. If it doesn't, cargo check will immediately return as there is nothing to recompile.

[pitaj]

That doesn't seem to align with @jbg's experience, unless he has a large amount of interdependent crates inside his editor-workspace.

edit: Also, technically crates only need to be rechecked if pub items on the dependency boundary changed. Which should be pretty rare when working on them independently.

[jbg]

My crates are mostly not interdependent. I gave up on using rust-analyser with vscode some time ago so can't comment on whether this still happens in the latest versions.

[bjorn3]

I think @jbg's problem is at startup and not while editing files. The fix would be to invoke all cargo check commands individually or maybe even better to setup a jobserver to limit the amount of parallel rustc instances while still allowing multiple cargo check commands at the same time.

[pitaj]

According to this comment that is not the case

[pacak]

I would like to be able to limit the check to currently open crates on save - in a big workspace working on anything imported by a bunch of other stiff is unnecessary painful - checking on saving take a long of time (during which I can't run tests because of a lock on the build directory) and I don't really care how other crates are affected until I decide to start working on them.

[pitaj]

What do you mean by "currently open crates"? Crates that are in your vscode workspace? Only crates that have files open in an editor?

[pacak]

What do you mean by "currently open crates"?

In my case it's crates that have open files in vim.

[pitaj]

Are those crates part of a cargo workspace, or just all open in vim?

[pacak]

Mostly belong to the same workspace, but then there's two separate crates that define their own workspaces in order to avoid messing with feature unification. It would be enough just to deal with the workspace though.

[pitaj]

I suspect the workspace may be where the target dir locking is really showing up. Have you tried disabling the cargo workspace?

[pacak]

I suspect the workspace may be where the target dir locking is really showing up.

Yes, pretty much.

Have you tried disabling the cargo workspace?

Not really an option. There's a bunch of dependencies and workspace allows to share compilation results, there's feature unification and some other stuff.

[pitaj]

So essentially what you want is a way to pass --package <open crate> option to cargo check.

[pacak]

I think this would do the trick, yes. Optionally with --lib --bin <name> / --test <name> / etc, but even with --all-targets it's going to be a huge improvement.

[pitaj]

Okay. I think that's kind of outside the scope of this particular issue, which is about vscode workspaces, not cargo workspaces.

There is an option to set the cargo check command. Maybe it's something to look into for you. I know vim is pretty customizable, maybe it's scriptable on your part.

Otherwise, consider opening a new issue for what you need.

[Henry-E]

So there is currently no way to manually limit what crates in workspace are checked?

The workaround as mentioned before is to open each crate in its own vscode window, which is not ideal. Maybe something like a .gitignore file would allow at least some customization of what gets checked until it's manually reduced down to a level that vscode / rust analyzer can actually handle.

Really missing those type hints!

[manaskarekar]

Glad to see some discussion around this, I'm experiencing this a lot lately, I've disabled rust analyzer because of it. Perhaps I should specify the workdir with a finer granularity.

[Veykril]

The original reported issue shouldn't necessary be a problem anymore I think? (unless there are 50 workspaces, not just crates)