Bazel creates a lot of write IO for sandboxed action

[abergmeier-dsfishlabs]

Description of the problem / feature request / question:

We have a rule related to packaging. That rule depends on > 25.000 files. Trying to build that rule it stays in Running for over 10 minutes without the action executable even being started. Since the system is mainly busy with writing, I guess Bazel is setting up the sandbox environment/files?

Question is whether there is anything that could be improved here or the general advise is to disable Sandboxing for that action.
Also is there a rule of thumb when Sandboxing is slowing down builds too much?

Environment info

• Operating System: Ubuntu 16.04
• Bazel version (output of bazel info release): 0.4.5

Anything else, information or logs or outputs that would be helpful?

The disk is a HDD - which makes this problem probably even more visible.

[AustinSchuh]

Take a look at Issue #1836 . Does the patch/workaround listed there help you?

[jmmv]

Yes, sandboxing is known to be slow precisely by how many symlinks it creates. It's usually not a problem on Linux with SSDs, but with an HDD I can see that being very painful. (And it's painful on macOS regardless.)

sandboxfs is something that you could try as it was designed precisely to avoid creating the symlinks.

But... given the age of this bug, I suspect this is not relevant any longer.

[ulfjack]

I suspect that a lot of people just disable sandboxing when they encounter this. The easier we can make it to use sandboxfs (assuming that solves it), the better. (Keeping in mind that FUSE is disallowed in some places by security policy.)

[AustinSchuh]

@ulfjack , we've been using --experimental_sandbox_base=/dev/shm to put all the symlinks in a tmpfs on Linux. It makes the performance difference negligible. Simpler than FUSE, and probably as fast or faster.