Tests run without (network) sandboxing when --java_debug is set

[Mantas-Ragauskas]

ATTENTION! Please read and follow:

• if this is a question about how to build / test / query / deploy using Bazel, ask it on StackOverflow instead: https://stackoverflow.com/questions/tagged/bazel
• if this is a discussion starter, send it to [email protected]
• if this is a bug or feature request, fill the form below as best as you can.

Description of the problem / feature request:

Running test in debug mode reverts network blocking, causing tests, that fail during run, pass on a debug

Feature requests: what underlying problem are you trying to solve with this feature?

Having network enabled on a debug mode causes tests, that try to access network & fail on normal run, to pass.

Bugs: what's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

Have a test calling external service. Normal run would fail, while debug - pass

What operating system are you running Bazel on?

MacOs

What's the output of bazel info release?

release 0.17.2

[Mantas-Ragauskas]

cc @ittaiz

[philwo]

What kind of test are we talking about here? Java tests? For these, this is working as intended - because "debugging a test" involves starting it with the options to listen on a remote debugging port and you can only connect to that port when the network sandbox is disabled.

[Mantas-Ragauskas]

Java tests, that interact with the outside world, to be precise.
Provided there is a test, that interacts with outside world via network, running it in bazel sandbox - fail the test because of the network blocking (as expected). Running same test in debug mode - passes the test.

It isn't a critical issue, but it comes to be a nuisance when investigating the test failures

The most basic idea that comes mind - is it possible to block network for all, but debugger traffic?

[talya]

just for completeness sake, tagging this as relevant to #2765 , as this can add to confusion when triaging tests in sandboxed/non-sandboxed mode.
using --cache_test_results=false is prob the best thing for these cases

[Mantas-Ragauskas]

While i do agree that it will help with not caching false positive test result, the issue lies in the fact, that running test in debug succeeds, while a normal run fails. And that, in turn, is caused by silently disabling the network blocking.
Usability-wise, at very least, a warning, stating that networking blocking has been disabled, would be helpful indicator of a change in environment

[jmmv]

I think this happens because of:

https://cs.opensource.google/bazel/bazel/+/master:src/main/java/com/google/devtools/build/lib/rules/java/JavaOptions.java;l=230;drc=ea0c110dcbf2da094b027c4769738b3a5cf1bf8a

whereby --java_debug sets the test strategy to exclusive, which in turn makes the test run via the standalone strategy.

As far as I can tell, the sandbox allows local network connectivity but not external network connectivity. (Though what "local" means currently depends on the platform, with Linux not allowing connectivity outside of the process' namespace, and macOS not allowing connectivity outside of the machine.) It seems like we could enable sandboxing for tests though and assume that the debugger has to run on the same machine?

Or we could maybe find a way to tunnel the connection out of the sandbox into the host, so that we could block everything except that...

[github-actions]

Thank you for contributing to the Bazel repository! This issue has been marked as stale since it has not had any activity in the last 2.5 years. It will be closed in the next 14 days unless any other activity occurs or one of the following labels is added: "not stale", "awaiting-bazeler". Please reach out to the triage team (@bazelbuild/triage) if you think this issue is still relevant or you are interested in getting the issue resolved.

[github-actions]

This issue has been automatically closed due to inactivity. If you're still interested in pursuing this, please reach out to the triage team (@bazelbuild/triage). Thanks!