block-network not working as expected in darwin-sandbox

[robbertvanginkel]

Description of the problem / feature request:

When using block-network tags with strategy darwin-sandbox, network calls are not blocked as expected.

To reproduce:

$ cat BUILD.bazel 
genrule(
   name = "test",
   cmd = "curl -I http://www.google.com | tee $@ && false",
   outs = ["out"],
   tags = ["block-network"],
)
$ bazel build //:test --sandbox_debug -s --incompatible_strict_action_env
INFO: Writing tracer profile to '/private/var/tmp/_bazel_robbert/2187421153de0d0eb425e00ad6f725cb/command.profile.gz'
INFO: Invocation ID: affcb0fb-4180-4685-9165-847573a115ff
INFO: Build option --incompatible_strict_action_env has changed, discarding analysis cache.
INFO: Analyzed target //:test (0 packages loaded, 9 targets configured).
INFO: Found 1 target...
SUBCOMMAND: # //:test [action 'Executing genrule //:test', configuration: 4c084a9de07c3dda45bf268b13a175a7]
(cd /private/var/tmp/_bazel_robbert/2187421153de0d0eb425e00ad6f725cb/execroot/__main__ && \
  exec env - \
    PATH=/bin:/usr/bin:/usr/local/bin \
  /bin/bash -c 'source external/bazel_tools/tools/genrule/genrule-setup.sh; curl -I http://www.google.com | tee bazel-out/darwin-fastbuild/bin/out.a && false')
ERROR: /private/var/folders/p4/px0ds0qs4w58bk9fjp34q32h0000gn/T/tmp.0Cqr3BSf/BUILD.bazel:1:1: Executing genrule //:test failed (Exit 1) sandbox-exec failed: error executing command 
  (cd /private/var/tmp/_bazel_robbert/2187421153de0d0eb425e00ad6f725cb/sandbox/darwin-sandbox/4/execroot/__main__ && \
  exec env - \
    PATH=/bin:/usr/bin:/usr/local/bin \
    TMPDIR=/var/folders/p4/px0ds0qs4w58bk9fjp34q32h0000gn/T/ \
  /usr/bin/sandbox-exec -f /private/var/tmp/_bazel_robbert/2187421153de0d0eb425e00ad6f725cb/sandbox/darwin-sandbox/4/sandbox.sb /var/tmp/_bazel_robbert/install/3347a66238555d260056bee3208b0d0f/_embedded_binaries/process-wrapper '--timeout=0' '--kill_delay=15' /bin/bash -c 'source external/bazel_tools/tools/genrule/genrule-setup.sh; curl -I http://www.google.com | tee bazel-out/darwin-fastbuild/bin/out.a && false')
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
HTTP/1.1 200 OK
...

Target //:test failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 0.264s, Critical Path: 0.14s
INFO: 0 processes.
FAILED: Build did NOT complete successfully
$ cat /private/var/tmp/_bazel_robbert/2187421153de0d0eb425e00ad6f725cb/sandbox/darwin-sandbox/1/sandbox.sb
(version 1)
(debug deny)
(allow default)
(deny network*)
(allow network* (local ip "localhost:*"))
(allow network* (remote ip "localhost:*"))
(allow network* (remote unix-socket))
(deny file-write*)
(allow file-write*
    (subpath "/dev")
    (subpath "/private/var/tmp/_bazel_robbert/2187421153de0d0eb425e00ad6f725cb/sandbox/darwin-sandbox/1/execroot/__main__")
    (subpath "/private/var/folders/p4/px0ds0qs4w58bk9fjp34q32h0000gn/C")
    (subpath "/private/var/tmp")
    (subpath "/Users/robbert/Library/Logs")
    (subpath "/Users/robbert/Library/Developer")
    (subpath "/private/var/folders/p4/px0ds0qs4w58bk9fjp34q32h0000gn/T")
    (subpath "/private/tmp")
)

What operating system are you running Bazel on?

macOS 10.14.6

What's the output of bazel info release?

$ bazel info release
INFO: Writing tracer profile to '/private/var/tmp/_bazel_robbert/2187421153de0d0eb425e00ad6f725cb/command.profile.gz'
INFO: Invocation ID: 183fb17f-2ab0-4360-a6f9-e8bbbe4c6316
release 1.0.0

Have you found anything relevant by searching the web?

Tried issues, bazel-discuss, and stack. Did not find anything relevant.

From reading the source, it seems like darwin-sanbox should support blocking network:
https://source.bazel.build/bazel/+/master:src/main/java/com/google/devtools/build/lib/sandbox/DarwinSandboxedSpawnRunner.java;l=341?q=%22allow%20network%22&ss=bazel

Any other information, logs, or outputs that you want to share?

[irengrig]

/cc @buchgr

[tetromino]

Confirmed; on 10.14.6, the following sandbox.sb profile will allow external network access from curl:

(version 1)
(allow default)
(deny network*)
(allow network* (local ip "localhost:*"))
(allow network* (remote ip "localhost:*"))
(allow network* (remote unix-socket))

(To test, call sandbox-exec -f sandbox.sb my_command.sh)

To block network access, either one of the following is sufficient:

• remove (allow network* (local ip "localhost:*"))
• remove (allow network* (remote unix-socket)

Pinging @jmmv and @philwo (see PR #3444)

[jmmv]

I think our sandbox-exec specification is just wrong and sandbox-exec swallows it without complaining. /usr/share/sandbox/ contains plenty of sample configurations which I'm pretty sure work fine, and from there I got the following:

(version 1)
(allow default)
(deny network*)
(allow network* (local tcp "localhost:*"))
(allow network* (remote tcp "localhost:*"))
(allow network* (remote unix-socket (path-literal "/")))

Note that there is no ip provider for networking: it has to be tcp or udp. (Well, there is ip, but it doesn't seem to accept an argument.)

Also note that unix-socket requires a path-literal: without specifying that, all networking is whitelisted.

[philwo]

Thanks @tetromino for reporting and thanks @jmmv for investigating!

I guess this is just the sandbox profile API changing underneath our feet. When I initially wrote the profile, I based it on the reverse engineered documentation available on the internet. Looking at the profiles in /usr/share/sandbox, I see a lot of features that didn't exist at the time.

I'd suggest to first add a test that proves that it's currently not working (e.g. test that it should block, but doesn't) and then add a fix for the profile that turns the test green.

[jmmv]

I think this deserves being a P1 bug given that, the longer this remains broken, the more projects will gain rules that violate the sandboxing restrictions.

[jmmv]

But I'm concerned that fixing this issue can actually be seen as a breaking change... so sent https://groups.google.com/g/bazel-dev/c/PDhzYQd6umE to seek guidance.

[jmmv]

To give an update on this, I have a one-line fix ready (it's just a matter of removing the local tcp line). However, I'm working first on extending the integration tests to properly validate all cases we care about (local networking ok, remote networking not ok, unix domains ok), both on Linux and on Mac, to ensure the fix does the right thing and continues to do so in the future.

[jmmv]

Should be fixed now. Please let me know if you confirm.