Add experimental_remote_cache_key_ignore_stamping to skip volatile stamping files on compute shared cache key

[bozaro]

Changes (enabled by --experimental_remote_cache_key_ignore_stamping flag):

• added the --experimental_remote_cache_key_ignore_stamping flag, which removes volatile files from the cache key;
• since without volatile files it is impossible to execute Action on build farm, Swap with volatile files is executes only locally.

As result, stamping spaws would execute locally, but with correct remote caching.

Related issues:

• Looks like volatile-status.txt content affects the caching key on shared cache #16231
• Bazel remote-caching does not properly treat volatile-status as if it never changes #10075

[ShreeM01]

Hi @bozaro! Could you please fix the build failures? Thanks!

[bozaro]

I don't undestand this test and can't reproduce error locally:

Checking https://mirror.bazel.build/github.com/protocolbuffers/upb/archive/e5f26018368b11aab672e8e8bb76513f3620c579.tar.gz ...
curl: (7) Failed to connect to mirror.bazel.build port 443: Connection timed out
-- Test log: -----------------------------------------------------------
------------------------------------------------------------------------
test_verify_urls FAILED: Invalid urls: https://mirror.bazel.build/github.com/protocolbuffers/upb/archive/e5f26018368b11aab672e8e8bb76513f3620c579.tar.gz.
/b/f/w/bazel-out/k8-fastbuild/bin/src/test/shell/bazel/verify_workspace.runfiles/io_bazel/src/test/shell/bazel/verify_workspace:66: in call to test_verify_urls

Tear down:

[fmeum]

I don't undestand this test and can't reproduce error locally:

Checking https://mirror.bazel.build/github.com/protocolbuffers/upb/archive/e5f26018368b11aab672e8e8bb76513f3620c579.tar.gz ...
curl: (7) Failed to connect to mirror.bazel.build port 443: Connection timed out
-- Test log: -----------------------------------------------------------
------------------------------------------------------------------------
test_verify_urls FAILED: Invalid urls: https://mirror.bazel.build/github.com/protocolbuffers/upb/archive/e5f26018368b11aab672e8e8bb76513f3620c579.tar.gz.
/b/f/w/bazel-out/k8-fastbuild/bin/src/test/shell/bazel/verify_workspace.runfiles/io_bazel/src/test/shell/bazel/verify_workspace:66: in call to test_verify_urls

Tear down:

Looks like a flake due to a network timeout. You can rerun the tests by amending and force-pushing.

[bozaro]

Rebase to master fixed the tests.

[bozaro]

Rebased to master

[bozaro]

I use changes from this PR in patched bazel versions: 5.3.0, 5.3.2, 5.4.0, 6.1.1 (Linux binaries and per-version branches avalialbe here: https://github.com/joomcode/bazel/releases) and it works as expected.

[tjgq]

Closing in favor of --experimental_remote_scrub_config (new in Bazel 7).

[bozaro]

Replace volatile check by --experimental_remote_scrubbing_config usage:

• Allow use scrubbing with remote execution (build actions with scrubbed inputs locally) #20070

[bozaro]

I finished migrating to Basel 7.0.0 + #20070 yesterday. We are using this version to build the production version. I am satisfied.

Now I see the situation as follows:

• Before version 7.0.0, there was no way to use stamping together with a remote cache and a remote build (any Action with volatile-status.txt never gets cache hit). I got out of the situation using the patched version with Add experimental_remote_cache_key_ignore_stamping to skip volatile stamping files on compute shared cache key #16240 (I was building versions 5.4.0, 6.0.0, 6.1.1, 6.2.0, 6.3.2, 6.4.0).
• Since version 7.0.0, it has been possible to use stamping with a remote cache via scrubbing. This is a big step in the right direction. But using srubbing with a remote build is prohibited, which I want to fix by this PR.