Build on CI machine failing with error "Couldn't find a valid ICU package installed on the system"

[Inirit]

As part of our effort to update our project's version of rules_dotnet from 0.8.0 to 0.10.x, we are encountering an error that is only happening in our CI environment. This seems to be related to #354 and I applied the suggested changes to our bazelrc as "build --action_env=DOTNET_SYSTEM_GLOBALIZATION_INVARIANT=1" but according to the build logs (with --subcommands) it doesn't seem like this environment variable is being propagated to the action:

(00:52:19) SUBCOMMAND: # //examples/services/example/hello:hello [action 'Compiling hello', configuration: 0e659f816117e00b08ca48f5633c0c4d76108d47883cbc033e19f1e081b9bee5, execution platform: @local_config_platform//:host]
(cd /root/.cache/bazel/_bazel_root/bf4737d62853c1db3f98f41cea262406/execroot/<PROJECT_NAME> && \
  exec env - \
    DOTNET_CLI_HOME=external/dotnet_x86_64-unknown-linux-gnu \
  external/rules_dotnet/dotnet/private/compiler_wrapper.sh external/dotnet_x86_64-unknown-linux-gnu/dotnet external/dotnet_x86_64-unknown-linux-gnu/sdk/6.0.400/Roslyn/bincore/csc.dll @bazel-out/k8-fastbuild-ST-19c30ace4401/bin/examples/services/example/hello/bazelout/net6.0/hello.dll-0.params)

...

(00:52:19) ERROR: /codebuild/output/src3297254181/src/github.com/<REPO_ORG>/<REPO_NAME>/examples/services/example/hello/BUILD:6:14: Compiling hello failed: (Aborted): compiler_wrapper.sh failed: error executing command (from target //examples/services/example/hello:hello) external/rules_dotnet/dotnet/private/compiler_wrapper.sh external/dotnet_x86_64-unknown-linux-gnu/dotnet external/dotnet_x86_64-unknown-linux-gnu/sdk/6.0.400/Roslyn/bincore/csc.dll ... (remaining 1 argument skipped)

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
Process terminated. Couldn't find a valid ICU package installed on the system. Please install libicu using your package manager and try again. Alternatively you can set the configuration flag System.Globalization.Invariant to true if you want to run with no globalization support. Please see https://aka.ms/dotnet-missing-libicu for more information.

It sounds like this is maybe related to bazelbuild/bazel#17062, but to tell you the truth I can't make complete sense of what they're saying.

It may be worth noting that this environment variable does apply during tests, in fact we originally applied this exact same bazelrc change to fix our tests not executing on our CI environment. There just seems to be something about rule configurations (perhaps the use of transitions?) changing the behavior of --action_env despite the documentation claiming it should affect the execution of all actions.

[purkhusid]

If you have control over your CI environment you can also work around this by installing libicu on the CI worker. I'm currently a bit too busy to dive into this.

[purkhusid]

Closing this since libicu is a host requirement of the .Net SDK and I'm not planning on taking a dependency to supply it to the compile actions for now.