Provide alternative to --action_env which doesn't force a rebuild when changed

[StephenWassell]

ATTENTION! Please read and follow:

• if this is a question about how to build / test / query / deploy using Bazel, ask it on StackOverflow instead: https://stackoverflow.com/questions/tagged/bazel
• if this is a discussion starter, send it to [email protected]
• if this is a bug or feature request, fill the form below as best as you can.

Description of the problem / feature request:

It would be very useful to have a command line option that works like --action_env to pass environment variables through to the compiler actions, but without invalidating the target if they are changed.

Feature requests: what underlying problem are you trying to solve with this feature?

We need this to run Bazel with the Coverity static code analysis tool. It works as a build wrapper, for example:
$ cov-build --dir cov-output-dir bazel build ...

The cov-build command sets various environment variables which need to be passed through to the compiler, but some are different for each build, such as COVERITY_TEMP which is set to a randomly named temporary directory. Without using --action_env the Coverity analysis doesn't happen; with --action_env it works but forces a full rebuild each time.

Bugs: what's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

n/a

What operating system are you running Bazel on?

Linux, Windows

What's the output of bazel info release?

release 0.22.0

If bazel info release returns "development version" or "(@non-git)", tell us how you built Bazel.

n/a

What's the output of git remote get-url origin ; git rev-parse master ; git rev-parse HEAD ?

n/a

Have you found anything relevant by searching the web?

No, I don't think many people have tried running Bazel and Coverity together.

Any other information, logs, or outputs that you want to share?

n/a

[jmmv]

As worded, this feature sounds very dangerous. You would be modifying action execution invisibly, making cached actions non-hermetic.

You talk about a wrapper, but then about --action_env. How is the wrapper added to the actions? What environment variables are you trying to pass to the command?

[StephenWassell]

By "build wrapper" I mean the Coverity static analysis tool. When running Coverity you tell it the build command (eg "bazel build //target") and it runs and monitors that command. I'm not sure excactly how the monitoring works, but it requires various environment variables to be passed to the compiler (eg COVERITY_TEMP). They don't affect the output of the build.

[jmmv]

Oh I see. I misread the cov-build calls as wrapping individual compiler invocations, not Bazel as a whole.

So... are you sure this is working? Bazel spawns a server in the background the first time it runs and will reuse the server in future invocations. I don't think running Coverity on Bazel when the server is already running will have any effect, because Coverity won't have any visibility on the server process. Maybe that's why you are seeing trouble. Try with --batch instead.

Can you provide a specific example of what works and what doesn't? It is not clear to me what you pass to --action_env yet. It sounds as if the wrapper is supposed to set some variables, but it also sounds like you are setting those yourself too.

Also, you said that COVERITY_TEMP is set by the wrapper and consumed by the compiler. But... the compiler is the standard compiler, right? It doesn't have any knowledge of these variables.

(Note: it doesn't matter if these variables don't affect the output of the build: exposing a flag to do what you say would be dangerous because people will pass environment variables that do affect the behavior of the build without knowing what the consequences are.)

[StephenWassell]

It works on Linux with the bazel server already running, but on Windows I need to run "bazel shutdown" before cov-build so a new instance of the server starts up within its monitoring environment.

The cov-build wrapper sets the environment variables, and the compiler (eg gcc) needs to see them, or Coverity isn't able analyse the source being compiled. I guess it hooks or replaces the compiler binary somehow.

[jmmv]

Again, please provide specific command lines.

[StephenWassell]

To run the build with Coverity doing static analysis: cov-build --dir covdir bazel build //target

Bazel will execute gcc as normal.

cov-build sets a number of environment variables, but these are the one that cause a problem because they have different values each time we run it:

COVERITY_TEMP # /tmp/cov-stephen/a04ad59bed25b76da4c7995cc122ca8d
COVERITY_TOP_CONFIG # /tmp/cov-stephen/a04ad59bed25b76da4c7995cc122ca8d/cov-configure
COVERITY_BUILD_INVOCATION_ID # 2

[jmmv]

I mean the Bazel invocations. What exactly are you invoking? I.e. what --action_env values are you passing when things work and when they don't?

[StephenWassell]

Sorry, I had them in .bazelrc. I'm not sure if this is the definitive list. The LD_PRELOAD is how it hooks the compiler. I've also been using --spawn_strategy=standalone. It does work like this, just forcing full rebuilds every time.

build --action_env=LD_PRELOAD # /opt/cov-analysis-linux64-2018.03/bin/libcapture-linux64-x86_64.so

build --action_env=COVERITY_BIN # /opt/cov-analysis-linux64-2018.03/bin
build --action_env=COVERITY_BUILD_INVOCATION_ID # 2
build --action_env=COVERITY_COMMON_TEMP # /tmp
build --action_env=COVERITY_COMPILER_PATH_MISMATCH_FILE # /home/stephen/git/engine/src/covdir/has_path_mismatches
build --action_env=COVERITY_CONFIG_FILE # /opt/cov-analysis-linux64-2018.03/config/build --action_env=COVERITY_config.xml
build --action_env=COVERITY_EMIT # /home/stephen/git/engine/src/covdir/emit
build --action_env=COVERITY_IDIR # /home/stephen/git/engine/src/covdir
build --action_env=COVERITY_IS_COMPILER # 0
build --action_env=COVERITY_IS_COMPILER_DESCENDANT # 0
build --action_env=COVERITY_LD_PRELOAD # /opt/cov-analysis-linux64-2018.03/bin/libcapture-linux64-x86_64.so
build --action_env=COVERITY_LOG # /home/stephen/git/engine/src/covdir/build-log.txt
build --action_env=COVERITY_OUTPUT # /home/stephen/git/engine/src/covdir/build-log.txt
build --action_env=COVERITY_OUTPUT_ENCODING # UTF-8
build --action_env=COVERITY_PATHLESS_CONFIGS_FILE # /home/stephen/git/engine/src/covdir/has_pathless_configs
build --action_env=COVERITY_PREV_XML_CATALOG_FILES # 
build --action_env=COVERITY_SITE_CC # 'g++;g++-3;g++-4;gcc;gcc-3;gcc-4;ld'
build --action_env=COVERITY_SYSTEM_ENCODING # UTF-8
build --action_env=COVERITY_TEMP # /tmp/cov-stephen/a04ad59bed25b76da4c7995cc122ca8d
build --action_env=COVERITY_TOP_CONFIG # /tmp/cov-stephen/a04ad59bed25b76da4c7995cc122ca8d/cov-configure/build --action_env=COVERITY_config.xml
build --action_env=COVERITY_TOP_PROCESS # 0

build --action_env=COVERITY_COVERAGE_TYPE # to bullseye
build --action_env=COVERITY_COVERAGE_OUTPUT # to /home/stephen/git/engine/src/covdir/coverage/c
build --action_env=COVERITY_BULLSEYE_BIN # to /opt/BullseyeCoverage/bin
build --action_env=COVAPPDATADIR # for Bullseye to /tmp/cov-stephen/6083c66ed964ab5d2a0be1974ac58856/bullseye-app-data
build --action_env=BULLSEYE_LIB # for Bullseye to /opt/BullseyeCoverage/lib
build --action_env=COVCCFG # for Bullseye to /tmp/cov-stephen/6083c66ed964ab5d2a0be1974ac58856/covc.cfg
build --action_env=COVLINKCFG # for Bullseye to /tmp/cov-stephen/6083c66ed964ab5d2a0be1974ac58856/covlink.cfg
build --action_env=COVERITY_BULLSEYE_DATA # to /home/stephen/git/engine/src/covdir/coverage/bullseye.cov
build --action_env=COVFILE # to /home/stephen/git/engine/src/covdir/coverage/bullseye.cov
build --action_env=COVERITY_SITE_CC_BULLSEYE
build --action_env=COVERITY_SITE_LNK_BULLSEYE

[jmmv]

Oh sorry, I see where my confusion was coming from. The description for --action_env is:

  --action_env (a 'name=value' assignment with an optional value part; may be used multiple times)
    Specifies the set of environment variables available to actions. Variables 
    can be either specified by name, in which case the value will be taken from 
    the invocation environment, or by the name=value pair which sets the value 
    independent of the invocation environment. This option can be used multiple 
    times; for options given for the same variable, the latest wins, options 
    for different variables accumulate.
      Tags: action_command_lines

and the first line implies that the values are name=value pairs, not just bare names. I hadn't read the full description so I couldn't understand what you were trying to say.

[Globegitter]

@StephenWassell One potential way around this could be by using volatile stamp values: https://docs.bazel.build/versions/master/user-manual.html#workspace_status

[adam-azarchs]

Is this maybe solved by --repo_env? You can make a repository rule that records the relevant environment variables into a .bzl file or whatever, which then gets read by downstream rules. That way only things which depend on that environment variable get rebuilt.

[github-actions]

Thank you for contributing to the Bazel repository! This issue has been marked as stale since it has not had any activity in the last 1+ years. It will be closed in the next 14 days unless any other activity occurs or one of the following labels is added: "not stale", "awaiting-bazeler". Please reach out to the triage team (@bazelbuild/triage) if you think this issue is still relevant or you are interested in getting the issue resolved.

[github-actions]

This issue has been automatically closed due to inactivity. If you're still interested in pursuing this, please reach out to the triage team (@bazelbuild/triage). Thanks!