-
Notifications
You must be signed in to change notification settings - Fork 4.1k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
rules/python: Add a
coverage_tool
attribute to py_runtime
.
This allows users to specify a target providing the coveragepy tool (and its dependencies). This is essential for hermetic python builds, where an absolute path will not really work. It's also superior to other potential methods using environment variables because the runfiles dependency on the coverage tool and its files is only incurred when building with coverage enabled. This also builds on the work @TLATER began with #14677 to integrate with `coveragepy`'s `lcov` support, with an additional step of at least attempting to convert the absolute paths which `coveragepy` uses in the lcov output into the relative paths which the rest of bazel can actually consume. This is my first time touching Java code professionally, so I'll admit to mostly cargo-culting those parts, and would welcome any feedback on how to improve things there. I also would have no objections to someone else taking over this PR to get it over the finish line. I've tested this out with our own team's internal monorepo, and have successfully generated a full combined coverage report for most of our python and go code. There's still a bunch of things which don't quite work, in particular when it comes to compiled extension modules or executables run from within python tests, but those will need to be addressed separately, and this is already a giant step forward for our team. Closes #14436. Closes #15590. PiperOrigin-RevId: 476314433 Change-Id: I4be4d10e0af741f4ba1a7b5367c6f7a338a3c43d
- 9.0.0-pre.20250106.2
- 9.0.0-pre.20241208.2
- 9.0.0-pre.20241208.1
- 9.0.0-pre.20241205.2
- 9.0.0-pre.20241125.4
- 9.0.0-pre.20241125.3
- 9.0.0-pre.20241119.2
- 9.0.0-pre.20241113.4
- 9.0.0-pre.20241105.2
- 9.0.0-pre.20241026.2
- 9.0.0-pre.20241023.1
- 9.0.0-pre.20241020.2
- 9.0.0-pre.20241016.1
- 8.0.0
- 8.0.0rc8
- 8.0.0rc7
- 8.0.0rc6
- 8.0.0rc5
- 8.0.0rc4
- 8.0.0rc3
- 8.0.0rc2
- 8.0.0rc1
- 8.0.0-pre.20240925.4
- 8.0.0-pre.20240911.1
- 8.0.0-pre.20240909.1
- 8.0.0-pre.20240826.1
- 8.0.0-pre.20240821.2
- 8.0.0-pre.20240819.2
- 8.0.0-pre.20240814.2
- 8.0.0-pre.20240812.1
- 8.0.0-pre.20240807.1
- 8.0.0-pre.20240805.3
- 8.0.0-pre.20240730.1
- 8.0.0-pre.20240729.1
- 8.0.0-pre.20240724.1
- 8.0.0-pre.20240718.2
- 8.0.0-pre.20240710.4
- 8.0.0-pre.20240701.1
- 8.0.0-pre.20240618.2
- 8.0.0-pre.20240607.2
- 8.0.0-pre.20240603.2
- 8.0.0-pre.20240530.1
- 8.0.0-pre.20240523.3
- 8.0.0-pre.20240516.1
- 8.0.0-pre.20240422.4
- 8.0.0-pre.20240415.1
- 8.0.0-pre.20240404.3
- 8.0.0-pre.20240401.3
- 8.0.0-pre.20240303.2
- 8.0.0-pre.20240303.1
- 8.0.0-pre.20240226.1
- 8.0.0-pre.20240213.1
- 8.0.0-pre.20240206.3
- 8.0.0-pre.20240128.3
- 8.0.0-pre.20240108.7
- 8.0.0-pre.20240108.6
- 8.0.0-pre.20240101.1
- 8.0.0-pre.20231030.2
- 7.4.1
- 7.4.1rc2
- 7.4.1rc1
- 7.4.0
- 7.4.0rc4
- 7.4.0rc3
- 7.4.0rc2
- 7.4.0rc1
- 7.3.2
- 7.3.2rc1
- 7.3.1
- 7.3.1rc2
- 7.3.1rc1
- 7.3.0
- 7.3.0rc2
- 7.3.0rc1
- 7.2.1
- 7.2.1rc2
- 7.2.1rc1
- 7.2.0
- 7.2.0rc3
- 7.2.0rc2
- 7.2.0rc1
- 7.1.2
- 7.1.2rc2
- 7.1.2rc1
- 7.1.1
- 7.1.1rc2
- 7.1.1rc1
- 7.1.0
- 7.1.0rc2
- 7.1.0rc1
- 7.0.2
- 7.0.2rc1
- 7.0.1
- 7.0.1rc2
- 7.0.1rc1
- 7.0.0
- 7.0.0-pre.20231018.3
- 7.0.0-pre.20231011.2
- 7.0.0-pre.20230926.1
- 7.0.0-pre.20230917.3
- 7.0.0-pre.20230906.2
- 7.0.0-pre.20230823.4
- 7.0.0-pre.20230816.3
- 7.0.0-pre.20230810.1
- 7.0.0-pre.20230724.1
- 7.0.0-pre.20230710.5
- 7.0.0-pre.20230628.2
- 7.0.0-pre.20230530.3
- 7.0.0-pre.20230524.3
- 7.0.0-pre.20230517.4
- 7.0.0-pre.20230504.4
- 7.0.0-pre.20230502.1
- 7.0.0-pre.20230426.1
- 7.0.0-pre.20230420.2
- 7.0.0-pre.20230417.1
- 7.0.0-pre.20230412.2
- 7.0.0-pre.20230410.1
- 7.0.0-pre.20230405.2
- 7.0.0-pre.20230330.3
- 7.0.0-pre.20230322.4
- 7.0.0-pre.20230316.2
- 7.0.0-pre.20230306.4
- 7.0.0-pre.20230302.1
- 7.0.0-pre.20230215.2
- 7.0.0-pre.20230209.2
- 7.0.0-pre.20230128.3
- 7.0.0-pre.20230123.5
- 7.0.0-pre.20230118.2
- 7.0.0-pre.20230104.2
- 7.0.0-pre.20221212.2
- 7.0.0-pre.20221207.2
- 7.0.0-pre.20221204.2
- 7.0.0-pre.20221123.2
- 7.0.0-pre.20221111.3
- 7.0.0-pre.20221102.3
- 7.0.0-pre.20221026.2
- 6.5.0
- 6.5.0rc2
- 6.5.0rc1
- 6.4.0
- 6.3.2
- 6.3.1
- 6.3.0
- 6.2.1
- 6.2.0
- 6.1.2
- 6.1.1
- 6.1.0
- 6.0.0
- 6.0.0-pre.20221020.1
- 6.0.0-pre.20221012.2
- 6.0.0-pre.20221007.4
1 parent
4c56431
commit 9d01630
Showing
10 changed files
with
680 additions
and
50 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
174 changes: 174 additions & 0 deletions
174
src/test/shell/bazel/bazel_coverage_hermetic_py_test.sh
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,174 @@ | ||
#!/bin/bash | ||
# | ||
# Copyright 2015 The Bazel Authors. All rights reserved. | ||
# | ||
# Licensed under the Apache License, Version 2.0 (the "License"); | ||
# you may not use this file except in compliance with the License. | ||
# You may obtain a copy of the License at | ||
# | ||
# http://www.apache.org/licenses/LICENSE-2.0 | ||
# | ||
# Unless required by applicable law or agreed to in writing, software | ||
# distributed under the License is distributed on an "AS IS" BASIS, | ||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
# See the License for the specific language governing permissions and | ||
# limitations under the License. | ||
|
||
set -eu | ||
|
||
# Load the test setup defined in the parent directory | ||
CURRENT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" | ||
source "${CURRENT_DIR}/../integration_test_setup.sh" \ | ||
|| { echo "integration_test_setup.sh not found!" >&2; exit 1; } | ||
|
||
# Fetch hermetic python and register toolchain. | ||
function set_up() { | ||
cat >>WORKSPACE <<EOF | ||
register_toolchains( | ||
"//:python_toolchain", | ||
) | ||
EOF | ||
} | ||
|
||
# Returns the path of the code coverage report that was generated by Bazel by | ||
# looking at the current $TEST_log. The method fails if TEST_log does not | ||
# contain any coverage report for a passed test. | ||
function get_coverage_file_path_from_test_log() { | ||
local ending_part | ||
ending_part="$(sed -n -e '/PASSED/,$p' "$TEST_log")" | ||
|
||
local coverage_file_path | ||
coverage_file_path=$(grep -Eo "/[/a-zA-Z0-9\.\_\-]+\.dat$" <<< "$ending_part") | ||
[[ -e "$coverage_file_path" ]] || fail "Coverage output file does not exist!" | ||
echo "$coverage_file_path" | ||
} | ||
|
||
function set_up_py_test_coverage() { | ||
# Set up python toolchain. | ||
cat <<EOF > BUILD | ||
load("@bazel_tools//tools/python:toolchain.bzl", "py_runtime_pair") | ||
py_runtime( | ||
name = "py3_runtime", | ||
coverage_tool = ":mock_coverage", | ||
interpreter_path = "$(which python3)", | ||
python_version = "PY3", | ||
) | ||
py_runtime_pair( | ||
name = "python_runtimes", | ||
py2_runtime = None, | ||
py3_runtime = ":py3_runtime", | ||
) | ||
toolchain( | ||
name = "python_toolchain", | ||
toolchain = ":python_runtimes", | ||
toolchain_type = "@bazel_tools//tools/python:toolchain_type", | ||
) | ||
EOF | ||
# Add a py_library and test. | ||
cat <<EOF >> BUILD | ||
py_library( | ||
name = "hello", | ||
srcs = ["hello.py"], | ||
) | ||
py_library( | ||
name = "mock_coverage", | ||
srcs = ["mock_coverage.py"], | ||
deps = [":coverage_support"], | ||
) | ||
py_library( | ||
name = "coverage_support", | ||
srcs = ["coverage_support.py"], | ||
) | ||
py_test( | ||
name = "hello_test", | ||
srcs = ["hello_test.py"], | ||
deps = [":hello"], | ||
) | ||
EOF | ||
echo "# fake dependency" > coverage_support.py | ||
cat <<EOF > mock_coverage.py | ||
#!/usr/bin/env python3 | ||
import argparse | ||
import os | ||
import subprocess | ||
import sys | ||
import coverage_support | ||
parser = argparse.ArgumentParser() | ||
mode = sys.argv[1] | ||
del(sys.argv[1]) | ||
parser.add_argument("--rcfile", type=str) | ||
parser.add_argument("--append", action="store_true") | ||
parser.add_argument("--branch", action="store_true") | ||
parser.add_argument("--output", "-o", type=str) | ||
parser.add_argument("target", nargs="*") | ||
args = parser.parse_args() | ||
tmp_cov_file = os.path.join(os.environ["COVERAGE_DIR"], "tmp.out") | ||
if mode == "run": | ||
subprocess.check_call([sys.executable]+args.target) | ||
with open(tmp_cov_file, "a") as tmp: | ||
tmp.write("TN:\nSF:") | ||
tmp.write(os.path.join(os.path.dirname(os.path.realpath(args.target[0])), "hello.py")) | ||
tmp.write(""" | ||
FNF:0 | ||
FNH:0 | ||
DA:1,1,fi+A0ud2xABMExsbhdW38w | ||
DA:2,1,3qA2I6CcUyJmcd1vpeVcRA | ||
DA:4,1,nFnrj5CwYCqkvbVhPUFVVw | ||
DA:5,0,RmWioilSA3bI5NbLlwiuSA | ||
LH:3 | ||
LF:4 | ||
end_of_record | ||
""") | ||
else: | ||
with open(args.output, "w") as out_file: | ||
with open(tmp_cov_file, "r") as in_file: | ||
out_file.write(in_file.read()) | ||
EOF | ||
cat <<EOF > hello.py | ||
def Hello(): | ||
print("Hello, world!") | ||
def Goodbye(): | ||
print("Goodbye, world!") | ||
EOF | ||
cat <<EOF > hello_test.py | ||
import unittest | ||
import hello | ||
class Tests(unittest.TestCase): | ||
def testHello(self): | ||
hello.Hello() | ||
if __name__ == "__main__": | ||
unittest.main() | ||
EOF | ||
cat <<EOF > expected.dat | ||
SF:hello.py | ||
FNF:0 | ||
FNH:0 | ||
DA:1,1,fi+A0ud2xABMExsbhdW38w | ||
DA:2,1,3qA2I6CcUyJmcd1vpeVcRA | ||
DA:4,1,nFnrj5CwYCqkvbVhPUFVVw | ||
DA:5,0,RmWioilSA3bI5NbLlwiuSA | ||
LH:3 | ||
LF:4 | ||
end_of_record | ||
EOF | ||
} | ||
|
||
function test_py_test_coverage() { | ||
set_up_py_test_coverage | ||
bazel coverage --test_output=all //:hello_test &>$TEST_log || fail "Coverage for //:hello_test failed" | ||
local coverage_file_path | ||
coverage_file_path="$( get_coverage_file_path_from_test_log )" | ||
diff expected.dat "$coverage_file_path" >> $TEST_log | ||
cmp expected.dat "$coverage_file_path" || fail "Coverage output file is different than the expected file for py_library." | ||
} | ||
|
||
run_suite "test tests" |