Skip to content

Commit

Permalink
feat(pip_repository): Enable PyPi dep cycles
Browse files Browse the repository at this point in the history
This patch adjusts the pip_repository interface to accept a new
parameter: `composite_libs`, being a list of PyPi package names which
form a cycle and must be installed together.

The intuition behind this design is that a dependency cycle {a <-> b}
is implemented simply as emplacing both a and b at once. Hence a
dependency graph {c -> a, c -> b} has the same effect.

If we modify the installation of a and b to remove their mutual
dependency, and generate a c which dominates a and b, we can then modify
the `requirement()` and `whl_requirement()` helper functions to
recognize the requirements a and b and provide a reference to c instead.
  • Loading branch information
arrdem committed May 18, 2023
1 parent 16126d0 commit 3a3b0f5
Show file tree
Hide file tree
Showing 18 changed files with 448 additions and 64 deletions.
4 changes: 2 additions & 2 deletions .bazelrc
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@
# This lets us glob() up all the files inside the examples to make them inputs to tests
# (Note, we cannot use `common --deleted_packages` because the bazel version command doesn't support it)
# To update these lines, run tools/bazel_integration_test/update_deleted_packages.sh
build --deleted_packages=examples/build_file_generation,examples/build_file_generation/get_url,examples/bzlmod_build_file_generation,examples/bzlmod_build_file_generation/other_module/other_module/pkg,examples/bzlmod_build_file_generation/runfiles,examples/bzlmod,examples/bzlmod/other_module/other_module/pkg,examples/bzlmod/runfiles,examples/multi_python_versions,examples/multi_python_versions/libs/my_lib,examples/multi_python_versions/requirements,examples/multi_python_versions/tests,examples/pip_install,examples/pip_parse,examples/pip_parse_vendored,examples/pip_repository_annotations,examples/py_import,examples/py_proto_library,examples/relative_requirements,tests/compile_pip_requirements,tests/pip_repository_entry_points,tests/pip_deps
query --deleted_packages=examples/build_file_generation,examples/build_file_generation/get_url,examples/bzlmod_build_file_generation,examples/bzlmod_build_file_generation/other_module/other_module/pkg,examples/bzlmod_build_file_generation/runfiles,examples/bzlmod,examples/bzlmod/other_module/other_module/pkg,examples/bzlmod/runfiles,examples/multi_python_versions,examples/multi_python_versions/libs/my_lib,examples/multi_python_versions/requirements,examples/multi_python_versions/tests,examples/pip_install,examples/pip_parse,examples/pip_parse_vendored,examples/pip_repository_annotations,examples/py_import,examples/py_proto_library,examples/relative_requirements,tests/compile_pip_requirements,tests/pip_repository_entry_points,tests/pip_deps
build --deleted_packages=examples/build_file_generation,examples/build_file_generation/random_number_generator,examples/bzlmod,examples/bzlmod/other_module/other_module/pkg,examples/bzlmod/runfiles,examples/bzlmod_build_file_generation,examples/bzlmod_build_file_generation/other_module/other_module/pkg,examples/bzlmod_build_file_generation/runfiles,examples/multi_python_versions/libs/my_lib,examples/multi_python_versions/requirements,examples/multi_python_versions/tests,examples/pip_install,examples/pip_parse,examples/pip_parse_vendored,examples/pip_repository_annotations,examples/py_proto_library,tests/compile_pip_requirements,tests/compile_pip_requirements_test_from_external_workspace,tests/ignore_root_user_error,tests/pip_repository_entry_points
query --deleted_packages=examples/build_file_generation,examples/build_file_generation/random_number_generator,examples/bzlmod,examples/bzlmod/other_module/other_module/pkg,examples/bzlmod/runfiles,examples/bzlmod_build_file_generation,examples/bzlmod_build_file_generation/other_module/other_module/pkg,examples/bzlmod_build_file_generation/runfiles,examples/multi_python_versions/libs/my_lib,examples/multi_python_versions/requirements,examples/multi_python_versions/tests,examples/pip_install,examples/pip_parse,examples/pip_parse_vendored,examples/pip_repository_annotations,examples/py_proto_library,tests/compile_pip_requirements,tests/compile_pip_requirements_test_from_external_workspace,tests/ignore_root_user_error,tests/pip_repository_entry_points

test --test_output=errors

Expand Down
8 changes: 5 additions & 3 deletions docs/pip_repository.md

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

18 changes: 16 additions & 2 deletions examples/pip_parse_vendored/BUILD.bazel
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
load("@bazel_skylib//rules:diff_test.bzl", "diff_test")
load("@bazel_skylib//rules:write_file.bzl", "write_file")
load("@rules_python//python:defs.bzl", "py_test")
load("@rules_python//python:pip.bzl", "compile_pip_requirements")
load("//:requirements.bzl", "requirement")

# This rule adds a convenient way to update the requirements.txt
# lockfile based on the requirements.in.
Expand All @@ -20,8 +22,10 @@ genrule(
# Replace the bazel 6.0.0 specific comment with something that bazel 5.4.0 would produce.
# This enables this example to be run as a test under bazel 5.4.0.
"""sed -e 's#@//#//#'""",
"""sed 's#"@python39_.*//:bin/python3"#interpreter#' >$@""",
]),
"""sed 's#"@python39_.*//:bin/python3"#interpreter#'""",
# Stick some buildifiler disables in as needed
"""sed 's/^def /# buildifier: disable=function-docstring\\\ndef /g'""",
]) + " >$@",
)

write_file(
Expand Down Expand Up @@ -50,3 +54,13 @@ diff_test(
file1 = "requirements.bzl",
file2 = ":make_platform_agnostic",
)

py_test(
name = "test",
srcs = ["test.py"],
deps = [
requirement("oletools"),
requirement("pcodedmp"),
requirement("requests"),
],
)
6 changes: 6 additions & 0 deletions examples/pip_parse_vendored/WORKSPACE
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,12 @@ load("@rules_python//python:pip.bzl", "pip_parse")
# It also wouldn't be needed by users of this ruleset.
pip_parse(
name = "pip",
composite_libs = {
"vbap": [
"oletools",
"pcodedmp",
],
},
python_interpreter_target = interpreter,
requirements_lock = "//:requirements.txt",
)
Expand Down
Loading

0 comments on commit 3a3b0f5

Please sign in to comment.