Poetry is a package manager for Python. Bazel is a build system. Together they form a powerful, (mostly) hermetic build system for Python.
- Poetry 1.0 or higher
- Bazel 1.0 or higher
The rules depend on Poetry's current lock file format. Versions 0.12.x or lower have a different format that is no longer supported.
Add these lines to your project's WORKSPACE
file:
# Poetry rules for managing Python dependencies
http_archive(
name = "com_sonia_rules_poetry",
sha256 = "8a7a6a5d2ef859ba4309929f3b4d61031f2a4bfed6f450f04ab09443246a4b5c",
strip_prefix = "rules_poetry-ecd0d9c66b89403667304b11da3bd99764797a63",
urls = ["https://github.com/soniaai/rules_poetry/archive/ecd0d9c66b89403667304b11da3bd99764797a63.tar.gz"],
)
load("@com_sonia_rules_poetry//rules_poetry:defs.bzl", "poetry_deps")
poetry_deps()
load("@com_sonia_rules_poetry//rules_poetry:poetry.bzl", "poetry")
poetry(
name = "poetry",
lockfile = "//:poetry.lock",
pyproject = "//:pyproject.toml",
# optional, if you would like to pull from pip instead of a Bazel cache
tags = ["no-remote-cache"],
)
and use it to transitively pull in package dependencies using the generated dependency
macro:
load("@poetry//:dependencies.bzl", "dependency")
load("@rules_python//python:defs.bzl", "py_library")
py_library(
name = "some_package",
srcs = ["example.py"],
deps = [
dependency("pytorch"),
dependency("tensorflow"),
],
)
Usage is nearly identical to Poetry without Bazel.
To add a dependency just run poetry add <package name>
.
To remove a package run poetry remove <package name>
.
- May not work in network isolated sandboxes (untested)
- Cache support is best-effort
- Source wheels are built on demand and may not always produce deterministic results
As you're probably aware, there are a number of Python rules for Bazel due to rules_python
being insufficient for
many real world applications. While there is significant overlap between these rules, they're not identical in
implementation nor in capability. A feature summary is listed below:
Feature | rules_poetry | rules_python |
---|---|---|
Backend | poetry | pip |
Incremental | yes | no |
Transitive | yes | no |
Isolation | yes | no |
Hash Validation | yes | depends |
Deterministic | yes | yes |
Crosstool | no | no |
- Backend: which program is used for package management.
- Incremental: adding, upgrading or removing packages only affects those that are modified. Unchanged packages are not reinstalled or downloaded.
- Transitive: depending on one package will include all other packages it indirectly depends on.
- Isolation: packages in the system or user site-packages should not be present in Python's path.
- Hash Validation: wheel and source hashes are validated before installation
- Deterministic: builds should be repeatable and cacheable
- Crosstool: cross compilation (target != execution platform)
- Investiage using
http_file
instead ofpip download
to fetch dependencies - Expose the Poetry binary as a Bazel py_binary
- Generate wrappers named
python3
python3.7
etc - Add the wrappers to the
PATH
so the interpreter entrypoint is consistent - Unpack wheels directly into container layers to improve cacheability
- Improve documentation
- Write more tests