From 186bade4f054c0a9be7037585584c9c572cba483 Mon Sep 17 00:00:00 2001 From: Alex Thompson Date: Sat, 27 Jul 2019 15:22:17 -0400 Subject: [PATCH] Updated README for new rules Resolves #31 --- README.md | 213 ++++++++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 208 insertions(+), 5 deletions(-) diff --git a/README.md b/README.md index 2f9c658..d03e19b 100644 --- a/README.md +++ b/README.md @@ -3,16 +3,20 @@ [![Build Status](https://travis-ci.org/apt-itude/rules_pip.svg?branch=master)](https://travis-ci.org/apt-itude/rules_pip) ## Overview -This repository provides rules for the [Bazel build tool](https://www.bazel.build/) that allow your Python code to depend on pip packages using a standard [requirements file](https://pip.pypa.io/en/stable/user_guide/#requirements-files). It is built in pure Python and uses the `pip` and `wheel` libraries to ensure that the resulting dependency set is the same as it would be by using those tools. +This repository provides rules for the [Bazel build tool](https://www.bazel.build/) that allow your Python code to depend on pip packages using a standard [requirements file](https://pip.pypa.io/en/stable/user_guide/#requirements-files). It utilizes Starlark wherever possible to provide hermetic rules, but uses Python and the `pip` library for dependency resolution and management in order to ensure that the development experience is consistent with that of `pip`. -This repository is designed to be compatible with both Python 2 and 3 in a single repo, as well as support multiple platforms. +This repository is designed to be support the use of both Python 2 and Python 3 in a single repo, execution on multiple platforms, and remote execution. + +## Getting Started + +### Importing the rules -## Setup Add the following to your `WORKSPACE` file: -``` + +```python git_repository( name = "com_apt_itude_rules_pip", - commit = "e5ed5e72bf5a7521244e1d2119821628bbf17263", + commit = "82d3393646982b233c7ed919a5b66ffb471dc649", remote = "https://github.com/apt-itude/rules_pip.git", ) @@ -20,3 +24,202 @@ load("@com_apt_itude_rules_pip//rules:dependencies.bzl", "pip_rules_dependencies pip_rules_dependencies() ``` + +### Defining your requirements + +All direct external pip dependencies should be defined in a `requirements.txt` file. This file should be located in a Bazel package (typically something like `thirdparty/pip`) and committed to source control. + +The same requirements file may be used to define all pip dependencies accross your workspace and shared among Python versions and platforms. To define requirements that only pertain to a particular Python version or platform, you should use [Environment Markers](https://www.python.org/dev/peps/pep-0508/#environment-markers). + +_NOTE_: At this time, only the `sys_platform` and `python_version` environment markers are supported. Using additional markers will not cause an error, but the `pip_lock` rule only generates one environment per major Python version per platform, so additional granularity in the `requirements.txt` file will be ignored. + +### Locking your requirements + +Instantiate the `pip_lock` rule in a `BUILD` file, typically within the same package as the `requirements.txt` file. For example: + +`thirdparty/pip/BUILD` +```python +load("@com_apt_itude_rules_pip//rules:lock.bzl", "pip_lock") + +pip_lock( + name = "lock", + requirements = ["requirements.txt"], +) +``` + +Next, execute the `pip_lock` binary: + +```bash +bazel run //thirdparty/pip:lock +``` + +This will do two things: +1. Generate a `requirements-lock.json` file alongside the `requirements.txt` file, which locks all direct and transitive dependencies to a specific version +1. Build wheels for any requirement that is not already distributed as a wheel on PyPI and store them in a `wheels` directory alongside the `requirements.txt` file + +The `requirements-lock.json` file should be committed to source control. + +The wheel files may either be source-controlled or published to a custom Python package index. See [Managing built wheel files](#managing-built-wheel-files) for more information. + +### Turning the requirements into Bazel dependencies + +Add the following to your `WORKSPACE` file: + +```python +load("@com_apt_itude_rules_pip//rules:repository.bzl", "pip_repository") + +pip_repository( + name = "pip", + requirements = "//thirdparty/pip:requirements-lock.json", +) + +load("@pip//:requirements.bzl", "pip_install") + +pip_install() +``` + +This creates an external workspace named `@pip` from which you can access all pip requirements. A `py_library` target is exposed for each requirement via the label `@pip//`, where `` is the canonical name of the Python distribution found in your `requirements.txt` file. The canonical name is all lowercase, with hyphens replaced by underscores. For example, `PyYAML` would become `@pip//pyyaml` and `pytest-mock` would become `@pip//pytest_mock`. + +### Using pip dependencies + +Simply add `@pip//` labels to the `deps` list of any `py_library`, `py_binary`, or `py_test` rule. For example: + +`some/package/BUILD` +```python +py_library( + name = "dopecode" + srcs = ["dopecode.py"], + deps = [ + "@pip//pytest_mock", + "@pip//pyyaml", + ] +) +``` + +When you build these targets, only the pip distributions that are directly or indirectly required by that target will be fetched. + +## Updating requirements + +To update all requirements simultaneously: +```bash +bazel run