Skip to content

Commit

Permalink
Revert "use the currently checked out version of the repo, don't clone (
Browse files Browse the repository at this point in the history
#247)"

This reverts commit 96fb728.
  • Loading branch information
peisun1115 committed Mar 5, 2021
1 parent 6e8a7a0 commit c0f5625
Show file tree
Hide file tree
Showing 4 changed files with 19 additions and 11 deletions.
1 change: 0 additions & 1 deletion .dockerignore

This file was deleted.

11 changes: 6 additions & 5 deletions pip_pkg_scripts/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,19 +2,20 @@

## Build a pip package locally using the docker container

```bash
```
docker build --tag=open_dataset_pip -f pip_pkg_scripts/build.Dockerfile .
mkdir /tmp/artifacts
docker run --mount type=bind,source=/tmp/artifacts,target=/tmp/artifacts -e "PYTHON_VERSION=3" -e "PYTHON_MINOR_VERSION=8" -e "PIP_MANYLINUX2010=1" -e "TF_VERSION=2.3.0" open_dataset_pip
mkdir /tmp/pip_pkg_build
docker run --mount type=bind,source=/tmp/pip_pkg_build,target=/tmp/pip_pkg_build -e "GITHUB_BRANCH=master" -e "PYTHON_VERSION=3" -e "PYTHON_MINOR_VERSION=8" -e "PIP_MANYLINUX2010=1" -e "TF_VERSION=2.3.0" open_dataset_pip
```
This command will execute the `build.sh` inside the container, which clones the
github repository, builds the library and outputs `.whl` packages under
`/tmp/artifacts/`
`/tmp/pip_pkg_build/`

## Build a pip package locally without docker container
First follow quick start to install all the depdencies such as bazel. Then run

```bash
```
export GITHUB_BRANCH=master
export PYTHON_VERSION=3
export PYTHON_MINOR_VERSION=7
export PIP_MANYLINUX2010=0
Expand Down
9 changes: 5 additions & 4 deletions pip_pkg_scripts/build.Dockerfile
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
FROM tensorflow/tensorflow:custom-op-ubuntu16

ENV GITHUB_BRANCH="master"
ENV PYTHON_VERSION="3"
ENV PYTHON_MINOR_VERSION=""
ENV PIP_MANYLINUX2010="1"
Expand All @@ -25,11 +26,11 @@ RUN for python in python3.5 python3.6 python3.7 python3.8; do \
$python -m pip install --upgrade grpcio>=1.24.3; \
done

VOLUME /tmp/artifacts
COPY . /tmp/repo
WORKDIR /tmp/repo
COPY pip_pkg_scripts/build.sh /

ENTRYPOINT ["pip_pkg_scripts/build.sh"]
VOLUME /tmp/pip_pkg_build

ENTRYPOINT ["/build.sh"]

# The default parameters for the build.sh
CMD []
9 changes: 8 additions & 1 deletion pip_pkg_scripts/build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@
set -e -x

# Override the following env variables if necessary.
export GITHUB_BRANCH="${GITHUB_BRANCH:-master}"
export PYTHON_VERSION="${PYTHON_VERSION:-3}"
export PYTHON_MINOR_VERSION="${PYTHON_MINOR_VERSION}"
export PIP_MANYLINUX2010="${PIP_MANYLINUX2010:-0}"
Expand All @@ -45,13 +46,19 @@ ${PIP} install tensorflow-gpu=="${TF_VERSION}" --user
# bazel.
python3 -m pip install --upgrade tensorflow-gpu=="${TF_VERSION}" --user

rm -rf waymo-od || true
git clone https://github.com/waymo-research/waymo-open-dataset.git waymo-od
cd waymo-od

git checkout remotes/origin/${GITHUB_BRANCH}

./configure.sh

bazel clean
bazel build ...
bazel test ...

DST_DIR="/tmp/artifacts"
DST_DIR="/tmp/pip_pkg_build"
rm -rf "$DST_DIR" || true
./pip_pkg_scripts/build_pip_pkg.sh "$DST_DIR" ${PYTHON_VERSION}
# Comment the following line if you run this outside of the container.
Expand Down

0 comments on commit c0f5625

Please sign in to comment.