Merge pull request #422 from freedomofpress/update-pypi-on-pr

update pip mirror update policy, build deb package on each PR to verify policy is adhered to

.circleci/config.yml

@@ -1,6 +1,35 @@
 version: 2
 jobs:
   build:
+    docker:
+      - image: circleci/python:3.5-stretch
+    steps:
+      - checkout
+
+      - run:
+          name: Install Debian packaging dependencies and download wheels
+          command: |
+            mkdir ~/packaging && cd ~/packaging
+            git clone https://github.com/freedomofpress/securedrop-debian-packaging.git
+            cd securedrop-debian-packaging
+            make install-deps && make fetch-wheels
+
+      - run:
+          name: Tag and make source tarball
+          command: |
+            cd ~/project
+            ./update_version.sh 1000.0  # Dummy version number, doesn't matter what we put here
+            python3 setup.py sdist
+
+      - run:
+          name: Build debian package
+          command: |
+            cd ~/packaging/securedrop-debian-packaging
+            export PKG_VERSION=1000.0
+            export PKG_PATH=~/project/dist/securedrop-client-$PKG_VERSION.tar.gz
+            make securedrop-client
+
+  test:
     docker:
       - image: circleci/python:3.5
     steps:
@@ -31,3 +60,10 @@ jobs:
             set -e
             source .venv/bin/activate
             make bandit
+
+workflows:
+  version: 2
+  securedrop_client_ci:
+    jobs:
+      - test
+      - build

README.md

@@ -52,9 +52,16 @@ pip install --require-hashes -r dev-requirements.txt
 
 ## Updating dependencies
 
-To add or update a dependency, modify either `dev-requirements.in` and `requirements.in` and then run `make update-pip-dependencies`. This will generate `dev-requirements.txt` and `requirements.txt`.
+If you're adding or updating a dependency, you need to:
 
-**IMPORTANT:** Do not modify `build-requirements.txt` during normal development. We use a pip mirror for our build process and the hashes in that file point to wheels on our mirror.
+1. Modify either `dev-requirements.in` and `requirements.in` (depending on whether it is prod or dev only) and then run `make update-pip-dependencies`. This will generate `dev-requirements.txt` and `requirements.txt`.
+
+2. For building a debian package from this project, we use the requirements in
+`build-requirements.txt` which uses our pip mirror, i.e. the hashes in that file point to
+wheels on our pip mirror. A maintainer will need to add
+the updated dependency to our pip mirror (you can request this in the PR).
+
+3. Once the pip mirror is updated, you should checkout the [securedrop-debian-packaging repo](https://github.com/freedomofpress/securedrop-debian-packaging) and run `make requirements`. Commit the `build-requirements.txt` that results and add it to your PR.
 
 ## Run the client
 
@@ -116,6 +123,16 @@ but developers should merge their migration into the latest migration that has b
 release. The above mentioned autogenerate command will not do this for you.
 
 
+## Making a Release
+
+**Note:** These are the release guidelines for pre-production alpha releases. Production release tags must
+be signed with the SecureDrop release key.
+
+1. Update versions: `./update_version.sh $new_version_number`.
+2. Commit the changes with commit message `securedrop-client $new_version_number` and make a PR.
+3. You should confirm via a manual debian package build and manual testing in Qubes that there are no regressions (this is limited pre-release QA).
+4. Once your PR is approved, you can add a tag and push: `git tag $new_version_number`.
+
 ## Qubes Debugging
 
 Using a version of this application installed from a deb package in Qubes,

update_version.sh

@@ -0,0 +1,31 @@
+#!/bin/bash
+## Usage: ./update_version.sh <version>
+
+set -e
+
+readonly NEW_VERSION=$1
+
+if [ -z "$NEW_VERSION" ]; then
+  echo "You must specify the new version!"
+  exit 1
+fi
+
+# Get the old version from securedrop_client/__init__.py
+old_version_regex="^__version__ = '(.*)'$"
+[[ "$(cat securedrop_client/__init__.py)" =~ $old_version_regex ]]
+OLD_VERSION=${BASH_REMATCH[1]}
+
+if [ -z "$OLD_VERSION" ]; then
+  echo "Couldn't find the old version: does this script need to be updated?"
+  exit 1
+fi
+
+# Update the version in securedrop_client/__init__.py and setup.py
+if [[ "$OSTYPE" == "darwin"* ]]; then
+    # The empty '' after sed -i is required on macOS to indicate no backup file should be saved.
+    sed -i '' "s@$(echo "${OLD_VERSION}" | sed 's/\./\\./g')@$NEW_VERSION@g" securedrop_client/__init__.py
+    sed -i '' "s@$(echo "${OLD_VERSION}" | sed 's/\./\\./g')@$NEW_VERSION@g" setup.py
+else
+    sed -i "s@$(echo "${OLD_VERSION}" | sed 's/\./\\./g')@$NEW_VERSION@g" securedrop_client/__init__.py
+    sed -i "s@$(echo "${OLD_VERSION}" | sed 's/\./\\./g')@$NEW_VERSION@g" setup.py
+fi