Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add pipeline for building Debian packages in CircleCI #44

Merged
merged 3 commits into from
Jun 17, 2019
Merged

Add pipeline for building Debian packages in CircleCI #44

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
d72cca2
ci: add pipeline for building workstation deb packages
redshiftzero May 15, 2019
5fc689b
ignore sort order of sha256sums.txt files
redshiftzero May 18, 2019
d92e18f
README: add ci status badge for master
redshiftzero May 19, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
92 changes: 92 additions & 0 deletions .circleci/config.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
---
common-steps:
- &installdeps
run:
name: Install Debian packaging dependencies
command: make install-deps

- &fetchwheels
run:
name: Download wheels and sources
command: make fetch-wheels
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hurray reusable steps!


- &makesourcetarball
run:
name: Get latest tag for the project and make a source tarball
command: |
cd ~/packaging/securedrop-*
export LATEST_TAG="$(git describe --tags $(git rev-list --tags --max-count=1))"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice solution on the tag lookup, I'd have gone with git tag | sort -nr | head -n1, but this looks more durable. 👍

# Enable access to this env var in subsequent run steps
echo $LATEST_TAG > ~/packaging/sd_version
echo 'export LATEST_TAG=$(cat ~/packaging/sd_version)' >> $BASH_ENV
# Create tarball
git checkout $LATEST_TAG
python3 setup.py sdist
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Crafty use of a fileglob to cd into the target dir without specifying the name. Given that the consolidated build logic operates the same way on all packages, does it make sense to condense these steps into a single wrapper, e.g. build-package? The only arg we need is the package name, e.g. securedrop-client.

The major advantage I see to such a wrapper is that it'd be easy to run locally in case we need to debug CI. Also, until we have full CD on these packages, providing guardrails wherever possible on the local dev story seems like it'll save time and minimize mistakes.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Even if you agree it's worth wrapping further, let's definitely keep the separate "jobs" for each package: doing so greatly clarifies the CI results list on PRs, so it'll be obvious if a PR breaks on a specific package.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just flagging that there is a makefile target that encapsulates the deb packaging logic in this repo and takes the package name: make securedrop-client, make securedrop-proxy which is called in the subsequent run step

the step you're commenting on here is preparing the release tarball and is something that is ran in the repo to be packaged... while i could add a makefile target for this (or subsume this into the existing makefile target, and have it take the directory where python3 setup.py sdist should be run), it seems like the wins are pretty small. the other steps above are just 1. getting in the right dir on the CI machine and 2. getting the latest tag and making it available across run steps


version: 2.1
jobs:
build-securedrop-client:
docker:
- image: circleci/python:3.5-stretch
steps:
- checkout
- *installdeps
- *fetchwheels

- run:
name: Clone the repository to be packaged
command: |
mkdir ~/packaging && cd ~/packaging
git clone https://github.com/freedomofpress/securedrop-client.git

- *makesourcetarball

- run:
name: Build securedrop-client debian package
command: |
export PKG_PATH=~/packaging/securedrop-client/dist/securedrop-client-$LATEST_TAG.tar.gz
export PKG_VERSION=$LATEST_TAG
make securedrop-client
ls ~/debbuild/packaging/*.deb

build-securedrop-proxy:
docker:
- image: circleci/python:3.5-stretch
steps:
- checkout
- *installdeps
- *fetchwheels

- run:
name: Clone the repository to be packaged
command: |
mkdir ~/packaging && cd ~/packaging
git clone https://github.com/freedomofpress/securedrop-proxy.git

- *makesourcetarball

- run:
name: Build securedrop-proxy debian package
command: |
export PKG_PATH=~/packaging/securedrop-proxy/dist/securedrop-proxy-$LATEST_TAG.tar.gz
export PKG_VERSION=$LATEST_TAG
make securedrop-proxy
ls ~/debbuild/packaging/*.deb

workflows:
build-debian-packages:
jobs:
- build-securedrop-client
- build-securedrop-proxy

nightly:
triggers:
- schedule:
cron: "0 5 * * *"
filters:
branches:
only:
- master
jobs:
- build-securedrop-client
- build-securedrop-proxy
2 changes: 2 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
# SecureDrop Debian Packaging

[![CircleCI](https://circleci.com/gh/freedomofpress/securedrop-debian-packaging/tree/master.svg?style=svg)](https://circleci.com/gh/freedomofpress/securedrop-debian-packaging/tree/master)

This repository contains the packaging files and tooling for building Debian packages for projects for the alpha [SecureDrop Workstation](https://github.com/freedomofpress/securedrop-workstation) based on Qubes OS. Packages are placed on `apt-test-qubes.freedom.press` for installation in Debian-based TemplateVMs. These packages are not yet ready for use in a production environment.

## Packaging a Python-based SecureDrop project
Expand Down
2 changes: 1 addition & 1 deletion scripts/verify-sha256sum-signature
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ function verify_sha256sum_no_changes() {
fi
cd "${repo_root}/localwheels"
sha256sum * > "$temp_sha256sum"
diff "$sha256sums_file" "$temp_sha256sum"
diff <(sort "$sha256sums_file") <(sort "$temp_sha256sum")
}

verify_sha256sum_signature
Expand Down