Skip to content
This repository has been archived by the owner on Aug 13, 2019. It is now read-only.

Commit

Permalink
Make release script (#422)
Browse files Browse the repository at this point in the history
* remove unnecessary paragraph

* make-release script

* deployment-bug.py

* utiloity function check_output

* review fixes
  • Loading branch information
Peter Bengtsson authored Apr 24, 2018
1 parent af9da69 commit 72b919a
Show file tree
Hide file tree
Showing 4 changed files with 485 additions and 25 deletions.
69 changes: 47 additions & 22 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,9 +13,9 @@ _Buildhub_ aims to provide a public database of comprehensive information about

## Development

1. Install Docker
2. To run tests: `make test`
3. To lint check Python code: `make lintcheck`
1. Install Docker
2. To run tests: `make test`
3. To lint check Python code: `make lintcheck`

## Continuous Integration

Expand All @@ -24,22 +24,47 @@ for all continous integration.

## Releasing

We don't use `zest.releaser` right now because of some problems with
releasing a package that is not at the root of the repo (`jobs/`), and
because we have no interest in uploading this project to PyPI, but
this could change if we figure out how.

The current procedure is:

* Bump version in `jobs/setup.py`
* Update the release date in `jobs/CHANGELOG.rst`
* `git commit -am "Bump x.y.z"`
* Open PR, wait for it to become green
* Merge PR
* `git tag x.y.z`
* `git push --tags origin`
* `make lambda.zip`
* Add a release on Github with the lambda.zip attached
* [Click here][bugzilla-link] to open a ticket to get it deployed

[bugzilla-link]: https://bugzilla.mozilla.org/enter_bug.cgi?comment=Could%20you%20please%20update%20the%20lambda%20function%20for%20Buildhub%20with%20the%20following%20one%3F%0D%0A%0D%0A%5BInsert%20a%20short%20description%20of%20the%20changes%20here.%5D%0D%0A%0D%0Ahttps%3A%2F%2Fgithub.com%2Fmozilla-services%2Fbuildhub%2Freleases%2Ftag%2FX.Y.Z%0D%0A%0D%0Ahttps%3A%2F%2Fgithub.com%2Fmozilla-services%2Fbuildhub%2Freleases%2Fdownload%2FX.Y.Z%2Fbuildhub-lambda-X.Y.Z.zip%0D%0A%0D%0AThanks%21&component=Operations%3A%20Storage&product=Cloud%20Services&qa_contact=chartjes%40mozilla.com&short_desc=Please%20deploy%20buildhub%20lambda%20function%20X.Y.Z
To make a release you need to have write access to
`github.com/mozilla-services/buildhub`. First you have to generate a
new `lambda.zip` file by running:

make lambda.zip

(This is generated inside Docker).

Then you need a [GitHub Personal Access Token](https://github.com/settings/tokens)
with `repos` scope. This is to generate GitHub Releases and upload assets
to them. Next, run `./bin/make-release.py`. The only required parameter
is the "type" of the release. The choices are:

* `major` (e.g. '2.6.9' to '3.0.0')
* `minor` (e.g. '2.6.7' to '2.7.0')
* `patch` (e.g. '2.6.7' to '2.6.8')

Like this for example:

GITHUB_API_KEY=895f...ce09 ./bin/make-release.py minor

This will bump the version in `setup.py`, update the `CHANGELOG.rst` and
make a tag and push that tag to GitHub.

Then, it will create a Release and upload the latest `lambda.zip` as an
attachment to that Release.

## Deployment

The outlined steps above only upgrade the cron job part of Buildhub.
And only for Stage is it automatically upgraded simply by making a new
Release.

At the time of writing we still need to file a Bugzilla bug to have
the Lambda job upgraded on Stage. [Issue #423](https://github.com/mozilla-services/buildhub/issues/423)
is about automating this away.

To upgrade the Lambda job on **Stage** run:

./bin/deployment-bug.py stage-lambda

To upgrade the cron job _and_ Lambda job on **Prod** run:

./bin/deployment-bug.py prod
131 changes: 131 additions & 0 deletions bin/deployment-bug.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,131 @@
#!/usr/bin/env python
# This Source Code Form is subject to the terms of the Mozilla Public
# License, v. 2.0. If a copy of the MPL was not distributed with this
# file, you can obtain one at http://mozilla.org/MPL/2.0/.

from urllib.parse import urlencode

import requests


OWNER = 'mozilla-services'
REPO = 'buildhub'

VALID_ENVIRONMENTS = ('stage', 'prod')
VALID_TASKS = ('cron', 'lambda', 'both')

QA_CONTACT = '[email protected]'


def main(
environment,
task,
tag=None,
dry_run=False,
):
api_url = f'https://api.github.com/repos/{OWNER}/{REPO}/releases'
if not tag:
api_url += '/latest'
else:
api_url += f'/tags/{tag}'
response = requests.get(api_url)
response.raise_for_status()
release_info = response.json()

# Prepare some variables for the string templates
release_url = release_info['html_url']
lambda_asset_url = None
for asset in release_info['assets']:
lambda_asset_url = asset['browser_download_url']
env = environment.title()
release_tag_name = release_info['tag_name']

if task == 'lambda':
summary = f"On {env}, please deploy Buildhub Lambda function {release_tag_name}" # noqa
comment = f"""
Could you please update the Lambda function for Buildhub {env} with the following one?
{release_url}
{lambda_asset_url}
Thanks!
""" # noqa
elif task == 'cron':
summary = f"On {env}, please deploy Buildhub Cron function {release_tag_name}" # noqa
comment = f"""
Could you please update the Cron function for Buildhub {env} with the following one?
{release_url}
Thanks!
""" # noqa
else:
summary = f"On {env}, please deploy Buildhub Cron and Lambda function {release_tag_name}" # noqa
comment = f"""
Could you please update the Cron *and* Lambda function for Buildhub {env} with the following one?
{release_url}
{lambda_asset_url}
Thanks!
""" # noqa

comment = '\n'.join(x.strip() for x in comment.strip().splitlines())
params = {
'qa_contact': QA_CONTACT,
'comment': comment,
'short_desc': summary,
'component': 'Operations: Storage',
'product': 'Cloud Services',
'bug_file_loc': release_url,
}
URL = 'https://bugzilla.mozilla.org/enter_bug.cgi?' + urlencode(params)
print('To file this bug, click (or copy) this URL:')
print('👇')
print(URL)
print('👆')
return 0


if __name__ == '__main__':
import argparse

def check_environment(value):
value = value.strip()
if value not in VALID_ENVIRONMENTS:
raise argparse.ArgumentTypeError(
f'{value!r} not in {VALID_ENVIRONMENTS}'
)
return value

def check_task(value):
value = value.strip()
if value not in VALID_TASKS:
raise argparse.ArgumentTypeError(
f'{value!r} not in {VALID_TASKS}'
)
return value

parser = argparse.ArgumentParser(
description='Deploy Buildhub (by filing Bugzilla bugs)!'
)
parser.add_argument(
'-t', '--tag', type=str,
help=(
f'Name of the release (e.g. "v1.2.0"). If ommitted will be looked '
f'on GitHub at https://github.com/{OWNER}/{REPO}/releases'
)
)

parser.add_argument(
'environment', help='stage or prod', type=check_environment
)
parser.add_argument(
'task', help='cron or lambda or both', type=check_task,
)
parser.add_argument('-d', '--dry-run', action='store_true')
args = parser.parse_args()
args = vars(args)
main(args.pop('environment'), args.pop('task'), **args)
Loading

0 comments on commit 72b919a

Please sign in to comment.