How to release

If you are releasing a version of Pravega, then you must read and follow the instructions. The steps here are based on the experience we are building across releases, so if you see any point that requires changes or more detail, feel free to raise it or modify the document.

Before you Begin

Before you begin, take a moment to ensure that you have the following:

• Pravega GPG signing key
• Pravega GPG signing key password
• Dockerhub username/password with access to the Pravega repository
• Sonatype username/password with access to the io.pravega group

Also, note the following:

• Read the instructions carefully. Also, read ahead a few steps to avoid errors.
• Some of the instructions (like the Checksum step) below assume that you are doing the release on a particular environment like Mac. You may have to customize the commands if you are releasing from a different environment.
• Some of the preparatory work for the release begins days in advance. For example, you will likely need to prepare the release notes in advance.
• The entire process from beginning to end may span more than a day. For example, you might cut a release a branch (say, r0.8) on one day and release from that branch another day. Similarly, before you publish a release in Maven Central it is staged there and manually verified for the release, which can take more than a day.

Release process overview

The release process requires a release to be done from a corresponding release branch and could be outlined in the following steps:

1. Prepare release branch
2. Keep cherry-picking fixes to the release branch until satisfied with its quality
3. Make a Release Candidate tag
4. Publish the Release Candidate out
5. If the Release Candidate is voted for then proceed with releasing it

Note that the above steps assume that a release branch could be open for quite some time and steps 2-4 could be taken multiple times as required.

Preparing the branch

Preparing the branch consists of making necessary changes to the branch you will be working on as part of releasing. There are two possible situations:

• Feature release or non-backward-compatible release: This is a change to either the first or the middle digit, and it requires a new release branch.
• Bug-fix release: This is a minor release version over an existing release branch

In both cases the final deliverables must be produced from a release branch which must be prepared accordingly.

It's quite common to have release branch cut a bit in advance so master branch is open for further development while the current release is getting stabilized/tested before it enters RC phase.

Major release

A major release changes either the middle or the most significant digit. In this case, you do need to create a new branch. Release branches follow rX.Y schema, e.g. r0.3

The process is the following, assuming for the sake of example that the new release is 0.3.0:

  > git checkout master
  > git tag branch-0.3
  > git push upstream branch-0.3
  > git checkout -b r0.3
  > git push upstream r0.3

Once the steps above are done, we need to make version changes to master:

• In master, create an issue and corresponding pull request to change the pravegaVersion in gradle.properties to 0.4.0-SNAPSHOT. Note that we are bumping up the middle digit because, in our example, we are releasing 0.3.0. If we were releasing say 1.0.0, then we would change pravegaVersion to 1.1.0-SNAPSHOT.
• Ensure that an automated build of master works fine and the artifacts of the new version are published in the snapshot repository

No version changes in the release branch needed as it carried out the version from master

Important: Ask the admin to prevent direct pushes of commits to the new branch except by the member performing the release and one or two other committers. Changes to the branch will then require an approved pull request.

Bug-fix release

If this is a bug-fix release, then there is no need to create a new branch. First identify a release branch you'll be working on. It will be named rX.Y, e.g., r0.2.

No additional preparation is needed in this case as the release branch should have the correction version set already. Just proceed with the first RC builds whenever ready

Tagging a release candidate

To produce a Release Candidate build from the release branch a corresponding git tag must be created:

1. Create a temp branch of current release branch for RC tagging. Branch name should follow the following schema: rX.Y.Z-rcN where X.Y.Z is Pravega version and N is a release candidate number. For example, r0.3.1-rc0 is for the first RC of 0.3.1 release.

      > git checkout r0.3
      > git checkout -b r0.3.1-rc1
   

2. Remove -SNAPSHOT suffix from pravegaVersion field in gradle.properties file and commit the change locally
3. Tag the latest commit as vX.Y.Z-rcN and push it to upstream repository.

      > git tag v0.3.1-rc1
      > git push upstream v0.3.1-rc1
   

4. Delete the branch created at step 1.

      > git checkout r0.3
      > git branch -d r0.3.1-rc1
   

Note that release candidate number (N) must be incremented for every new RC build.

Pushing a release candidate out

Step 1: Create a release on GitHub

On the GitHub repository page, go to releases and create a new draft. On the draft:

• Mark it as a "pre-release".
• Fill out the tag field and select the appropriate branch. Note that this is a release candidate, so the tag should look like vX.Y.Z-rcA

Save it as a draft. You will make it visible once you get all the release notes and uploaded artifacts in order.

Step 2: Build the distribution

Run the following commands, assuming for the sake of example that the branch we are working on is r0.3 and it was tagged with RC tag already:

   > git checkout v0.3.1-rc1
   > ./gradlew clean distribution

The files resulting from the build will be under build/distributions. For each one of the .zip and .tgz files in that directory, generate checksums (currently md5, sha1, and sha256). It is easy to do it with a simple bash script along the lines of:

#!/bin/bash

for file in ./*.tgz ; do md5 $file > $file.md5 ; done
for file in ./*.tgz ; do shasum -a 1 $file > $file.sha1 ; done
for file in ./*.tgz ; do shasum -a 256 $file > $file.sha256 ; done
for file in ./*.tgz ; do gpg --armor --output $file.asc --detach-sig $file ; done
for file in ./*.zip ; do md5 $file > $file.md5 ; done
for file in ./*.zip ; do shasum -a 1 $file > $file.sha1 ; done
for file in ./*.zip ; do shasum -a 256 $file > $file.sha256 ; done
for file in ./*.zip ; do gpg --armor --output $file.asc --detach-sig $file ; done

Note:

• If you are using Ubuntu, you will need to replace md5 command with md5sum. On Mac md5 will work.
• If you encounter an error gpg: signing failed: Operation cancelled when the gpg command is executed, execute the script using sudo.
• You might also see a warning gpg: WARNING: unsafe ownership on homedir '/home/.... Ignore it.

This script assumes the presence of a GPG key to sign the artifacts. Make sure to generate a key in the case you don't have one and to upload it to your GitHub profile. Here is a pointer on how to do it:

https://help.github.com/en/articles/generating-a-new-gpg-key

Once you generate your GPG key, upload it to sks-keyservers.net. This will enable Sonatype to validate your archives and we'll also have all our Pravega keys in one place.

Note: In the future, we might want to automate the generation of checksums via gradle.

Step 3: Upload the files to the pre-release draft

In the pre-release draft on GitHub, upload all the files under build/distributions. Follow the instructions on the page, it is straightforward.

Step 4: Release notes

Create a release notes text file containing the following:

1. Some introductory text, highlighting the important changes going in the release.
2. A full list of commits that you can get with the following command:

   > git log <commit-id-of-last-release>..<current-commit-id>

• <commit-id-of-last-release> depends on the kind of release you are doing. If it is bug-fix release, then we can use the tag of the last branch release. For new branches, we have been adding branch-X.Y tags at the branch point for convenience.
• <current-commit-id> is the commit point of the release candidate you are working on. If you have manually tagged the release candidate, then you can go ahead and use it in the log command above.

Add the list to the release notes file and attach it the notes box in the release draft. See previous releases for an example of how to put together notes.

Step 5: Publishing

Once this is all done, publish the release candidate by clicking on the button on the draft page. Once published, request the developers and community to validate the candidate.

Releasing

Once you are happy with the release candidate, we can start the release process. There are three main parts for a Pravega release:

1. Releasing on GitHub
2. Pushing images to Docker Hub
3. Publishing on Sonatype -> Maven Central

Releasing on GitHub

The process to do this is pretty much the same as the one of creating a release candidate. The only two differences are:

1. The tag should not have an rcA in it. If the successful rc is v0.3.0-rc0, then the release tag is v0.3.0.
2. Do not check the pre-release box.

Pushing the images to Docker Hub

For this step, you need a Docker Hub account. If your account is not associated to Pravega, then please get in touch with some committer who can grant you access.

Once you are ready, run the following steps:

• docker login (using a dockerhub account with write access)
• ./gradlew clean docker dockerPush (this pushes the specific version tags)
• docker push pravega/pravega:latest pravega/bookkeeper:latest (this updates the latest tags, if desired)

Note: You might need to use sudo to run the last two commands. Also, for the second one, you might need to run push separately for the two images.

Publishing on Sonatype -> Maven Central

For this step, you need a Sonatype account. See this guide for how to create an account. Your account also needs to be associated to Pravega to be able to publish the artifacts.

Once you are ready, run the following steps:

• Build with the following command:

# Note: Add your signing key id (example: 00B5050F) to `signing.keyId` property in gradle.properties before you run this command. 
./gradlew clean assemble publish -PdoSigning=true -Psigning.password=<signing-password> -PpublishUrl=mavenCentral -PpublishUsername=<sonatype-username> -PpublishPassword=<sonatype-password>

• Login to Nexus Repository Manager using sonatype credentials with write access to io.pravega group.
• Under Build Promotion -> Staging Repositories, locate the staging repository that was created for the latest publish (format iopravega-XXXX, for example iopravega-1004)
• Select the repository and select the Close button in the top menu bar. This will perform validations to ensure that the contents meets the maven requirements (contains signatures, javadocs, sources, etc). This operation takes a short time to complete, press the Refresh button in the top menu bar occasionally until the operation completes.
• Once the operation completes, locate the URL field in the Summary tab of the newly closed repository (it will be something like https://oss.sonatype.org/content/repositories/iopravega-XXXX where XXXX is the number of the staging repository). This should be tested to ensure that all artifacts are present and functions as expected.
• To test, use for example, the flink-connectors to verify that it can locate and build with the staging artifacts. Concretely:
  1. Change pravegaVersion in gradle.properties to the staging version
  2. Run ./gradlew clean build
• When satisfied that everything is working, you can select the Release button in the top menu bar.
• Wait until it shows up in Maven Central, it takes some time.

Change the Pravega version.

Once the release is done, create an issue and corresponding pull request to change the pravegaVersion in gradle.properties to X.Y.(Z+1)-SNAPSHOT for the release branch.

Done!

Check that everything is fine and go have a drink.