PreparingReleases

This document outlines how to manage the MDAnalysis sources and how to prepare distribution packages. See also DevelopmentWorkflow for new ideas how to organize codebase and workflow.

Release policy and release numbering

We use a MAJOR.MINOR.PATCH scheme to label releases. We adhere to the idea of semantic versioning (semantic versioning was introduced with release 0.9, see Issue 200): Given a version number MAJOR.MINOR.PATCH, we increment the:

• MAJOR version when we make incompatible API changes,
• MINOR version when we add functionality in a backwards-compatible manner, and
• PATCH version when we make backwards-compatible bug fixes.

However, as long as the MAJOR number is 0 (i.e. the API has not stabilized), even MINOR increases may introduce incompatible API changes. As soon as we have a 1.0.0 release, the public API can only be changed in a backward-incompatible manner with an increase in MAJOR version.

Additional labels for pre-release and build metadata are available as extensions to the MAJOR.MINOR.PATCH format.

The CHANGELOG lists important changes for each release.

MAJOR, MINOR, PATCH number are integers that increase monotonically. Pre-releases are labeled with the postfix -rc1, -rc2, etc.

The release number is set in setup.py and in MDAnalysis.__version__ (MDAnalysis/version.py), e.g.

RELEASE = '0.7.5'

While the code is in development (i.e. whenever we are not preparing a release!) the release number gets the suffix -dev0, e.g.

RELEASE = '0.7.6-dev0'

so that people using the development branch from the sources see immediately that this is not a final release. For example, "0.7.6-dev0" is the state before the 0.7.6 release.

git repository: branches

Releases should exist on the master branch:

• declare feature freeze on develop via developer list
• finalise the CHANGELOG file with date
• merge develop into master (make sure that it gets the full release number)
• increment version
• rebuild c files (See Issue #667)
• tag the release

Preparing distribution tar balls

Preparing

• Update local repository (git pull) and make sure that there are no conflicts.
• Update CHANGELOG with the release number and summarize important changes. Add all authors that contributed to this release. See the file itself for guidelines and formatting.
• Commit updated tree git commit
• Set RELEASE in setup.py and MDAnalysis.__version__ (MDAnalysis/version.py); note that there is a setup.py in the MDAnalysis package and the MDAnalysisTests testsuite!). There's a script maintain/change_release.sh that typically does the job:

./maintainer/change_release.sh 0.9.0

• Make sure to add and commit the changes!
• Test if the distribution (MDAnalysis and MDAnalysisTests) builds successfully and run the UnitTests:

./maintainer/run_tests.sh

• Fix code so that all tests are passed (except for known failures). Note that if you worked on any of the Cython code then you should check that the new code was also compiled (monitor the output from python setup.py build).

Note on Cython code (see Issue 85 for details):

• If no Cython, setup.py will use C-code that was previously generated; of you updated any *.pyx files then you must have Cython installed (but you probably new this already).
  
• If Cython if installed, it is used to compile extension and .pyx source files are used instead of .c files.
  
• From there, .pyx files are converted to .c files if they are newer than the already present .c files or if the --force flag is set (e.g setup.py build --force).

Therefore, as a developer you should (1) have Cython installed and (2) only use --force if you are positive that you want to regenerate all .c files.

• commit any changes and fixes that were necessary to make it work with git commit
• Build a source distribution tar ball with

# MDAnalysis
cd package
rm -f setup.cfg
python setup.py sdist

# MDAnalysisTests
cd ../testsuite
python setup.py sdist

This builds the distribution under package/dist/MDAnalysis-MAJOR-MINOR-PATCH.tar.gz and testsuite/dist/MDAnalysisTests-MAJOR-MINOR-PATCH.tar.gz.

Testing

Unpack the distribution in a tmp directory and try to build it:

mkdir tmp && cd tmp
tar -zxvf ../dist/MDAnalysis-0.7.5.tar.gz
cd MDAnalysis-0.7.5
python setup.py build --build-lib=.

The new test cases should be in your system as developer installation (from the previous step) so you should be able to run the UnitTests (run python from the same directory):

>>> import MDAnalysis.tests
>>> MDAnalysis.tests.test(label='full', extra_argv=['--exe'])

(this imports the module compiled in ./MDAnalysis.)

The above should work at least on Linux and Mac OS X. If it fails then go back and fix things and do not release.

If everything works then we are now reasonably confident that a user can also compile and use the package so we push the source code for the release to the master repository. Push it to the master with

git push origin master

(or similar command).

Tagging

The distribution is now ready. Tag it in git:

git tag -m 'release 0.7.5 of MDAnalysis and MDAnalysisTests' release-0.7.5
git push --tags origin master

The tag format is the string release- followed by major.minor.patch.

Upload to PyPi

• upload the source distribution tar balls of MDAnalysis and MDAnalysisTests to the python package index (e.g. using

python setup.py sdist bdist bdist_egg upload

(you need to have run python setup.py register at least once before).

• build the docs and upload to PyPi
• make the new tar ball a featured release so that it shows up on the front page (and unfeature any older releases).
• provide a short description (a condensed version of the CHANGELOG)

Update/create a release page on the wiki

Create a ReleaseXYZ wiki page, modelled after e.g. Release062 (using the CHANGELOG as a reference). Also add it to the Release Notes.

Update docs on PyPI

Since 0.7.4 there's also a MDAnalysis page on PyPi (the Python Package Index).

For full releases we have the documentation at http://packages.python.org/MDAnalysis/ . It is updated by building and then zipping the html docs and uploading manually:

cd package/doc/sphinx
make html
cd ../html
zip -r ../pypidoc.zip *

Then upload documentation (the file pypidoc.zip) via the MDAnalysis: Edit page.

Update release on anaconda

Update the release version and build the package in one vagrant box from this repo.

anaconda upload -u MDAnalysis /path/to/build/package/mdanalysis-version-py27_0.tar.bz

Test data

UnitTests and test data are maintained in the same git repository as the MDAnalysis package. Tests are checked in together with code (for details see Issue 87, UnitTests and MDAnalysisTests).

The library, MDAnalysis and the tests, MDAnalysisTests share the same release number: whenever there's a new MDAnalysis there will be a corresponding MDAnalysisTests. Therefore, the two packages are maintained and built together (see above).