Create a release checklist

[inclement]

I'm thinking to create a release checklist and instructions that we can reuse, in preparation for acting on #1833. Once this is complete, I'll move it to proper documentation somewhere.

The following is what I currently would do to make a release. With a git flow workflow we'd want to do the same thing, except with the nice feature that the release gets its own branch to run tests and checks on - I've never been very good about that.

• Check that the build is passing
• Run the tests locally via tox: this performs some long-running tests that are skipped on Travis.
• Build and run the on_device_unit_tests app using buildozer. Check that they all pass.
• Build and run the following testapps for arch armeabi-v7a and arm64-v8a:
  • python3 setup_testapp_python3_sqlite_openssl.py apk
  • python3 setup_testapp_python2.py apk
• Check that the version number is correct
• Tag the release
• Create the release distributions: python3 setup.py sdist
• Upload the release distribution to pypi: python3 -m twine upload
  • ^^ this one I'm not certain about, last time we released I found things had changed and twine was now compulsory, I'll check it and fix the instruction on the next release.

The release should also involve an announcement, which I intend to get on top of but don't consider part of this core process.

Questions:

• How does this change for git flow? The release branch procedure is then well defined, but this needs writing down.
• Are there any quick improvements to be made to the tests I've proposed? Obviously in the medium and long term there's lots we'd like to do, but this is the minimal set of things that roughly represents checks I've done in the past (in addition to general use of p4a).
• Is there anything missing from this list?
• How should we actually manage dev branch and release version numbers using calver?

I'm intending to test this procedure as currently written for making a release at the same time as setting up the develop branch, in order to check it works.

[ghost]

.travis.yml does currently exclude some pythonforandroid/pythonpackage.py tests (intentionally, due to long runtime). tox runs them.

Therefore I propose that running tox and checking that no tests fail should be added to the release checklist! It might also be a good idea to add a remark / document that this runs more test than travis for time constraint reasons, just so people understand why this is included into the checklist

[inclement]

Good point @Jonast, I've added it to the list.
I also added that the testapps should be run under both armeabi-v7a and arm64-v8a. We'll want to do more general work to make sure arm64 is tested properly, but it should already be good to have it in the checklist.

[inclement]

Closing in favour of PR #1838