Sorting out python-for-android releases

[inclement]

For the last few months, we've been mostly encouraging people to use python-for-android master. Buildozer does this by default. This has worked okay, especially for the intended purpose of bringing people up to date due to all the rapid changes, but it really isn't a good long term plan.

Main thoughts:

• Releases should be frequent. p4a releases are actually quite straightforward (just push a new sdist to pypi and increment the version), but I've obviously not done a great job of doing this often.
• It's important for p4a to have its own releases on pypi, but also buildozer needs a good way to use the latest release (and for this to also be recent code, old releases are bad).
• Until recently we did have a stable branch that buildozer used, with the idea being that all commits could be merged there once they were tested a bit on master. I think this was not a bad idea, except that we never merged nearly often enough, and once PRs got out of order it was annoying to resolve conflicts (plus I'm not sure if buildozer even behaved right when it came to updating the repo).

I've been wondering more recently if we should try a calver approach, with a release (say) every week. The creation and even pushing of the release package could be automated. This would have the problem that issues merged just before the release time would not be tested for long, but perhaps this could be resolved by e.g. releasing the master branch from 1 week ago, so nothing gets released for at least a week.

This would have the nice feature of a clear release trail on pypi, but also a clear 'most recent' target for buildozer. It would also respect that p4a is a rapidly moving project where it's usually best to be near the top of trunk. I think it might also be an advantage that broken releases could be easily revoked if necessary (not that semantic versions couldn't be, but it somehow seems more natural with scheduled releases), but I'm not sure about it.

Alternatively, we could reinstate the stable branch and automatically schedule merges to it. I think that would be fine, except it doesn't give a clear set of releases for pypi. We could schedule releases on top of that, but then it's the same as the above proposal except with an additional stable branch that only buildozer will really use.

Does anyone have any thoughts about this? All ideas welcome, the above are not strong opinions and I'm sure there could be better ideas.

[ghost]

My first reaction is, a release schedule sounds like a good idea but going more rapid has both advantages and disadvantages. Given how fast we merge or don't merge things sometimes, I think weekly might be too ambitious, although I like the idea of aiming for something regular. But maybe something like every 3 or 4 weeks would make more sense?

Also I think we need way way more automated integration tests. Could we use one travis slot for something really long running that e.g. can run like half a day or something? Because I think otherwise it might be difficult to keep all the basics working in every release, since realistically I doubt we're going to test every aspect of it manually 😬

One test I think would be absolutely necessary is to fire up a device emulator and actually launch a kivy app and make sure it's responsive. Not any elaborate UI tests, but basically that the event loop actually runs and it doesn't instant crash or freeze or whatever, which could be tested by adding in something to the test app code using sockets or such that will respond as soon as the app has launched

And another test would be a full buildozer run, since all our current tests only use p4a directly. Which makes totally sense for the regular tests for speed reasons, but for a full release I think that's something that should run

[inclement]

I think weekly might be too ambitious, although I like the idea of aiming for something regular. But maybe something like every 3 or 4 weeks would make more sense?

I was already thinking after writing this that monthly would probably be more sensible, especially since extra releases would still be possible if necessary. So, no argument there.

Could we use one travis slot for something really long running that e.g. can run like half a day or something? Because I think otherwise it might be difficult to keep all the basics working in every release, since realistically I doubt we're going to test every aspect of it manually

I don't see why not, and this could also be something to explore on other ci providers (not sure what @AndreMiras' most recent thoughts were on that). I didn't mention this mostly because our testing has always been dominated by people testing what they can when they can, which has tended to work adequately for release purposes even though it's clearly far from ideal!

[AndreMiras]

Thanks for opening this discussion @inclement
Yes I like the calver approach and I actually use it for most of my projects.
However we miss the built-in feature of semantic versioning which documents breaking changes by design. To be honest I still prefer the calver specially for releasing often.
As for the branching model, I also like the gitflow approach. All the cutting edge development happens in the develop branch and we release to master. So buildozer could still be using master as a default branch. While master would be considered stable it would still be updated often enough if we release every 3-4 weeks or so.

@opacam @Jonast

Also I think we need way way more automated integration tests. Could we use one travis slot for something really long running that e.g. can run like half a day or something? Because I think otherwise it might be difficult to keep all the basics working in every release, since realistically I doubt we're going to test every aspect of it manually grimacing

I'm not aware of Travis or other CI allowing running tests for too long. Even if it did it might be painful to develop, troubleshot and maintain. But yes we definitely need to find ways to improve our test coverage/quality while not increasing too much the running time.

[ghost]

while not increasing too much the running time.

to me that seems like wishful thinking that will just lead to broken releases, I just don't see it happen @ significant reduction in runtime. and with the current test suite I personally think we're not very well-equipped for a quick release. I doubt it'll work out without major breakages unless we get in a full buildozer run and an actual launch in a device emulator. and that might easily get us 1+ hour runtime

I'm obviously not suggesting to run this after each pull request, it could run daily & before a release. if travis doesn't allow that much runtime, I think we should see if there is another platform (maybe to use in addition?) which does

[matham]

Regarding travis, there's a 50 min limit for each test. So you'd need to create a test for each recipe that is run sequentially, otherwise it'd time out.

Just to bring this up, but I don't want to get off topic, the way conda-forge does it, is have each recipe in its own repo. Each recipe inherits from a common recipe and is updated automatically by a bot when the base recipe changes using a PR made by the robot. The recipes are individual repos under https://github.com/conda-forge/. This means that each recipe gets to run on the CI independently so the 50 min limitation is fine. And each recipe gets to define how it is tested.

But this kind of change would require a lot of infrastructure work and time. But just FYI, in case this is useful.

[inclement]

I noticed that we're essentially following a 'Github flow' approach right now, which is mildly interesting but probably not that important.

As for the branching model, I also like the gitflow approach. All the cutting edge development happens in the develop branch and we release to master.

I would be happy to try git flow - I'm loosely familiar with the idea, but I haven't actually followed that workflow before.

From a practical point of view, I think this would require:

• Split off a develop branch from master
• Immediately do a new release (not essential but a nice way to have a clear marker for what's going on)
• Set the develop branch to the default

After this it seems like everyone can carry on like before, except that user stuff will get merged to develop and it's up to us to manage releases and (if necessary) hotfixes. This seems like a fairly minor burden. What do others think?

[ghost]

I like the idea of develop & having master in a bit more stable state 👍

[inclement]

I've made #1836 to start writing down how we actually do releases. The lack of this list is one reason I handle it so rarely, even though it's easy to do! I intend to do one soon, alongside setting up a master/develop git flow structure, if there's no opposition to that.

Edit: We probably should decide what we're doing about versioning though. It could be as it is now, it could be semantic, it could be calver. I'd like to try calver probably. I don't see any strong reason not to version as YYYY.MM.DD.

[tshirtman]

Considering a lot of changes can be driven by upstream evolutions, calver does make a lot of sense, weekly would be great, but indeed maybe a bit too much, monthly should be enough to mitigate most issues? As noted, it's always possible to trigger a new version of needed, the point is making releasing such a common thing that it's not something you even have to think about.

If timing is short to test new things before a release, it's always possible to delay a merge for a few days and resume developments after the release itself.

[opacam]

calver versioning system is good, and certainly will make easier to create schedule releases (I think that should be monthly for now).

About git flow, I've been using the proposed scheme for years in some of my projects and I may add that, it's a good practice to create a release branch from the develop branch whenever the team decides to create a new non automatized release.This release branch will contain any action that should be done (bump version number in files, apply translations, finish docs, or whatever...plus last bug fixes). To finish the release branch some actions should be taken: merge it into master, then tag the master branch for future reference and then the release branch is also merged into develop branch. Following this procedure allows the team to continue the development in develop branch as usual without any interruption. I learned this process some years ago from an article which I share with you, A successful git branching model, because it's well explained, with nice pictures and the git commands to apply in each step...(I find myself going back to that article whenever I have some doubt...so maybe it will be useful for the team)

[inclement]

Closing in favour of PR #1838, which documents the proposed new development method. As described in that issue, once that's approved I'll go ahead and immediately set up the develop branch and first release branch. We can also use this first release to resolve any issues that come up with the proposed procedure.