-
Notifications
You must be signed in to change notification settings - Fork 41
Continuous Delivery [draft]
See also: Release Plan
First, before be start with what should be done, let us start with the problems with current approach of feature branching / freezing. The main reason we wanted to have continuous integration and eventually continuous delivery is to have a quicker release cycle, so that we bridge the gap between "are we doing things right" and "are we doing the right things". So, smaller incremental releases are better so that users immediately get to interact with the product as we make updates and then we can pivot based on the users needs. Note that this would be sorta of a self correcting mechanism because users will only file bugs on stuff they care about so, the faster we send stuff to users a) Lesser the risk of us wasting time on unwanted features b) Lesser the risk of us freezing and eventually merging a big feature which can break pre existing stuff and c) Developers see their code in production more often which motivates them to contribute. The current proposed model is a traditional approach where a feature is frozen until fully polished without any feedback from the users. I find this problematic.
Let us split our entire pipeline into 2 sections. One is the "are we doing things right" section. This is something which is part of github actions currently. Build tests, Unit Tests, Some basic automated performance tests. All go into this. The major thing missing here is unit tests. Second is the "are we doing the right things" section. This is where most people do "acceptance testing", anything which passes this is a release. We have to split this into 2 parts, specific and general tests. What we need is a general testing team, an artifact repository, and an acceptance criteria. The pipeline is something along the lines of "commit from master (or some commit every day) -> CI -> Uploads the final releasable product to an artifact repository with the commit ID -> Testing team gives us feedback for a particular commit". Any commit which passes the criteria becomes releasable. Milestones are only for the developers to keep track of the direction of development, not the things which decide when a particular product is released. Waiting for milestones makes the development slowdown to a crawl.
What we need is a deployment pipeline. A simple transparent methodology which gives us a releasable product at the end of it. The point being, every commit (or nearly every commit) goes through the deployment pipeline and the majority of commits are always in a releasable state. This way, the release is more a matter of time (Say a release every 15 days with whatever we have in master or a release every 100 commits or so initially until we gain traction).
For testing team, We can pick a bunch of our players, who are active but don't wanna code for testing purposes. They are our "focus group". They download the most recent artifacts from the artifact repository and approve a particular commit.
This is a repository with binaries which keep track of the commit id, the binary was created from master. Some sort of version control for binaries.
3 levels of approval, approved, approved with bugs, unapproved because of bugs. 3rd one is obviously high priority for further commits. Second one is for bugs which are just opened as an issue but the release goes though. Every 15 days (or maybe even 2 months), we can release a product with the most recent approved commit ID, and it goes to the general user. In this approach, stable vs unstable purely depends on the feedback from the "focus group". Second factor is, an individual developer isn't expected to iron out the bugs of a feature like in feature freezes. It goes to master, and the bugs are picked up by any other developer for the incremental changes.
What this does is to give a very transparent pipeline to developers. They can see their commit go through the various stages and see it come out in the main release, I think that is an important aspect. Secondly, it sticks to master being at a "releasable" state all the time, features freezes means, we go back to the stable-unstable cycle where right after a merge it is unstable.
One more aspect which ties it all together is, individual issues/features need to be small. That way, we can lay the groundwork of the features within master and previous releases where both the focus group and the players get the feature in waves. That way a) We can always pivot fast b) The features have lesser chances of breaking stuff. We have to ask the developer to split his work into playable chunks if needed. Something which cant do that is non-modular and problematic in the first place
To summarize, Somewhere along the way, in order to orient ourselves we started using milestones. But the milestones alone shouldn't dictate our release cycle. Releasing only with big flashy features is a very traditional approach and leads to a lot of branching, and slower feedback cycle from actual users. Releasing should be something as simple as anything at the end of deployment pipeline is releasable, and the actual releases are made maybe only from a marketing perspective not based on code stability.
[1] https://www.youtube.com/watch?v=mBzDPRgue6s [2] https://www.youtube.com/watch?v=x9l6yw1PFbs These videos is by a person who pioneered continuous delivery. The channel also has some related stuff on test-driven development which facilitate CD, but these 2 are the relevant ones for release cycles.