Thoughts on adapting release process to cater for consumers

[siggerzz]

After our team discussion yesterday around versioning, particularly how we're going to handle beta releases, I just wanted to create this discussion, mainly for my own benefit around how we're currently releasing, and things we need to start thinking about before consumers start adopting packages.

With changesets, we currently have a versioning mechanism that allows us to tag components on npm as either next / beta / latest - see here.

Despite the ability to publish components under different tags, we don't really have a process in place for when a tag other than latest is required.

Current Problems

If we continue with our current process of merging into main, we're going to run into maintenance problems on our side, and potential confusion for consumers.

At the moment, we tend to always merge changes directly into main, meaning packages will always get versioned as latest.

This will need to change when we have consumers adopting our packages. If we advertise beta versions of our packages, they should be tagged as such to prevent issues around consumers accidentally consuming a version of a package that isn't ready for adoption. It also allows us to have full control of when we promote a package from beta > latest (perhaps a wider discussion needed around fortnightly / monthly releases or similar?)

Proposed workflow changes to combat some of these problems:

This approach would bring our process more in-line with other design systems:

Other systems that follow this repos that utilise this approach with changesets:

Shopify Polaris - Shopify/polaris#9865
Astro - withastro/astro#7897

It also presents a number of questions that would need to be answered:

• Would an hosted instance of Storybook be required for beta components?
• Should beta process be mandatory for all changes?
• How does this impact the docs site? (Similarly to Storybook, would we have a docs beta site?)
• How do we handle hotfixes?

[webbertakken]

I'd recommend a simpler approach, using 2 persistent branches

• main (default branch)
• next (holds breaking changes)

With the following rationale

• Changes that get merged into main will also land in next by merging main into next.
• Each feature branch targets main until breaking changes are identified, at which point target will change to next.
• Use tags on both main (i.e. v1.0.0) and next (i.e. v2.0.0-beta.0) branch
• Merge next into main when releasing the next major version.

---

With regards to the questions

Would an hosted instance of Storybook be required for beta components?

I don't see why not. Storybook compiles to a static site. You can easily host it at very low cost.

Should beta process be mandatory for all changes?

I'd recommend to only do it for breaking changes, including redesigns. I'd also recommend to only have 1 next branch and using that as a constraint.

How does this impact the docs site? (Similarly to Storybook, would we have a docs beta site?)

Docs are statically generated to. Hosting them doesn't really cost much.

How do we handle hotfixes?

Allow hotfixes both for main and for beta. Treat them as normal feature branches before merging them, with the difference of them having higher priority.

[siggerzz]

Thanks for this @webbertakken!

[siggerzz]

Regarding the impact this has has on documentation. Carbon have multiple versions of their documentation portal.

https://carbondesignsystem.com/ - Production Docs for v11 of their design system (current)
https://v10.carbondesignsystem.com/ - Production Docs for v10 of their design system (old version)
https://next.carbondesignsystem.com/- Production Docs for v11 beta version of their design system

[ashleynolan]

Thanks for this writeup - really good. My main thoughts:

Should beta process be mandatory for all changes?

I agree with @webbertakken on this one – I'd only use next for major breaking changes. By default, minor releases should be less risky (especially for components we consider stable at that point).

From using Changesets previously for beta releases, having multiple feature branches as the component next branch makes sense. But we should only have one next branch per component. So pie-button could have its own next feature branch, while pie-modal had a separate next feature branch. That way, these branches can be merged to main as each component needs to come out of beta. For bigger rebrand style releases, I agree that should be handled by a single next branch across all components – and we should avoid having other next branches running at the same time as a major beta like this, as it'll likely cause conflicts.

Would a hosted instance of Storybook be required for beta components?

My main question would be – wouldn't the storybook builds on PRs anyway cater for this? Do the Version PRs have their own deployed temporary storybook currently? If not, that would probably be the way I'd handle this.

How does this impact the docs site? (Similarly to Storybook, would we have a docs beta site?)

We've talked about doing this before. As above, I imagine our PR previews should cater for this mostly. The main thing we may want in the future is to keep older versions of the docs site for previous major releases. We've talked about this in the past, but may be one to think about again in the future.

How do we handle hotfixes?

Agree with @webbertakken – should just be regular releases into both branches (if a next branch exists for the component being hotfixed).

[webbertakken]

Agreed. A quick but relevant nuance to the reasoning about feature-next

@ashleynolan: But we should only have one next branch per component. So pie-button could have its own next feature branch, while pie-modal had a separate next feature branch. That way, these branches can be merged to main as each component needs to come out of beta.

While it's possible to have a separate feature branch for each component that hasn't been released, I wouldn't necessarily refer to these as the next or <feature>-next branch as it adds some ambiguity to the meaning of next and potentially adds quite a bit of real estate to your CI for keeping all next branches up-to-date in the long run.

In that I'd argue that it is in fact better to just refer to it as a regular feature branch, or simply version the component itself and merge it into main as any other feature. From a repository point of view: adding new features or components can be done using a patch or minor version. If it does introduce breaking changes then the component will only be usable in the next major version and therefor reside in next branch; this is the entire point of having a next branch.

This approach would simplify your developers git life by quite a bit in case you can see this working for you.

[siggerzz]

I guess for @webbertakken's benefit, the main thing i'm trying to get to the bottom of is this:

There has been talk within the team about our first Web Component consumers 'beta testing our components', with the first one likely to be pie-button.

If this is the case, I'd prefer to 'go by the book' / follow common practice (at least how I interpret it) and ensure that if we as a team refer to any package as 'beta', that is reflected in both git/npm. Mainly because:

• Incorrect usage of npm tags could give the wrong impression to consumers that a package is considered stable, when in fact it's still experimental / potentially unstable.
• Allow potential consumers to make informed decisions about which repos make use of beta components (for example a team that's risk adverse may only want to adopt beta components in lower-tier applications.
• Continuing to tag everything as 'latest' causes confusion both internally & to consumers about which versions of a package are suitable for adoption.

My point is, is that I'm less concerned (for now) about how this affects working in the monorepo, but I want to make sure we're being transparent with consumers.

To @ashleynolan's comments.

My main question would be – wouldn't the storybook builds on PRs anyway cater for this? Do the Version PRs have their own deployed temporary storybook currently? If not, that would probably be the way I'd handle this.

If we go with the approach I'm suggesting, in that we by-default go from PR > Beta Version > Promote to 'latest' when ready, it is highly likely that a beta version could require slightly different implementation (new / removed component props / tokens etc).

As a result, I think it's important that this is reflected in both Storybook to aid consumers in their integration, as well as our documentation.

Personally, I think we should be looking to achieve what Carbon have implemented in my previous comment.

I originally wrote this post after trying to integrate an unrelated library, where the beta version of a package different from 'latest' and their documentation didn't reflect differences in implementation, which I found incredibly frustrating.

I'm just very, very keen to make sure we don't fall into the same trap.

I struggle getting my thoughts written down on this kinda thing, but I have a lot more to say on this, should we ever want to discuss further :)

[webbertakken]

Creating additional branches just so you can relate (repository wide) tags directly to every specific component doesn't seem what you call "by the book" at all. It's a ton of overhead that you should only accept if you really need it.

Reason: You'll lose some of the benefits of having a monorepo if you effectively put every project in their own branch and you'll end up spending much more time integrating changes.

I think a good question to ask would be: How do we version our packages in a way that is clear to our end users and lightweight to our process (and CI maintenance)?

[webbertakken]

A couple other things to consider:

• Tags with a prefix in the main branch my-project-v2.0.0-beta.1, where my-project refers to project and the v2.0.0 to the next version, and -beta.1 to the first beta for that future version. This might come in handy if you usually make forward fixes anyway.
• regarding tags as repository wide instead of per project, but using the version field in package.json. Then repo tag latest or v1.0.0 will refer to the main version of the design system, but components are still allowed to be versioned independently. (i.e. don't use tags for every project, just the version field).
  • You can easily update that version number the moment when you publish it
  • You can easily extract the version number programmatically using import { version } from 'packages/<package-name>/package.json'
• Using component or project level opt-in
• Using a new name when completely rebuilding a component, then @deprecate the old component in favour of <name of new component>, because <rationale>