Moving to prescheduled network upgrades

[kaitlin-beegle]

Last week, a conversation was started amongst Core Devs as to whether or not we should begin prescheduling network upgrades.

We have had this conversation at various points in the past; however, it was always determined that Filecoin's ad hoc scheduling of network upgrades was a more responsive way of managing network development goals.

Now, as more and more participants come into the network, it is increasingly difficult for Core Devs to account and plan for network upgrades in a way that is comparatively advantageous to a prescheduled set of upgrades.

IMO the benefits of a prescheduled upgrade schedule could include...

• Network upgrades are a finite resource; prescheduling them gives all network participants equal access to information about when new FIPs may be able to enter the network.
• Gives advanced notice to network participants who contribute to upgrades, including SPs, exchanges, network infra teams, etc.
• Offers scheduling benchmarks that allow us to set advanced priorities for specific upgrades, etc.
• Replaces a closed planning process with an open standard.
• Saves me and @jennijuju a lot of time :)

Trade-offs include:

• Challenge of navigating public exposure during emergency upgrades
• Pre-set upgrades may create a sense of scarcity around proposing and passing FIPs, a process which should be decoupled from the process of network upgrading

Ultimately, I think the trade-offs here are quite manageable, and we should move forward with a set of 3-4 prescheduled upgrades in 2024. Open to hearing what others think, including if there are trade-offs that I didn't list above; would also love to know what cadence of upgrades feels most appropriate.

[anorth]

Sounds good to me, thank you. I think it's definitely worth switching to as an experiment. We can always change back to ad-hoc if we learn things about a scheduled approach that change the analysis.

[jsoares]

I second. Even the trade-offs seem a bit moot: having a predictable schedule does not stop us from skipping one (in the unlikely event of an empty queue) or doing an extraordinary update (in case too much stuff starts piling up). But at least it gives the community a predictable baseline for steady-state conditions.

[momack2]

I support this too! I generally think it really simplifies the upgrade planning process! 🙏

[momack2]

Proposed schedule from my end is once per quarter, a little over 1/2 way through the quarter:

• Late February
• Late May
• Late August
• Mid November

This gives us "not too long" before the next upgrade for anything that just misses cut-offs (decreasing the temptation to delay a release for a non-critical feature), and also nice predictability with the quarterly release process. Doing it a little over half way through the quarter avoids most major holidays and breaks, and makes space to build, iterate, and test before quarter boundaries.

[raulk]

A quarterly upgrade schedule makes sense, as long as the dates are not set in stone. This removes the ad-hoc negotiation/planning activity that happens for every release. A few thoughts on my side:

1. Let's build in some flexibility on the dates to account for "meatier" releases (which may require heavier testing), testnet disaster and recovery, community events, etc. For example, I don't think a release should be postponed a quarter if it failed to make the original schedule by a few weeks. This is probably implicit in our thinking, but doesn't hurt to make explicit.
2. This is most useful if core devs commits to higher visibility into the innovation pipeline -- tracking what's being prototyped, implemented, specificied, etc. on a monthly basis by the various core implementation teams. That minimises surprises and enables more intelligent planning with higher foresight. Currently every change core devs aim to ship to the network requires a conversation about network upgrades. Having pre-planned network upgrades removes the need for that check-in, so we still need a way to socialise about what's in the pipeline so nothing comes as a surprise.

[anorth]

A few weeks is a quarter of a release cycle. While I agree we need some flex for the node implementation teams actively aiming at a release date to take a few days longer than expected due to fixing a late bug or similar, I think feature teams should expect the train to leave the station whenever the node implementation teams say it must.

[jennijuju]

Well - y’all know how I feel about this, strong yes!

Not only this allows implementers to better plan their resources and roadmap, and also set clearer expectations for FIP authors (which I advocate core devs also have a clear path for community to reach out for making support and inquiry)

I have been coordinating filecoin upgrades for the past 2years across teams and community, and from experience, I propose 3 network upgrade / year:

• late Feb
• early July
• late Oct

This avoids major holidays and regular vacation gaps for teams, and leaves enough room for network monitoring/analysis post each upgrade.

We shall have a clear guideline on when an out-of-schedule upgrade can be granted, i.e: emergency security upgrade.

[kaitlin-beegle]

@jennijuju I think this sounds good. It seems like folks broadly agree that this change is needed, and there are no counter proposals for a schedule.

The only thing I'd like to note is that, by pre-planning upgrades and "setting" the timeline in place, we are fixing one of the most difficult variables in upgrade planning. I would bet that we actually find significant efficiencies by doing this pre-planning, and only hosting three upgrades per year will seem like too few.

However, I think we should start with this, and see how it works!

Is there anything you can think of, operationally, that we should do to put this into effect? Perhaps @luckyparadise can set up placeholder discussions for nv22, nv23, and nv24, noting the anticipated timeline for each?