Design Solana ABI management process

[mvines]

The Solana ABI (binary interface to the cluster) is currently only defined implicitly by the implementation, and requires a very careful eye to notice breaking changes. This makes it extremely difficult to upgrade the software on an existing cluster without rebooting the ledger. During early development we deliberately decided to ignore breaking ABI changes but we can no longer afford this luxury.

As a starting point, I think we will need:

1. A way to identify all structs/enums that cannot be modified without breaking the ABI (Transactions, Shreds, Ledger bits).
2. A mechanism to turn CI red when those structs/enums are modified.
3. A human process to decide whether a breaking change should be accepted, and if so how to manage the resulting hard fork.
4. A comprehensive test suite (cc: Network ABI Testing #6123) of golden binary ledgers that should always be consumable by newer software versions
5. Guidelines to the eng team for all of the above, including general agreement on how strict we want to be. eg, once the system program is published/locked down, would it be permitted to even add a new system instruction? Most strictly, no

[ryoqun]

(Coming from here)

I'll give this issue a try!

First I'll lightly research what other projects are doing with regard to tracking ABIs in the rust community and by other DLTs. And I'll write a draft design proposal for it based on these findings (This will be my first design proposal, btw!).