[Meta] versioning

[sydhds]

Intro

After mainnet, we will need to have a way to update some parts of the code without shutting down the whole network.

A really obvious example, would be if we decide to change the Virtual machine for Smart contract execution:

• we could decide to support Solidity written smart contracts

Components using versioning

• Address
  • Update hashing algo (from blake3 to XXX)
• Smart Contract VM
  • Introduce a new VM (for Solidity written SC)
• Block
  • Add new items in Blocks
  • Update maximum number of items (e.g. Operations, for now it's limited to 5000)

Edit from sydhds (05/01/2022):

• Endorsements
• Operations
• Async message
• Storage format
  • Discussion about storage modifications
    • Remove identifier in ledger keys. #3056
    • SC Datastore #2970

General overview

For this overview, we start with:

• a Block structure like this:
  • [CURRENT VERSION][FUTURE VERSION][BLOCK DATA]
    • where current_version / future_version is in form: [b_1] / [b_2] ... / [b_N]
    • [] is a valid value (empty value provided)
• the blockchain code version: [mainet_1]
  • this version implements support for Block version 1 only
  • it expects Block like: [b_1][empty][block_data] or [b_1][b_2][block_data]
• the blockchain updated code version: [mainet_2]
  • this version implements support the Block version 1 & 2
  • this version has a trigger (for now a simple timestamp: [TS]) defining when block like [b_2][empty] will be produced

Here is an expected timeline:

• T0:
  • Node [A] in the blockchain use code version [mainet_1] and expect blocks: [b_1][][block data]
  • Node [B] in the blockchain use code version [mainet_1] and expect blocks: [b_1][][block data]
• T1:
  • A new code is out: [mainet_2], node [B] use it and produce blocks like: [b_1][b_2][block_data]
  • node [A] running [mainet_1] can still process blocks
    • by processing blocks with a future version, it is now aware of code update
• TS:
  • node [B] will now produce blocks like: [b_2][][block_data] and can still process [b_1][][block_data] && [b_1][b_2][block_data]
  • Same for node [A] (if updated to [mainet_2])

TODO: add a schema here?

General overview: edge cases

'Bad future' version

Receiving block like

• [b_1][b_BAD][block_data] are still ok. However, announcing [b_BAD] should be monitored and user warned? (possible attack)
• [b_BAD][][block_data] are ignored but the sender should be banned

Some nodes do not update to [mainet_2]

• These nodes will fork the blockchain after [TS] by being unable to process block like: [b_2][][block_data]
  • What incentive are for node runners to update?
    • Cool new features
    • Security & bug fixes

What if a majority of users do not switch

That's decentralization and that might mean that the update is not good / has no incentive. This can lead to a fork.

What if a node with [mainet_2] produces block [b_2][][block_data] before [TS]

Node should be ban from the network

Bitcoin and versioning

Specs (aka BIP)

• Pay to Script Hash - BIP-0016

  • type: [New transaction]
  • note: applied when validating transactions in blocks with timestamps >= 1333238400 (Apr 1 2012)

• BIP-030

• Block v2, Height in Coinbase - BIP-0034

  • [2012][Goal] Introduces an upgrade path for versioned transactions and blocks from v1 to v2 -
  • Implementation / PR
  • BIP using it for deployement
    • Strict DER signatures
      • type: [Restrict signatures validity]
      • nVersion = 3
    • OP_CHECKLOCKTIMEVERIFY
      • type: [New opcode]
      • nVersion = 4

TODO: clarify what is nVersion

• Version bits with timeout and delay - BIP-009

  • [2015][Goal] Allowing multiple backward-compatible changes to be deployed in parallel
    • TODO
  • Tracking updates: https://github.com/bitcoin/bips/blob/master/bip-0009/assignments.mediawiki
  • BIP using it for deployment:
    • https://github.com/bitcoin/bips/blob/master/bip-0016.mediawiki
    • Relative lock-time using consensus-enforced sequence numbers
      • type: [Consensus modification]
      • deployed simultaneously with BIP 112 / 113
    • CHECKSEQUENCEVERIFY
      • type: [New opcode]
    • Median time-past as endpoint for lock-time calculations
      • type: [Block inclusion]
      • TODO
    • Segregated Witness (Consensus layer)
      • type: [Add a new structure to Block: Witness]
      • deployed by "version bits" BIP9 with the name "segwit" and using bit 1.
      • deployed simulateously with BIP 143 / 147

• Version bits with lock-in by height - BIP-008

  • [2017][Goal] Correct mistakes on BIP-009

BIP-009 notes:

• Futures changes:
  • Modified thresholds: beyond BIP-0034 threshold (95%)
  • Conflicting soft forks: When two mutually exclusive soft forks are proposed
  • Multi-stage soft forks:

Components

Note: From bitcoin.it/wiki (so info might not be up-to-date)

• block: https://en.bitcoin.it/wiki/Protocol_documentation#block
• adresses: https://en.bitcoin.it/wiki/Protocol_documentation#Addresses
• tx: https://en.bitcoin.it/wiki/Protocol_documentation#tx
• alert: https://en.bitcoin.it/wiki/Protocol_documentation#alert (WARNING: DEPRECATED)

Etherum and versioning

EDIT from Aurelien (05/01/2022):

I found some interesting resources on Ethereum by digging their EIP process. A good article from a developer community organizer.

I dug into the EIP that cause fork or not. For the lights changes they are basically just adding new features with control flow (for example if).

For update that can cause forks they are using a block number to make the change or not.

An example is the Constantinople Update in 2018. They define a new block height in the code that will switch to new version and they have if all across the code to get the correct values depending on the current version.

They don't remove code from update that are passed long time ago. For example :

blockReward := FrontierBlockReward
	if config.IsByzantium(header.Number) {
		blockReward = ByzantiumBlockReward
	}
	if config.IsConstantinople(header.Number) {
		blockReward = ConstantinopleBlockReward
	}

They still have this two if condition the two are valid now and so this kind of code could be refactored. Maybe they keep it if they want to replay the chain from start with update made as the right block heights.

In case they want to postpone a version they just revert the block height defined.

Specs

• https://eips.ethereum.org/EIPS/eip-1702
  • [2017][Goal] Introduce account versioning for smart contracts so upgrading the VM or introducing new VMs can be easier.
• https://eips.ethereum.org/EIPS/eip-2124
  • [2019][Goal] Fork identifier for chain compatibility checks

Next steps

Discussions

We propose to make this discussion a Meta discussion and links to several new discussion:

• Version info & trigger
  • Should version be a simple integer?
  • Defining a timestamp for a new version might not be enough so explore others strategies
• Versioning code organisation
  • Code will be massively impacted, and we should explore strategy to handle it
  • Extra care should be taken to avoid code spaghetti and ease code maintenance in the long term
• Code testing
  • Explore how to test versioning in functional tests
• Other blockchain organisation ?
  • Explore others blockchain (note that Bitcoin & Ethereum info should be kept here)

Code

In the meantime, we also think that the following dummy versioning implementation/test could be made:

• New address v2 with sha-512 hash
  • Impact on the code of address v1 (length: 256 - blake-3 hash && v2 (length: 512 - sha-512 hash)
  • Very simple test to have a first look of code organisation / modification
• New VM implementing support for Wasm + SIMD instruction
  • Easy to implement in VM (enable a Wasmer feature) and test (AS already provides direct access to SIMD instruction)
  • Could be used as a starting point if we want to investigate cross VM calls & compat
• Add block v2
  • Accept both address v1 & address v2
  • Add new items in blocks: dummy Denunciations objects
  • Open question: after we switched to block v2 - should we still accept block v1? address v1 ?
  • This test is an opportunity to test if we would want to deploy 2 versioning at the same time:
    • adresse + block
• Add trigger to switch from block v1 to block v2 by using a simple timestamp
  • This is an opportunity to do some functional testing

[AurelienFT]

In components using versioning I think we can add: operations, endorsements and async messages. Maybe we should think about the storage format also. In the case of this kind of RFC is accepted : #3056

At TS, I don't think node that have updated should still process : [b_1][][block_data] and [b_1][b_2][block_data]. I think they should ban the people that didn't make the update.

However, announcing [b_BAD] should be monitored and user warned? (possible attack)

It's complicated to know if a version is a "bad" or not. What could be possible is that when a version is propagated by X numbers of blocks in Y latest periods we strongly recommend to update. COuld talk more about it in the separate discussion

When looking at some resources you pointed out, especially : https://github.com/bitcoin/bitcoin/pull/1526/files it feels like they are just making some conditions around the code and don't have a complex framework or something like that.

[sydhds]

Components have been added.

"""
At TS, I don't think node that have updated should still process : [b_1][][block_data] and [b_1][b_2][block_data]. I think they should ban the people that didn't make the update.
"""

That's an option yes! But maybe we want to have a delay in order to let everyone to update (even after [TS])?

"""
When looking at some resources you pointed out, especially : https://github.com/bitcoin/bitcoin/pull/1526/files it feels like they are just making some conditions around the code and don't have a complex framework or something like that.
"""

I'm afraid this could lead to complicated code if there multiple conditions like that but maybe I'm wrong...

[AurelienFT]

I'm afraid this could lead to complicated code if there multiple conditions like that but maybe I'm wrong...

I'm not saying that's a good way to go. I just find it wired that Bitcoin/Ethereum don't have generic and modular system and just place conditions everywhere.

[damip]

Maybe if we push the concept of "generic and modular system" we will end up realizing that it's not as easy/flexible as it sounds and spend more time handling edge cases than benefiting from the added modularity ?

[AurelienFT]

Maybe, yes.

[Ben-PH]

One question I had attending Damirs lessons, was regarding things like tweaking configs such as the interval times between periods. The context I have in mind, is that bitcoin is locked into the very conservative 10min. To app apply that context to massa, a 16s interval for massa decision is appropriate for today, but will that prove to be conservative in the future, and will we want/be able to propagate changes to such things to the network?

[sydhds]

Meta issue for Versioning: #3438

[sydhds]

After last week discussion, @Leo-Besancon and @sydhds will investigate the part that we previously called (Versioning info & trigger).

As bitcoin did with BIP-0034 & BIP-0009, we are going to share the work similarly:

• BIP-0034 for @Leo-Besancon (~ gathering stats on blocks for new version & switching between version)
• BIP-0009 for @sydhds (in a simpler version, no parallel deployment for now)
• At some point, we might need to work in a common PR together.

[damip]

To better understand what you wrote, I did the exercise on my side as well.

NOTE: this IS NOT meant to be implemented, it is just my personal notes so we can discuss

Versioning versions_config.rs file in the project

This file is hardwritten in the code since it is not meant to be changed by the user.

It lists two things:

// contains all active active versions so far (excluding the current transition version if any)
let active_version_history = vec![
    StableVersionInfo {        
        // version number
        id:0u32,
    
        // versions of the various modules
        address_version: 0u32,
        ledger_version: 0u32,
        bytecode_version: 0u32,
        block_version: 1u32,

        // timestamp (included) at which this version became active
        activated_at: MassaTime::from_millis(0),
    },
    StableVersionInfo {        
        // version number
        id: 1u32,
    
        // versions of the various modules
        address_version: 1u32,
        ledger_version: 3u32,
        bytecode_version: 5u32,
        block_version: 6u32,

        // timestamp (included) at which this version became active
        activated_at: MassaTime::from_millis(20000000),
    },
    StableVersionInfo {        
        // version number
        id: 2u32,
    
        // versions of the various modules
        address_version: 2u32,
        ledger_version: 3u32,
        bytecode_version: 6u32,
        block_version: 10u32,

        // timestamp (included) at which this version became active
        activated_at: MassaTime::from_millis(30000000),
    },
];


// contains an optional new version towards which we are transitionning (None if we are not transitionning)
let transition_version = Some(TransitionVersion {
        // version number
        id: 4u32,
        
        // timestamp (included) at which we start announcing this version
        start_announcing_at: MassaTime::from_millis(40000000),
        
        // trigger condition that must be reached during announcement
        trigger_condition: VersionTrigger::AnnouncementPercentage(Ratio::new(70, 100), MassaTime::from_millis(1000000000)) // 70% of blocks during X time
        
        // trigger timeout after which we cancel the version if the trigger condition was not reached
        trigger_timeout: MassaTime::from_millis(1000000),
        
        // delay after triggering before the version becomes active
        activation_delay: MassaTime::from_millis(1000000),
        
        // versions of the various modules once active
        address_version: 4u32,
        ledger_version: 4u32,
        bytecode_version: 6u32,
        block_version: 15u32,
        
        // state history with pairs (time at which we entered that status, state description)
        // this gives a history of all status changes that happened in the past
        // note that this can be pre-set as a non-empty vector in case the network needs a hard restart during an unfinished version transition
        state_history: Vec<(MassaTime, VersionTransitionState)> = vec![],
});

Note that we don't use slots but absolute timestamps to avoid any consequences of changing t0, T etc...

The version module

The version module is initialized with config.active_version_history and config.transition_version.
The module stores the active_version_history as is, and inits a state machine with the initial state history as given in the config.

Then, when a slot SCE-finalizes, the versioning machine is notified of the slot timestamp, and the slot content (block version announcement if present, or None if miss) in a synchronous way (blocking call in the execution worker).
The version module keeps track of the statistics it needs to detect triggers etc... to transition the state and add the transition to the state history.

The version module needs to be bootstrappable in order to sync transition machine states.

The version module can be included in the global final state hash but we must be careful:
people who haven't yet started announcing could desync from the people who started announcing because they didn't update yet even tho they are still compatible !!!
So maybe we should keep this as a followup.

The version module can be interrogated by external modules to ask stuff like "what's the address version at timestamp XX" or "what announcement should we put in blocks for timestamp XX". When this happens:
* first check in the transition state machine to deduce the versions given the state at that time
* as a fallback, check using the config.active_version_history for module versions

Important notes:

• the beginning of the announcement time is TIMER-BASED
• the trigger detection is based on the slot notifications coming from execution
• the trigger timeout is NOT TIMER-BASED : it transitions on slot notifications coming from execution
• the activation delay after triggering IS TIMER-BASED

[sydhds]

To better understand what you wrote, I did the exercise on my side as well.

NOTE: this IS NOT meant to be implemented, it is just my personal notes so we can discuss

Versioning versions_config.rs file in the project

This file is hardwritten in the code since it is not meant to be changed by the user.

It lists two things:

// contains all active active versions so far (excluding the current transition version if any)
let active_version_history = vec![
    StableVersionInfo {        
        // version number
        id:0u32,
    
        // versions of the various modules
        address_version: 0u32,
        ledger_version: 0u32,
        bytecode_version: 0u32,
        block_version: 1u32,

        // timestamp (included) at which this version became active
        activated_at: MassaTime::from_millis(0),
    },
    StableVersionInfo {        
        // version number
        id: 1u32,
    
        // versions of the various modules
        address_version: 1u32,
        ledger_version: 3u32,
        bytecode_version: 5u32,
        block_version: 6u32,

        // timestamp (included) at which this version became active
        activated_at: MassaTime::from_millis(20000000),
    },
    StableVersionInfo {        
        // version number
        id: 2u32,
    
        // versions of the various modules
        address_version: 2u32,
        ledger_version: 3u32,
        bytecode_version: 6u32,
        block_version: 10u32,

        // timestamp (included) at which this version became active
        activated_at: MassaTime::from_millis(30000000),
    },
];


// contains an optional new version towards which we are transitionning (None if we are not transitionning)
let transition_version = Some(TransitionVersion {
        // version number
        id: 4u32,
        
        // timestamp (included) at which we start announcing this version
        start_announcing_at: MassaTime::from_millis(40000000),
        
        // trigger condition that must be reached during announcement
        trigger_condition: VersionTrigger::AnnouncementPercentage(Ratio::new(70, 100), MassaTime::from_millis(1000000000)) // 70% of blocks during X time
        
        // trigger timeout after which we cancel the version if the trigger condition was not reached
        trigger_timeout: MassaTime::from_millis(1000000),
        
        // delay after triggering before the version becomes active
        activation_delay: MassaTime::from_millis(1000000),
        
        // versions of the various modules once active
        address_version: 4u32,
        ledger_version: 4u32,
        bytecode_version: 6u32,
        block_version: 15u32,
        
        // state history with pairs (time at which we entered that status, state description)
        // this gives a history of all status changes that happened in the past
        // note that this can be pre-set as a non-empty vector in case the network needs a hard restart during an unfinished version transition
        state_history: Vec<(MassaTime, VersionTransitionState)> = vec![],
});

Note that we don't use slots but absolute timestamps to avoid any consequences of changing t0, T etc...

The version module

The version module is initialized with config.active_version_history and config.transition_version. The module stores the active_version_history as is, and inits a state machine with the initial state history as given in the config.

Then, when a slot SCE-finalizes, the versioning machine is notified of the slot timestamp, and the slot content (block version announcement if present, or None if miss) in a synchronous way (blocking call in the execution worker). The version module keeps track of the statistics it needs to detect triggers etc... to transition the state and add the transition to the state history.

The version module needs to be bootstrappable in order to sync transition machine states.

The version module can be included in the global final state hash but we must be careful: people who haven't yet started announcing could desync from the people who started announcing because they didn't update yet even tho they are still compatible !!! So maybe we should keep this as a followup.

The version module can be interrogated by external modules to ask stuff like "what's the address version at timestamp XX" or "what announcement should we put in blocks for timestamp XX". When this happens: * first check in the transition state machine to deduce the versions given the state at that time * as a fallback, check using the config.active_version_history for module versions

Important notes:

• the beginning of the announcement time is TIMER-BASED
• the trigger detection is based on the slot notifications coming from execution
• the trigger timeout is NOT TIMER-BASED : it transitions on slot notifications coming from execution
• the activation delay after triggering IS TIMER-BASED

Glossary

• Network version is the version announced in block header

• Component versions are the version per component like address, keypair, vm...

• trigger (or threshold): condition to have a MipInfo accepted by the community (but not yet enabled)

• trigger timeout: timeout after which the MipInfo is considered rejected (or failed).

• Available states:

  • Defined: initial state
  • Started: We can start announcing this network version in block header - timer based (not important to reach consensus as it is a vote and it requires software update)
  • LockedIn: After successful trigger - still announcing this network version
  • Active: After trigger timestamp + activation delay (time based) - this becomes the current network version
  • Failed: After trigger timeout

Implementation proposal

The MipStore as it name implies already store a list of MipInfo & MipState. We plan to also add the statistic part & logic
(in order to keep track of trigger & trigger timeout). Note that the MipStore is a "Arc<RwLock<DATA>>" as it will be used by
the various component factories (for instance to create an address).

So in the execution module and after a block become final, we update the MipStore stats (in a blocking way) then continue one.
By updating the stats, we mean sending: (Slot timestamp, Option<(current: u32, advertising: Option)>). Using the slot timestamp, ensure that
the trigger (and the trigger timeout) is a consensus by all nodes (This means that the trigger and the trigger timeout are not timer based).
In order to have all nodes in sync (because of node various delays), the state is se to active after an activation delay (duration is required to be > 1 cycle).
Note that the current version and advertising version (send by the execution module), can be used to warn the node for if an update is needed.

When bootstrapping, the MipStore will be synced in order to the new nodes to be in sync with current version(s) and votes.
For now, we will not include the MipStore in the global final state hash.