proposal: Fee Market for Storage

[brooksprumo]

This PR continues the discussion from #27029, as a proper proposal.

Update: This proposal has been moved over to the SIMD repo/process here:
solana-foundation/solana-improvement-documents#8

[brooksprumo]

Tagging the folks that already commented over in the other issue:
@jstarry @taozhu-chicago @joncinque

Please tag in others that would be valuable for the discussion. TIA!

[aeyakovenko]

my gut on how to price storage is to allow Xkb per block, and any remaining storage goes into a "large slab allocator". Devs can request Zmb from the slab allocator that runs an auction every minute or something, then sub allocate from the slab.

[aeyakovenko]

@brooksprumo so the other idea

1. network sets a maximum target state size in GB for the state
2. when the state hits that size, the lowest accounts by lamport balance compress - gets put into a Merkle tree
3. compressing mans the account stays in the index, but the data is merkelized, like the NFT merkle tree
4. anyone can provide the data and decompress the account as long as the lamport balance > min

We need some way to deal with the corner case of the hard cap, the idea I have is that

1. When state size hits 100% of the target maximum size, price doubles at every half way point to 120%, so 110, 115, 117.5...
2. at 120% of the target state size allocations fail
3. every epoch the state shrinks back down to at most 80% of the target maximum.

[brooksprumo]

2. when the state hits that size, the lowest accounts by lamport balance compress - gets put into a Merkle tree

Sounds like this would require scanning all the accounts to find those particular accounts, which may be expensive.

3. every epoch the state shrinks back down to at least 80% of the target maximum.

I assume "at most" instead of "at least", yes? Is this the thought of when to perform the compression?

[aeyakovenko]

3. when the state hits that size, the lowest accounts by lamport balance compress - gets put into a Merkle tree

Sounds like this would require scanning all the accounts to find those particular accounts, which may be expensive.

Yea, we would do this asynchronously.

3. every epoch the state shrinks back down to at least 80% of the target maximum.

I assume "at most" instead of "at least", yes? Is this the thought of when to perform the compression?

Yes, correct.

[aeyakovenko]

@brooksprumo in theory, once the state fills up and lowest price is actually > default, it means that there is more economically useful state that can fit into the chain. Validators should be motivated to bump up the capacity to capture all of it.

[brooksprumo]

This proposal has been moved over to the SIMD repo/process here:
solana-foundation/solana-improvement-documents#8