Transaction Fee Schedule For Workers

[bennetyee]

We need to establish mechanisms and a fee schedule to split up the gas earned from processing transactions among the compute committee nodes, storage committee nodes, root hash committee, etc, including a tax for common infrastructure (merkle tree maintenance, etc).

Details

We need to design & implement mechanisms to (flexibly) have a fee schedule. The EVM compute nodes have the gas fee schedule for EVM operations, and this could be similar; this could also be maintained as (read-mostly) data in a contract changes to which would be decided by vote by the community/foundation, though compiling in the fee schedule is more efficient. Design decisions need to be made regarding:

• Which system component divies up the earned gas (compute committee, merge committee, ...)? The compute committee already has to know about the account balance location in the merkle tree, but the merge committee decides whether a batch would be reverted due to data-dependency induced conflict set incompatibilities, and we need to decide whether (and how) such a batch would pay. If the merge committee has to know this, then it will also have to know (more) about the structure of the merkle tree.

• What is a fair division for the gas earned? This is a policy question and can be decided later, but the mechanisms for accounting for work done by various system entities (registry, committee scheduler, transaction scheduler, storage, merge, etc) has to be able to support the policy that we decide on.

Linked Issues

• Design docs discussing the trade-offs of choices. Implementation.
• Runtime gas disbursement
• Special case account balance handling to reduce conflicts

[kostko]

The compute committee already has to know about the account balance location in the merkle tree

I assume you mean that it needs to know this to be able to reduce conflicts and implement CRDT semantics for certain subtrees? This is true for scalability reasons, but I don't think this is the only way to implement fee disbursement.

One other way is to update what is returned by the runtime as the "computed batch" which currently just contains the outputs, new state root and write log (this will change slightly when we implement merging but that is orthogonal).

We could add another item, signalling the amount that should be disbursed to the workers (this would of course be part of state that is checked via DD). I think this would be more explicit than extracting this information from the state data structure and assuming certain state semantics from all runtimes (again, we must design this to be more general and work with runtimes that are not runtime-ethereum).

[bennetyee]

Yes, I think that would work. I am worried about having to put txns into batches when they might otherwise be independent because we are not taking advantage of the commutivity of additions/subtractions -- txns that do subtractions from account balances may have to be colocated in the same batch, unless we trust the pre-execution to get the amounts to be deducted and can prove that there is never a negative excursion (or allow "temporary loans" for that).

So if disbursement -- which are purely additive and thus safe wrt negative excursions, esp since they are paid to EOAs which cannot run code on receipt -- is a separate txn output to be handled during merge fix-up, but subtraction is immediate, then all txns with the same txn.origin has to be placed into the same batch within a schedule, or be in different schedules. This might be okay -- as long as disbursements do not force batch consolidation, this could be good enough.

It would be nice to generalize this for other balance-changing operations, like send, transfer, etc. (are these all implemented via the CALL evm opcode?) This might be more complicated, since the target of the call can do BALANCE which forces the serial order to be consistent with the returned value. I need to think more about how this might work.

If no txn invokes BALANCE and we take advantage of commutivity, we still have the follow potential drawback. Users might be used to being able to put in a bunch of transactions into the pool, some of which result in their balance increasing and some in it decreasing, and expect that (most of the time) they would all go through because the additive txns are interspersed among the subtractive txns. If we separate and delay additions until the end of the schedule, however, such a mix of txns will almost surely not all complete -- the last N will revert -- due to running out of funds. If we do disbursements only for gas -- and not generalize for CALL -- then running out of funds should be less likely, but txns that result in value transfers w/ overlaps among src/dest accounts would definitely force those txns to colocate.

[kostko]

So one major question is - where are the accounts of the entities running the compute nodes? One possible way is to have them inside the runtime. But since compute nodes may run multiple runtimes, does this mean that they need to have accounts with each runtime in addition to being registered in our registry?

We will already have staking and staking accounts for entities/compute nodes. Can we also disburse fees that way, independent of any runtime?

[kostko]

Replying to myself, could we have a combination of both? (In the following text, the participants are the staked nodes.)

• The runtime emits how much gas a certain batch used through a standardized interface (e.g., as part of the computed result) and moves those fees into "escrow" internally.
• After the merge committee submits commitments, the BFT committee records how much gas fees the participants have accrued (a ledger should be kept for each runtime and the number would always represent that runtime's internal gas token).
• There is a mechanism for the participants to convert these 1:1 inside the runtime, by transferring them to a specific account inside the runtime. This would require a way for the runtime to query the balance kept by the BFT committee of an arbitrary participant outside the runtime and a way to signal withdrawal as part of the finalized root.

Care needs to be taken to prevent parallel withdrawal (conflict detection at merge and BFT committees).

[kostko]

An alternative design is to keep participants' gas accounts entirely within the runtime, but for that we need to introduce special semantics for updating storage state without introducing conflicts (e.g., increment and subtract operations).

[lostnfound]

Another alternative is

• Entities have two accounts, stake and runtime. The stake account is for deposit and slashing, and the runtime account is for receiving rewards and fees. The stake account state is stored in the consensus/BFT committee and the runtime account state is stored in the Storage committee.
• TX scheduler committee checks that the runtime account has enough balance to pay for the transaction/gas fee.
• Compute committee sends "reimbursement requests" for computing and storage committees' work along with the results to the merge committee. (Compute committee does not have to inquire/update the runtime accounts.)
• If the merge committee doesn't find any discrepancy, the reimbursement requests are sent to the consensus (BFT) committee.
• Consensus committee puts the reimbursement payment transactions in the blocks. Note that this does not update the runtime account balances.
• As the Storage committee receives these new blocks to store, the storage committee updates the runtime transactions according to the payment transactions in the blocks.
• If the Storage committee doesn't update the runtime account balances in a timely manner, the TX scheduler will detect this. When the TX scheduler checks the balance is enough, it sends the Storage committee (root hash/block height, runtime account info). The Storage committee should update the runtime account balances up to this root hash/block height and then return the balances so that it will not have discrepancy.

In other words, only the consensus/BFT committee updates the stake accounts, and only the Storage committee updates the runtime accounts.