diff --git a/governance/20221005-dynamic-inclusion-fee.md b/governance/20221005-dynamic-inclusion-fee.md index 1edb90aa..f9a85a14 100644 --- a/governance/20221005-dynamic-inclusion-fee.md +++ b/governance/20221005-dynamic-inclusion-fee.md @@ -21,6 +21,16 @@ The goal of this proposal is to come up with a formula incentivizing developers > 💬 We are not looking for perfectly precise math models this time, instead we are looking for a simple equation to calculate inclusion fees to reflect relative potential inclusion effort of a given transaction, including network transmission, memory usage etc. +#### Proposal + +In this FLIP we propose to replace the existing consistent inclusion fee charge into a dynamic charging pattern, in which total size of a transaction and its number of authoriers(signatures) will be factored in: + + = 287.1 * (6.39e-7 * **TxByteSize** + 3.29e-5 * **NumOfSignature** + 0.0023) * 1e-6 FLOW + +Details on how we collect metrics, how the final equation is derived and validated will be elaborated in the rest part of this FLIP. + +> 💬 Mathematical terms in this FLIP are from [this parent FLIP](https://github.com/onflow/flow/blob/master/flips/20211007-transaction-fees.md#terminology). + #### How Transactions are Created in Load Test To get performance metrics with a batch of fixed-sized transactions, we need to compose such transactions that we can control each of their: @@ -29,13 +39,13 @@ To get performance metrics with a batch of fixed-sized transactions, we need to * Total transaction size -* Number of authorizers +* Number of signatures * Number of keys of payer * Argument size * Comment size -In [PR 2816](https://github.com/onflow/flow-go/pull/2816), we added this feature in the loader app. We first add all indicated number of authorizers and keys of payer, compose transaction argument string, and at last leave all remaining bytes as comments. When these transactions are ready, the loader app will submit them to a given access cluster with give TPS. +In [PR 2816](https://github.com/onflow/flow-go/pull/2816), we added this feature in the loader app. We first add all indicated number of signatures and keys of payer, compose transaction argument string, and at last leave all remaining bytes as comments. When these transactions are ready, the loader app will submit them to a given access cluster with give TPS. #### How Transaction Size is Calculated @@ -45,7 +55,7 @@ According to our [code](https://github.com/onflow/flow-go/blob/master/engine/acc * Byte size of transaction code -* Number of authorizers +* Number of Signatures * Authorizer/payer keys The above fields are the only available information of one transaction which we can use to calculate inclusion fees, because we have to calculate it without executing one transaction. @@ -65,21 +75,33 @@ We checked localnet, benchnet and canary to collect metrics. Except that benchne chart for metrics from canary

-#### Data Modeling +Number of Signatures(one key for each authorizer) also demonstated a significant impact to TPS: -Assuming that with given configuration, relative inclusion fee is simply the inverse of stable peak TPS, now we have below linear equation with transaction size as X and the resulting Inclusion Fee as Y: +

+chart for metrics from localnet +
+chart for metrics from localnet +chart for metrics from canary +
+chart for metrics from canary +

- Y = C1 * X + C2 +#### Data Modeling -We used some existing math tool ([https://mycurvefit.com/](https://mycurvefit.com/) in our case), plugging in the metrics data we collected (simply deriving the inclusion fee by 1.0/TPS) from canary and we got the equation below. The reason we use canary data to derive the equation is that its hardware\network configuration is closer to mainnet compared with localnet. +Assuming that with given configuration, relative inclusion fee is simply the inverse of stable peak TPS, and for the number of signatures, we want to charge it linearly by that number, now we have below linear equation, in which C1, C2, C3 are the constant coefficients in the equation: + = C1 * TxByteSize + C2 * NumOfSignature + C3 + - Inclusion Fee Factor = 6.370641e-7 * TxByteSize + 2.585227e-3 +We used some existing math tool ([LINEST from Google Sheet](https://support.google.com/docs/answer/3094249?hl=en) in our case), plugging in the metrics data we collected (simply deriving the inclusion fee by 1.0/TPS) from canary and we got the equation below. The reason we use canary data to derive the equation is that its hardware\network configuration is closer to mainnet compared with localnet. -We’d like to charge average sized transactions by 1e-6 FLOW, and according to [this](https://console.cloud.google.com/bigquery?sq=1039464903934:01f22b6020e1403690eeb67822d56c87) BigQuery, we can tell that the average size of production transactions are around 1800B, plugging this into the equation above and now we have the final equation as below: + = 6.39e-7 * TxByteSize + 3.29e-5 * NumOfSignature + 0.0023 + - Inclusion Fee Factor = 267.956977406 * (6.370641e-7 * TxByteSize + 2.585227e-3) +We’d like to charge average sized transactions by 1.0 factor (1e-6 FLOW), which means we are trying to keep inclusion fees of average sized transactions the same as before. According to [this](https://console.cloud.google.com/bigquery?sq=1039464903934:01f22b6020e1403690eeb67822d56c87) BigQuery, we can tell that the average size of production transactions are around 1800B, and according to [this](https://console.cloud.google.com/bigquery?sq=1039464903934:8ce5d7d1677a4c5d99aa3341b46375cd) BigQuery, majority of transaction only have 1 authorizers, unfortunately on GCP we don't record number of keys per authorizer, so we are assuming that there's only one key for each authorizer. After plugging these data into the equation above and now we have the final equation as below: + = 287.1 * (6.39e-7 * TxByteSize + 3.29e-5 * NumOfSignature + 0.0023) + #### Model Validation To make sure our mathematical model above works as expected with real mainnet transactions, we ran the equation against sampled transactions in BigQuery where our recent production transactions are stored. @@ -87,39 +109,40 @@ To make sure our mathematical model above works as expected with real mainnet tr * New Inclusion Fee calculation in BigQuery - From [this](https://console.cloud.google.com/bigquery?sq=1039464903934:9f7c25b503cf4fcdaf2301d2e51f539b) BigQuery where we calculate new inclusion fees for sampled transactions with different sizes, we can see that the new inclusion fee factor of transactions around 1800B are around 1.0 (or 1e-6 FLOW), and the smaller one tx is, the less the inclusion fee is, and vice versa. Listing several pieces of the result as below: + From [this](https://console.cloud.google.com/bigquery?sq=1039464903934:9f7c25b503cf4fcdaf2301d2e51f539b) BigQuery where we calculate new inclusion fees for sampled transactions with different sizes, we can see that the new inclusion effort of transactions around 1800B are around 1.0 (or 1e-6 FLOW), and the smaller one tx is, the less the inclusion fee is, and vice versa. Listing several pieces of the result as below. Note that there's no number of keys per authorizer info on BigTable.
Transaction with size less than average -| Tx Size | New Inclusion Fee Factor | -|:----------:|:-------------:| -| 793 | 0.828099289 | -| 809 | 0.8308305813 | -| 810 | 0.8310012871 | -| 811 | 0.8311719928 | -| 821 | 0.8328790505 | +| Tx Size(Bytes) | Num of Auth | New Inclusion Effort | +|:----------:|:-------------:|:-------------:| +| 742 | 1 | 0.8059 | +| 757 | 1 | 0.8086 | +| 763 | 1 | 0.8097 | +| 766 | 1 | 0.8103 | +| 767 | 1 | 0.8104 | + Transaction with size around average -| Tx Size | New Inclusion Fee Factor | -|:----------:|:-------------:| -| 1755 | 0.9923182403 | -| 1786 | 0.9976101192 | -| 1804 | 1.000682823 | -| 1843 | 1.007340348 | -| 1865 | 1.011095875 | +| Tx Size(Bytes) | Num of Auth | New Inclusion Effort | +|:----------:|:-------------:|:-------------:| +| 1756 | 1 | 0.9919 | +| 1786 | 1 | 0.9974 | +| 1804 | 3 | 1.0196 | +| 1816 | 1 | 1.0029 | +| 1843 | 2 | 1.0173 | Transaction with size above average -| Tx Size | New Inclusion Fee Factor | -|:----------:|:-------------:| -| 4538 | 1.4673924 | -| 4801 | 1.512288018 | -| 5184 | 1.577668328 | -| 5823 | 1.686749315 | -| 7880 | 2.037891086 | +| Tx Size(Bytes) | Num of Auth | New Inclusion Effort | +|:----------:|:-------------:|:-------------:| +| 3386 | 1 | 1.2909 | +| 3387 | 1 | 1.2911 | +| 3412 | 2 | 1.3051 | +| 3412 | 1 | 1.2957 | +| 3418 | 1 | 1.2968 |
* Min\Max charges @@ -128,35 +151,35 @@ Transaction with size above average
    1. Inclusion Fee of Empty Transaction
      2. - Inclusion Fee Factor(empty tx) - = 267.956977406 * (6.370641e-7 * **0** + 2.585227e-3) - = 0.69272961282 -After applying this new inclusion fee factor onto the existing inclusion effort charge(1e-6 FLOW), we get 1e-6 * 0.69272961282 as the new charge. + Inclusion Effort(empty tx) + = 287.1 * (6.39e-7 * 0 + 3.29e-5 * 1 + 0.0023) + = 0.6697 +After applying this new inclusion effort onto the existing inclusion effort charge(1e-6 FLOW), we get 1e-6 * 0.6697 as the new charge.
    2. Inclusion Fee of Maximal Transaction
      3. -According to the [code](https://github.com/onflow/flow-go/blob/master/model/flow/constants.go#L32), the max possible byte size of one transaction is 1500000B, now we have: +According to the [code](https://github.com/onflow/flow-go/blob/master/model/flow/constants.go#L32), the max possible byte size of one transaction is 1500000B, and if we simply set the number of signatures as the currently max value of authorizers, 14, in our BigQuery, now we have: - Inclusion Fee Factor(largest tx possible) - = 267.956977406 * (6.370641e-7*1500000 + 2.585227e-3) - = 256.751385588 + Inclusion Effort(largest tx possible) + = 287.1 * (6.39e-7 * 1500000 + 3.29e-5 * 14 + 0.0023) + = 275.9779 -After applying this new inclusion fee factor onto the existing inclusion effort charge(1e-6 FLOW), we get 1e-6 * 256.751385588 as the new charge. +After applying this new inclusion effort onto the existing inclusion effort charge(1e-6 FLOW), we get 1e-6 * 275.9779 as the new charge.
* New inclusion fees for different types of transactions - It will also be useful to see inclusion fee changes with the new equation, for some typical types of transactions we have. We are assuming there is only one authorizer in these transactions which is the most common scenario. + It will also be useful to see inclusion fee changes with the new equation, for some typical types of transactions we have. We are assuming there is only one signature in these transactions which is the most common scenario. Assuming all calculation below has only 1 signature.
    1. Create Account transaction
      2. Cadence script we use for the calculation: [link](https://github.com/onflow/flow-go/blob/master/integration/benchmark/scripts/createAccountsTransaction.cdc) - size: 984B - New inclusion fee factor - = 267.956977406 * (6.370641e-7*984 + 2.585227e-3) - = 0.86070409114 + New inclusion effort + = 287.1 * (6.39e-7 * 984 + 3.29e-5 * 1 + 0.0023) + = 0.8503 -The new fee is roughly **14% less** than the existing inclusion charge (1e-6 FLOW) +The new fee is roughly **15% less** than the existing inclusion charge (1e-6 FLOW)
    2. Token transfer transaction
      3. @@ -164,22 +187,22 @@ The new fee is roughly **14% less** than the existing inclusion charge (1e-6 FLO Cadence script we use for the calculation: [link](https://github.com/onflow/flow-go/blob/master/integration/benchmark/scripts/tokenTransferTransaction.cdc) - size: 718B - New inclusion fee factor - = 267.956977406 * (6.370641e-7*718 + 2.585227e-3) - = 0.81529635615 + New inclusion effort + = 287.1 * (6.39e-7 * 718 + 3.29e-5 * 1 + 0.0023) + = 0.8015 -The new fee is roughly **18% less** than the existing inclusion charge (1e-6 FLOW) +The new fee is roughly **20% less** than the existing inclusion charge (1e-6 FLOW)
    3. NFT transfer transaction
      4. Cadence script we use for the calculation: [link](https://github.com/onflow/flow-go/blob/master/fvm/fvm_bench_test.go) - size 818B - New inclusion fee factor - = 267.956977406 * (6.370641e-7*818 + 2.585227e-3) - = 0.83236693322 + New inclusion effort + = 287.1 * (6.39e-7 * 818 + 3.29e-5 * 1 + 0.0023) + = 0.8198 -The new fee is roughly **16% less** than the existing inclusion charge (1e-6 FLOW) +The new fee is roughly **18% less** than the existing inclusion charge (1e-6 FLOW)
@@ -192,30 +215,60 @@ The new fee is roughly **16% less** than the existing inclusion charge (1e-6 FLO ![mainnet tx size histogram ](20221005-dynamic-inclusion-fee/tx-size-histogram-gcp.png) So most transactions are with sizes between 500 - 2000 bytes, and there are much fewer number of transactions with sizes above 4000B. The average transaction size from our mainnet is 1800B, which is marked on the diagram too. We used this value to calibrate our inclusion fees as mentioned above. +
+ +* Distribution of mainnet number of authorizers + + From [BigQuery](https://console.cloud.google.com/bigquery?sq=1039464903934:8ce5d7d1677a4c5d99aa3341b46375cd&project=dapperlabs-data) querying all our mainnet transactions, we can see the distribution of number of authorizers in below table: + +
+ +| Number of Authorizers | Count of Transactions | +|:----------:|:-------------:| +| 1 | 216049901 | +| 2 | 41611744 | +| 3 | 7548509 | +| 4 | 1843914 | +| 5 | 869991 | +| 6 | 528975 | +| 7 | 442313 | +| 8 | 403601 | +| 9 | 272371 | +| 10 | 119965 | +| 11 | 181782 | +| 14 | 53 | + +
#### Limitations -* Using metrics from canary to estimate mainnet fees +* Using metrics from canary to estimate mainnet fees. Since we were using the loader app to stress test certain environments (localnet, canary), it is impractical to run a similar stress test to mainnet which will impact production traffic. Thus, we are only able to apply the mathematical equation derived from those environments to mainnet directly. It is fine in our use cases because we are not looking for precise mathematical calculation to new inclusion fees, and we are looking for a better calculation than the current constant inclusion charge: we should charge less for smaller transactions and more for bigger transactions. So as long as the new equation demonstrates this characteristic on production transactions, we will treat it as a working math model. -* Benchnet didn’t give stable peak TPS metrics +* Benchnet didn’t give stable peak TPS metrics. We also tried the loader app on benchnet configurations. However the existing benchnet didn’t give us stable peak TPS and it made it unusable to collect performance related metrics. The reason is still under investigation, but since we were still able to get metrics from canary, it is not a blocking issue for us. -* Other factors impacting stable peak TPS -In our benchmark testing on both localnet and canary environments, we actually also found that another two factors can also impact the stable peak TPS: number of authorizers and number of keys per payer. However we didn’t include these two factors into our final equation, because they don’t seem to be major factors impacting performance, thus including them will make our equation unnecessarily complex. +* Other factors impacting stable peak TPS. +In our benchmark testing on both localnet and canary environments, we actually also found that number of keys can also impact the stable peak TPS. However the actual impact measured from data collection was not major enough for us to include it into the equation. + +* More descriptive Cadence code will be discouraged. +For most cases, Cadence code takes a big portion of one transaction. With transaction size as one factor to charge for inclusion fees, it may discourage Cadence authors to write less readable and descriptive smart contract code on Flow. **Future Work** -* Collecting more data points for a more accurate model +* Collecting more data points for a more accurate model. Due to the limitation mentioned, it was quite a cumbersome process to collect even one single performance metrics data point, so we were only able to collect a limited amount of data for modeling purposes. We don’t need a super precise model for now, but if in the future when higher precision is required, we will need to collect more data to support the precision we want from a model. -* Self-adaptive Inclusion Fee model +* Self-adaptive Inclusion Fee model. In this FLIP we propose a fixed-coefficient linear equation, however the hardware configuration, network capabilities and even Flow itself may change. In these cases, the coefficients we have now may not give optimal inclusion fee calculations accordingly. It will be ideal if we can have a self-adaptive mechanism, to automatically adjust parameters of the inclusion fee model, to continuously produce appropriate inclusion fee calculation. -* Checking balance of payer to see if Inclusion Fees can be covered in access\collection nodes +* Checking balance of payer to see if Inclusion Fees can be covered in access\collection nodes. When we have a dynamic inclusion fee, we will have to check if a certain payer’s account balance can cover this new inclusion fee. -* More infrastructure setup is nice-to-have for easier metrics collection in future +* Non-linear Inclusion Fees Calculation for extra large transactions. +We might want to charge extra more, like quadratically for super large transactions, whose total size exceeds a certain cutoff size, to encourage people to keep their transaction size within a reasonable limit. Transaction smart contract code compress may be something worth investigation. + +* More infrastructure setup is nice-to-have for easier metrics collection in future. As previously mentioned, for almost all existing testing environments(localnet, benchnet, canary), we had to do a trial-and-error method to collect one stable peak TPS with a given configuration. This process is a very cumbersome manual process and it does not scale with the number of metrics to collect. So in future when we need to collect similar metrics, we may want to have an easier or even automated process. It will enable us to collect relatively comprehensive metrics data for our future data related work. At the time of writing, the automation team has been actively working on this already. #### References diff --git a/governance/20221005-dynamic-inclusion-fee/num-auth-tps-canary.png b/governance/20221005-dynamic-inclusion-fee/num-auth-tps-canary.png new file mode 100644 index 00000000..3ff1cfbb Binary files /dev/null and b/governance/20221005-dynamic-inclusion-fee/num-auth-tps-canary.png differ diff --git a/governance/20221005-dynamic-inclusion-fee/num-auth-tps-localnet.png b/governance/20221005-dynamic-inclusion-fee/num-auth-tps-localnet.png new file mode 100644 index 00000000..beb8f314 Binary files /dev/null and b/governance/20221005-dynamic-inclusion-fee/num-auth-tps-localnet.png differ