Use production configuration for fuel-core during benches
#1227
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Voxelot
approved these changes
Jun 22, 2023
bvrooman
approved these changes
Jun 23, 2023
Merged
bvrooman
pushed a commit
that referenced
this pull request
Jun 29, 2023
Related issues: - Ref #1134 These benches demonstrate the performance of: - Inserting 1 key/value pair into the RocksDB table that contains `n` state entries for a single contracts. `n = [0, 1, 10, 100, 1,000, 10,000, 100,000, 1,000,000]` - Inserting 1 key/value pair into the RocksDB table that contains 1 state entries with `n` existing contracts. `n = [0, 1, 10, 100, 1,000, 10,000, 100,000, 1,000,000]` Bench results: ``` state single contract/insert state with 0 preexisting entries time: [11.991 µs 12.132 µs 12.310 µs] change: [-4.7172% -1.6523% +1.0884%] (p = 0.29 > 0.05) No change in performance detected. Found 9 outliers among 100 measurements (9.00%) 5 (5.00%) high mild 4 (4.00%) high severe state single contract/insert state with 1 preexisting entries time: [25.009 µs 28.694 µs 32.962 µs] Found 6 outliers among 100 measurements (6.00%) 5 (5.00%) high mild 1 (1.00%) high severe state single contract/insert state with 10 preexisting entries time: [39.002 µs 41.617 µs 44.324 µs] Found 1 outliers among 100 measurements (1.00%) 1 (1.00%) high severe state single contract/insert state with 100 preexisting entries time: [60.968 µs 63.985 µs 67.259 µs] Found 4 outliers among 100 measurements (4.00%) 3 (3.00%) high mild 1 (1.00%) high severe state single contract/insert state with 1,000 preexisting entries time: [87.465 µs 91.207 µs 95.394 µs] change: [+0.4239% +4.5684% +8.9542%] (p = 0.04 < 0.05) Change within noise threshold. Found 7 outliers among 100 measurements (7.00%) 6 (6.00%) high mild 1 (1.00%) high severe state single contract/insert state with 10,000 preexisting entries time: [117.22 µs 121.38 µs 125.93 µs] Found 6 outliers among 100 measurements (6.00%) 6 (6.00%) high mild state single contract/insert state with 100,000 preexisting entries time: [145.05 µs 148.30 µs 151.60 µs] change: [-3.4471% -0.8145% +1.9661%] (p = 0.56 > 0.05) No change in performance detected. state single contract/insert state with 1,000,000 preexisting entries time: [186.36 µs 190.05 µs 193.80 µs] Found 2 outliers among 100 measurements (2.00%) 2 (2.00%) high mild state multiple contracts/insert state with 0 preexisting entries time: [11.957 µs 12.080 µs 12.237 µs] change: [-19.431% -7.6642% +0.8636%] (p = 0.27 > 0.05) No change in performance detected. Found 9 outliers among 100 measurements (9.00%) 5 (5.00%) high mild 4 (4.00%) high severe state multiple contracts/insert state with 1 preexisting entries time: [11.872 µs 12.008 µs 12.199 µs] Found 5 outliers among 100 measurements (5.00%) 2 (2.00%) high mild 3 (3.00%) high severe state multiple contracts/insert state with 10 preexisting entries time: [12.189 µs 12.252 µs 12.320 µs] Found 4 outliers among 100 measurements (4.00%) 1 (1.00%) high mild 3 (3.00%) high severe state multiple contracts/insert state with 100 preexisting entries time: [12.538 µs 12.691 µs 12.867 µs] Found 8 outliers among 100 measurements (8.00%) 5 (5.00%) high mild 3 (3.00%) high severe state multiple contracts/insert state with 1,000 preexisting entries time: [13.041 µs 13.278 µs 13.640 µs] change: [-1.9317% +1.1895% +4.0288%] (p = 0.46 > 0.05) No change in performance detected. Found 2 outliers among 100 measurements (2.00%) 2 (2.00%) high severe state multiple contracts/insert state with 10,000 preexisting entries time: [13.350 µs 13.529 µs 13.729 µs] Found 3 outliers among 100 measurements (3.00%) 2 (2.00%) high mild 1 (1.00%) high severe state multiple contracts/insert state with 100,000 preexisting entries time: [13.973 µs 14.312 µs 14.719 µs] change: [+4.3928% +9.0688% +14.995%] (p = 0.00 < 0.05) Performance has regressed. Found 6 outliers among 100 measurements (6.00%) 2 (2.00%) high mild 4 (4.00%) high severe state multiple contracts/insert state with 1,000,000 preexisting entries time: [13.891 µs 14.096 µs 14.314 µs] Found 4 outliers among 100 measurements (4.00%) 3 (3.00%) high mild 1 (1.00%) high severe ``` **Conclusion:** For inserting `n` state entries for the same contract, the performance appears to be logarithmic with `n`. For inserting 1 state entry for a contract, with `n` existing contracts, the performance appears to be fairly constant, i.e., the number of existing contracts does not have an impact on the performance of inserting state for a contract. Note that these benchmarks were taken after the change to use RocksDB in benches (#1227).
Merged
xgreenx
added a commit
that referenced
this pull request
Jul 17, 2023
## Release v0.20.0 The release brings a couple of new breaking changes from the [`fuel-vm 0.35.0`](https://github.com/FuelLabs/fuel-vm/releases/tag/v0.35.0) with bugfixes. Check the description of the VM release for more details. The `fuel-core` release mostly improved the internal codebase but also brought some breaking changes: - Removed `Trigger::Hybrid` PoA block trigger mode. Only `Trigger::Instante` and `Trigger::Interval` are available for block production now. The main mode for testnets and mainnet will be `Interval`. - Removed support for `OpaqueReceipt` and the `Receipt` type doesn't have the `raw_payload` field anymore. - A `Receipt` type got two new variants: `Mint` and `Burn`. The corresponding opcodes emit these new events. - The `AssetId` is derived from `ContractId` and additional nonce. So the `ContractId` and `AssetId` can't be the same anymore. ## What's Changed * bump rocksdb to enable compiling with GCC 13 by @segfault-magnet in #1219 * setting peer reputation params by @leviathanbeak in #1202 * Take into account the actually used gas by the transactions and fetch more transaction by @xgreenx in #1223 * Use production configuration for `fuel-core` during benches by @xgreenx in #1227 * Speedup and stabilize unit and integration tests by @xgreenx in #1231 * test: State benchmarks by @bvrooman in #1226 * Remove hybrid PoA block trigger mode by @Dentosal in #1232 * test: Benchmark contract state insertions with DB vs. DB transactions by @bvrooman in #1230 * multiplatform docker builds by @Voxelot in #1233 * Fix typo in architecture.md by @eltociear in #1241 * Expose gas cost in chain info by @MitchTurner in #1244 * Reuse calculated tx id in executor by @MitchTurner in #1248 * Fix multi-platform images by @Voxelot in #1251 * Add logging of the long GraphQL queries for future debug by @MitchTurner in #1250 * Reused `CheckedTransaction` from transaction pool in the executor by @xgreenx in #1249 * Bump `fuel-vm` to `0.35.0` version by @xgreenx in #1256 ## New Contributors * @segfault-magnet made their first contribution in #1219 * @eltociear made their first contribution in #1241 * @MitchTurner made their first contribution in #1244 **Full Changelog**: v0.19.1...v0.20.0
crypto523
added a commit
to crypto523/fuel-core
that referenced
this pull request
Oct 7, 2024
Related issues: - Ref FuelLabs/fuel-core#1134 These benches demonstrate the performance of: - Inserting 1 key/value pair into the RocksDB table that contains `n` state entries for a single contracts. `n = [0, 1, 10, 100, 1,000, 10,000, 100,000, 1,000,000]` - Inserting 1 key/value pair into the RocksDB table that contains 1 state entries with `n` existing contracts. `n = [0, 1, 10, 100, 1,000, 10,000, 100,000, 1,000,000]` Bench results: ``` state single contract/insert state with 0 preexisting entries time: [11.991 µs 12.132 µs 12.310 µs] change: [-4.7172% -1.6523% +1.0884%] (p = 0.29 > 0.05) No change in performance detected. Found 9 outliers among 100 measurements (9.00%) 5 (5.00%) high mild 4 (4.00%) high severe state single contract/insert state with 1 preexisting entries time: [25.009 µs 28.694 µs 32.962 µs] Found 6 outliers among 100 measurements (6.00%) 5 (5.00%) high mild 1 (1.00%) high severe state single contract/insert state with 10 preexisting entries time: [39.002 µs 41.617 µs 44.324 µs] Found 1 outliers among 100 measurements (1.00%) 1 (1.00%) high severe state single contract/insert state with 100 preexisting entries time: [60.968 µs 63.985 µs 67.259 µs] Found 4 outliers among 100 measurements (4.00%) 3 (3.00%) high mild 1 (1.00%) high severe state single contract/insert state with 1,000 preexisting entries time: [87.465 µs 91.207 µs 95.394 µs] change: [+0.4239% +4.5684% +8.9542%] (p = 0.04 < 0.05) Change within noise threshold. Found 7 outliers among 100 measurements (7.00%) 6 (6.00%) high mild 1 (1.00%) high severe state single contract/insert state with 10,000 preexisting entries time: [117.22 µs 121.38 µs 125.93 µs] Found 6 outliers among 100 measurements (6.00%) 6 (6.00%) high mild state single contract/insert state with 100,000 preexisting entries time: [145.05 µs 148.30 µs 151.60 µs] change: [-3.4471% -0.8145% +1.9661%] (p = 0.56 > 0.05) No change in performance detected. state single contract/insert state with 1,000,000 preexisting entries time: [186.36 µs 190.05 µs 193.80 µs] Found 2 outliers among 100 measurements (2.00%) 2 (2.00%) high mild state multiple contracts/insert state with 0 preexisting entries time: [11.957 µs 12.080 µs 12.237 µs] change: [-19.431% -7.6642% +0.8636%] (p = 0.27 > 0.05) No change in performance detected. Found 9 outliers among 100 measurements (9.00%) 5 (5.00%) high mild 4 (4.00%) high severe state multiple contracts/insert state with 1 preexisting entries time: [11.872 µs 12.008 µs 12.199 µs] Found 5 outliers among 100 measurements (5.00%) 2 (2.00%) high mild 3 (3.00%) high severe state multiple contracts/insert state with 10 preexisting entries time: [12.189 µs 12.252 µs 12.320 µs] Found 4 outliers among 100 measurements (4.00%) 1 (1.00%) high mild 3 (3.00%) high severe state multiple contracts/insert state with 100 preexisting entries time: [12.538 µs 12.691 µs 12.867 µs] Found 8 outliers among 100 measurements (8.00%) 5 (5.00%) high mild 3 (3.00%) high severe state multiple contracts/insert state with 1,000 preexisting entries time: [13.041 µs 13.278 µs 13.640 µs] change: [-1.9317% +1.1895% +4.0288%] (p = 0.46 > 0.05) No change in performance detected. Found 2 outliers among 100 measurements (2.00%) 2 (2.00%) high severe state multiple contracts/insert state with 10,000 preexisting entries time: [13.350 µs 13.529 µs 13.729 µs] Found 3 outliers among 100 measurements (3.00%) 2 (2.00%) high mild 1 (1.00%) high severe state multiple contracts/insert state with 100,000 preexisting entries time: [13.973 µs 14.312 µs 14.719 µs] change: [+4.3928% +9.0688% +14.995%] (p = 0.00 < 0.05) Performance has regressed. Found 6 outliers among 100 measurements (6.00%) 2 (2.00%) high mild 4 (4.00%) high severe state multiple contracts/insert state with 1,000,000 preexisting entries time: [13.891 µs 14.096 µs 14.314 µs] Found 4 outliers among 100 measurements (4.00%) 3 (3.00%) high mild 1 (1.00%) high severe ``` **Conclusion:** For inserting `n` state entries for the same contract, the performance appears to be logarithmic with `n`. For inserting 1 state entry for a contract, with `n` existing contracts, the performance appears to be fairly constant, i.e., the number of existing contracts does not have an impact on the performance of inserting state for a contract. Note that these benchmarks were taken after the change to use RocksDB in benches (FuelLabs/fuel-core#1227).
crypto523
added a commit
to crypto523/fuel-core
that referenced
this pull request
Oct 7, 2024
## Release v0.20.0 The release brings a couple of new breaking changes from the [`fuel-vm 0.35.0`](https://github.com/FuelLabs/fuel-vm/releases/tag/v0.35.0) with bugfixes. Check the description of the VM release for more details. The `fuel-core` release mostly improved the internal codebase but also brought some breaking changes: - Removed `Trigger::Hybrid` PoA block trigger mode. Only `Trigger::Instante` and `Trigger::Interval` are available for block production now. The main mode for testnets and mainnet will be `Interval`. - Removed support for `OpaqueReceipt` and the `Receipt` type doesn't have the `raw_payload` field anymore. - A `Receipt` type got two new variants: `Mint` and `Burn`. The corresponding opcodes emit these new events. - The `AssetId` is derived from `ContractId` and additional nonce. So the `ContractId` and `AssetId` can't be the same anymore. ## What's Changed * bump rocksdb to enable compiling with GCC 13 by @segfault-magnet in FuelLabs/fuel-core#1219 * setting peer reputation params by @leviathanbeak in FuelLabs/fuel-core#1202 * Take into account the actually used gas by the transactions and fetch more transaction by @xgreenx in FuelLabs/fuel-core#1223 * Use production configuration for `fuel-core` during benches by @xgreenx in FuelLabs/fuel-core#1227 * Speedup and stabilize unit and integration tests by @xgreenx in FuelLabs/fuel-core#1231 * test: State benchmarks by @bvrooman in FuelLabs/fuel-core#1226 * Remove hybrid PoA block trigger mode by @Dentosal in FuelLabs/fuel-core#1232 * test: Benchmark contract state insertions with DB vs. DB transactions by @bvrooman in FuelLabs/fuel-core#1230 * multiplatform docker builds by @Voxelot in FuelLabs/fuel-core#1233 * Fix typo in architecture.md by @eltociear in FuelLabs/fuel-core#1241 * Expose gas cost in chain info by @MitchTurner in FuelLabs/fuel-core#1244 * Reuse calculated tx id in executor by @MitchTurner in FuelLabs/fuel-core#1248 * Fix multi-platform images by @Voxelot in FuelLabs/fuel-core#1251 * Add logging of the long GraphQL queries for future debug by @MitchTurner in FuelLabs/fuel-core#1250 * Reused `CheckedTransaction` from transaction pool in the executor by @xgreenx in FuelLabs/fuel-core#1249 * Bump `fuel-vm` to `0.35.0` version by @xgreenx in FuelLabs/fuel-core#1256 ## New Contributors * @segfault-magnet made their first contribution in FuelLabs/fuel-core#1219 * @eltociear made their first contribution in FuelLabs/fuel-core#1241 * @MitchTurner made their first contribution in FuelLabs/fuel-core#1244 **Full Changelog**: FuelLabs/fuel-core@v0.19.1...v0.20.0
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Closes #1216