Revert "Add support for TLS connections with self-signed cert on validator gRPC interface" #20232
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…dator gR…" This reverts commit b248f35.
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
|
aschran
temporarily deployed
to
sui-typescript-aws-kms-test-env
GitHub Actions
Inactive
arun-koshy
approved these changes
Nov 12, 2024
ebmifa
pushed a commit
that referenced
this pull request
Nov 12, 2024
…dator gRPC interface" (#20232) (#20234) Reverts #19796 due to suspicion it's causing performance issues --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API:
carlosbmamoru
pushed a commit
to Mamoru-Foundation/sui
that referenced
this pull request
Nov 14, 2024
commit dc117c586a01d6dbef41eafcaf3d0be2e37f2d12 Merge: 32514d1c36 b8eb8920ae Author: Carlos Baez <[email protected]> Date: Thu Nov 14 11:36:36 2024 +0100 Merge tag 'testnet-v1.37.3' into testnet4-wit-v1.37.3 Sui testnet v1.37.3 release commit b8eb8920aeca592d0cf15fbddc7f64ed4aad7202 Author: Andrew Schran <[email protected]> Date: Tue Nov 12 15:34:20 2024 -0500 Revert "Add support for TLS connections with self-signed cert on validator gRPC interface" (#20232) (#20234) Reverts MystenLabs/sui#19796 due to suspicion it's causing performance issues --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit e205b302bd5c5742d21fbc18811b705980714c81 Author: Eugene Boguslavsky <[email protected]> Date: Tue Nov 12 16:56:05 2024 +0000 Sui v1.37.3 Version Bump commit f77489d1cf177b6bacda47c3c3ec6f120d1fed45 Author: Emma Zhong <[email protected]> Date: Mon Nov 11 19:22:47 2024 -0800 [cherrypick][jsonrpc] fix estimated rewards during safe mode (#20182) (#20187) ## Description Currently if the exchange rate for an epoch is not found, we would assign it the default exchange rate while we should've used the exchange rate from the previous existing epoch instead. This PR fixes that. ## Test plan Added some unit tests. --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [x] JSON-RPC: fixed reward estimation in `getStakes` API so that we don't overestimate stake rewards of stakes activated during safe mode epochs. - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: ## Description Describe the changes or additions included in this PR. ## Test plan How did you test the new or updated feature? --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit 102280ae7591ec1cf187a433624443b8798e5605 Author: Eugene Boguslavsky <[email protected]> Date: Thu Nov 7 19:25:00 2024 +0000 Sui v1.37.2 Version Bump commit b15a076619641002381f4be001f28be5dce7a3e9 Author: wlmyng <[email protected]> Date: Tue Nov 5 14:18:50 2024 -0800 [graphql][cherry-pick] need to add object_status=0 to use partial indexes (#20176) (#20181) ## Description cherry pick ## Test plan How did you test the new or updated feature? --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit 7839b9501066108cb2322ba9039120a41781a1b0 Author: John Martin <[email protected]> Date: Mon Nov 4 14:39:56 2024 -0800 add client-request-method to CORS allowed headers (#20160) (#20168) ## Description adds `client-request-method` to allowed cors methods ## Test plan --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: ## Description Describe the changes or additions included in this PR. ## Test plan How did you test the new or updated feature? --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit ecffa454e86b994b79dfc6fb9032d02ee95a0697 Author: John Martin <[email protected]> Date: Mon Nov 4 13:13:22 2024 -0800 fix: remove high cardinality address label from safe_client_latency (… (#20169) …#20165) ## Description safe_client_latency is the highest cardinality metric across our observability system due to the `address` label + being a histogram: ``` root@sui-node-mysten-rpc-0:/sui# curl -s localhost:9184/metrics | grep safe_client_latency_bucket | wc -l 8512 ``` --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: ## Description Describe the changes or additions included in this PR. ## Test plan How did you test the new or updated feature? --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit 1493483238672e1d5c22dbff15eb670b1d57883a Author: Eugene Boguslavsky <[email protected]> Date: Mon Nov 4 20:00:02 2024 +0000 Sui v1.37.1 Version Bump commit 6646b75dd68808b7bedb679c41706b12ae79a5a9 Author: Eugene Boguslavsky <[email protected]> Date: Thu Oct 31 18:46:36 2024 -0700 Sui `v1.37.0` Framework Bytecode Snapshot (#20131) ## Description Sui `v1.37.0` Framework Bytecode Snapshot ## Test plan `cargo run --bin sui-framework-snapshot` commit fa5d48fcb3e06a11677dc14e90f12cac59e8f2fc Author: Ashok Menon <[email protected]> Date: Fri Nov 1 01:03:44 2024 +0000 chore(indexer-alt): alphabetize Cargo.toml commit e116dfa6547e96608bee6767a483e080bb0a1e99 Author: Ashok Menon <[email protected]> Date: Fri Nov 1 00:39:20 2024 +0000 indexer-alt: watermark timestamps and other logging improvements (#20120) ## Description Track the timestamp associated with a watermark, include it in tracing messages, and introduce gauges to track the timestamps of the latest checkpoint gathered and written to the DB. This change also introduces a couple of other timestamp/watermark related tracing improvements: - Measure watermarks gathered and in DB for the sequential pipeline (previously just tracked the watermark after they were written to the DB). These will usually be the exact same, but may differ if the pipeline needs to retry a write. - Gets rid of the ordering and equality impls for `CommitterWatermark` -- these are no longer required. - Standardise the ordering of watermark traces between the sequential and concurrent pipelines, so that they are easier to compare with each other (they now share a common prefix). - Standardised tracing messages during committer/watermark teardown: Now in each exit edge, we log the reason for the exit, and then a message with the final watermark. ## Test plan Run `sum_obj_types` and `wal_obj_types` with a consistent range of `3600` and note that the difference in their timestamps is roughly an hour (to begin with, it's about 70 minutes): ``` sui$ cargo run -p sui-indexer-alt --release -- \ --database-url 'postgres://postgres:postgrespw@localhost:5432/sui_indexer_alt' \ indexer --remote-store-url 'https://checkpoints.mainnet.sui.io' \ --consistent-range 3600 --pipeline wal_obj_types --pipeline sum_obj_types` Running `/Users/ashokmenon/sui/idx-poc/target/release/sui-indexer-alt --database-url 'postgres://postgres:postgrespw@localhost:5432/sui_indexer_alt' indexer --remote-store-url 'https://checkpoints.mainnet.sui.io' --consistent-range 3600 --pipeline wal_obj_types --pipeline sum_obj_types` 2024-10-31T15:16:26.867221Z INFO sui_indexer_alt::db: Running migrations ... 2024-10-31T15:16:26.889145Z INFO sui_indexer_alt::db: Migrations complete. 2024-10-31T15:16:27.024406Z INFO sui_indexer_alt: Skipping pipeline ev_emit_mod 2024-10-31T15:16:27.024413Z INFO sui_indexer_alt: Skipping pipeline ev_struct_inst 2024-10-31T15:16:27.024414Z INFO sui_indexer_alt: Skipping pipeline kv_checkpoints 2024-10-31T15:16:27.024414Z INFO sui_indexer_alt: Skipping pipeline kv_objects 2024-10-31T15:16:27.024415Z INFO sui_indexer_alt: Skipping pipeline kv_transactions 2024-10-31T15:16:27.024416Z INFO sui_indexer_alt: Skipping pipeline obj_versions 2024-10-31T15:16:27.024416Z INFO sui_indexer_alt: Skipping pipeline tx_affected_objects 2024-10-31T15:16:27.024417Z INFO sui_indexer_alt: Skipping pipeline tx_balance_changes 2024-10-31T15:16:27.024417Z INFO sui_indexer_alt: Skipping pipeline wal_coin_balances 2024-10-31T15:16:27.033550Z INFO sui_indexer_alt: Skipping pipeline sum_coin_balances 2024-10-31T15:16:27.033607Z INFO sui_indexer_alt::pipeline::processor: Starting processor pipeline="wal_obj_types" 2024-10-31T15:16:27.033856Z INFO sui_indexer_alt::pipeline::concurrent::collector: Starting collector pipeline="wal_obj_types" 2024-10-31T15:16:27.033869Z INFO sui_indexer_alt::pipeline::concurrent::committer: Starting committer pipeline="wal_obj_types" 2024-10-31T15:16:27.033983Z INFO sui_indexer_alt: Skipping pipeline sum_packages 2024-10-31T15:16:27.034011Z INFO sui_indexer_alt::pipeline::processor: Starting processor pipeline="sum_obj_types" 2024-10-31T15:16:27.034183Z INFO sui_indexer_alt::pipeline::concurrent::watermark: Starting watermark pipeline="wal_obj_types" watermark=CommitterWatermark { pipeline: "wal_obj_types", epoch_hi_inclusive: 0, checkpoint_hi_inclusive: 0, tx_hi: 0, timestamp_ms_hi_inclusive: 0 } 2024-10-31T15:16:27.034260Z INFO sui_indexer_alt::pipeline::sequential::committer: Starting committer pipeline="sum_obj_types" watermark=CommitterWatermark { pipeline: "sum_obj_types", epoch_hi_inclusive: 0, checkpoint_hi_inclusive: 0, tx_hi: 0, timestamp_ms_hi_inclusive: 0 } 2024-10-31T15:16:27.034305Z INFO sui_indexer_alt: Ingestion range first_checkpoint=0 last_checkpoint=None 2024-10-31T15:16:27.034339Z INFO sui_indexer_alt::metrics: Starting metrics service on 0.0.0.0:9184 2024-10-31T15:16:27.034351Z INFO sui_indexer_alt::ingestion::regulator: Starting ingestion regulator 2024-10-31T15:16:27.034355Z INFO sui_indexer_alt::ingestion::broadcaster: Starting ingestion broadcaster 2024-10-31T15:16:27.541400Z INFO sui_indexer_alt::pipeline::concurrent::watermark: Watermark pipeline="wal_obj_types" epoch=0 checkpoint=212 transaction=213 timestamp=2023-04-13 13:28:25.836 UTC updated=true elapsed_ms=5.486208 2024-10-31T15:16:28.043533Z INFO sui_indexer_alt::pipeline::concurrent::watermark: Watermark pipeline="wal_obj_types" epoch=0 checkpoint=515 transaction=516 timestamp=2023-04-13 13:36:17.240 UTC updated=true elapsed_ms=5.512666 2024-10-31T15:16:28.538421Z INFO sui_indexer_alt::pipeline::concurrent::watermark: Watermark pipeline="wal_obj_types" epoch=0 checkpoint=1017 transaction=1018 timestamp=2023-04-13 13:48:04.584 UTC updated=true elapsed_ms=3.468209 2024-10-31T15:16:29.041788Z INFO sui_indexer_alt::pipeline::concurrent::watermark: Watermark pipeline="wal_obj_types" epoch=0 checkpoint=1422 transaction=1423 timestamp=2023-04-13 13:57:11.577 UTC updated=true elapsed_ms=7.184292 2024-10-31T15:16:29.540675Z INFO sui_indexer_alt::pipeline::concurrent::watermark: Watermark pipeline="wal_obj_types" epoch=0 checkpoint=1824 transaction=1825 timestamp=2023-04-13 14:05:59.154 UTC updated=true elapsed_ms=5.7144580000000005 2024-10-31T15:16:30.040987Z INFO sui_indexer_alt::pipeline::concurrent::watermark: Watermark pipeline="wal_obj_types" epoch=0 checkpoint=2226 transaction=2227 timestamp=2023-04-13 14:14:54.721 UTC updated=true elapsed_ms=5.955416 2024-10-31T15:16:30.539394Z INFO sui_indexer_alt::pipeline::concurrent::watermark: Watermark pipeline="wal_obj_types" epoch=0 checkpoint=2630 transaction=2631 timestamp=2023-04-13 14:23:21.636 UTC updated=true elapsed_ms=5.0003340000000005 2024-10-31T15:16:31.035944Z INFO sui_indexer_alt::pipeline::concurrent::watermark: Watermark pipeline="wal_obj_types" epoch=0 checkpoint=3142 transaction=3143 timestamp=2023-04-13 14:34:00.192 UTC updated=true elapsed_ms=1.368042 2024-10-31T15:16:31.532346Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_obj_types" epoch=0 checkpoint=71 transaction=72 timestamp=2023-04-13 13:24:36.662 UTC 2024-10-31T15:16:31.535237Z INFO sui_indexer_alt::pipeline::concurrent::watermark: Watermark pipeline="wal_obj_types" epoch=0 checkpoint=3646 transaction=3647 timestamp=2023-04-13 14:45:19.227 UTC updated=true elapsed_ms=0.7080409999999999 2024-10-31T15:16:31.556459Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_obj_types" epoch=0 checkpoint=103 transaction=104 timestamp=2023-04-13 13:25:35.491 UTC 2024-10-31T15:16:31.605247Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_obj_types" epoch=0 checkpoint=151 transaction=152 timestamp=2023-04-13 13:27:01.087 UTC 2024-10-31T15:16:31.658676Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_obj_types" epoch=0 checkpoint=203 transaction=204 timestamp=2023-04-13 13:28:14.431 UTC 2024-10-31T15:16:31.709093Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_obj_types" epoch=0 checkpoint=251 transaction=252 timestamp=2023-04-13 13:29:41.350 UTC 2024-10-31T15:16:31.774101Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_obj_types" epoch=0 checkpoint=304 transaction=305 timestamp=2023-04-13 13:31:05.962 UTC 2024-10-31T15:16:31.823847Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_obj_types" epoch=0 checkpoint=351 transaction=352 timestamp=2023-04-13 13:32:06.748 UTC 2024-10-31T15:16:31.875613Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_obj_types" epoch=0 checkpoint=404 transaction=405 timestamp=2023-04-13 13:33:26.122 UTC 2024-10-31T15:16:31.923673Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_obj_types" epoch=0 checkpoint=452 transaction=453 timestamp=2023-04-13 13:34:41.672 UTC 2024-10-31T15:16:31.973523Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_obj_types" epoch=0 checkpoint=502 transaction=503 timestamp=2023-04-13 13:35:54.245 UTC 2024-10-31T15:16:32.020968Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_obj_types" epoch=0 checkpoint=553 transaction=554 timestamp=2023-04-13 13:37:12.723 UTC 2024-10-31T15:16:32.034336Z INFO sui_indexer_alt::pipeline::concurrent::watermark: Watermark pipeline="wal_obj_types" epoch=0 checkpoint=4147 transaction=4148 timestamp=2023-04-13 14:56:30.470 UTC updated=true elapsed_ms=0.478916 2024-10-31T15:16:32.081115Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_obj_types" epoch=0 checkpoint=603 transaction=604 timestamp=2023-04-13 13:38:26.226 UTC 2024-10-31T15:16:32.143228Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_obj_types" epoch=0 checkpoint=657 transaction=658 timestamp=2023-04-13 13:39:32.583 UTC 2024-10-31T15:16:32.201323Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_obj_types" epoch=0 checkpoint=710 transaction=711 timestamp=2023-04-13 13:40:44.461 UTC 2024-10-31T15:16:32.239715Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_obj_types" epoch=0 checkpoint=751 transaction=752 timestamp=2023-04-13 13:41:40.955 UTC 2024-10-31T15:16:32.306828Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_obj_types" epoch=0 checkpoint=803 transaction=804 timestamp=2023-04-13 13:43:00.433 UTC 2024-10-31T15:16:32.351995Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_obj_types" epoch=0 checkpoint=854 transaction=855 timestamp=2023-04-13 13:44:21.487 UTC 2024-10-31T15:16:32.408044Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_obj_types" epoch=0 checkpoint=903 transaction=904 timestamp=2023-04-13 13:45:33.468 UTC 2024-10-31T15:16:32.478623Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_obj_types" epoch=0 checkpoint=963 transaction=964 timestamp=2023-04-13 13:46:48.531 UTC 2024-10-31T15:16:32.517822Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_obj_types" epoch=0 checkpoint=1004 transaction=1005 timestamp=2023-04-13 13:47:43.997 UTC 2024-10-31T15:16:32.540701Z INFO sui_indexer_alt::pipeline::concurrent::watermark: Watermark pipeline="wal_obj_types" epoch=0 checkpoint=4550 transaction=4551 timestamp=2023-04-13 15:05:30.690 UTC updated=true elapsed_ms=5.776375 2024-10-31T15:16:32.574600Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_obj_types" epoch=0 checkpoint=1052 transaction=1053 timestamp=2023-04-13 13:48:55.253 UTC 2024-10-31T15:16:32.625726Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_obj_types" epoch=0 checkpoint=1102 transaction=1103 timestamp=2023-04-13 13:49:57.288 UTC 2024-10-31T15:16:32.679135Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_obj_types" epoch=0 checkpoint=1159 transaction=1160 timestamp=2023-04-13 13:51:06.819 UTC 2024-10-31T15:16:32.729858Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_obj_types" epoch=0 checkpoint=1203 transaction=1204 timestamp=2023-04-13 13:52:11.201 UTC 2024-10-31T15:16:32.777044Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_obj_types" epoch=0 checkpoint=1257 transaction=1258 timestamp=2023-04-13 13:53:26.750 UTC 2024-10-31T15:16:32.819603Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_obj_types" epoch=0 checkpoint=1301 transaction=1302 timestamp=2023-04-13 13:54:17.123 UTC 2024-10-31T15:16:32.870028Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_obj_types" epoch=0 checkpoint=1351 transaction=1352 timestamp=2023-04-13 13:55:24.124 UTC 2024-10-31T15:16:32.923059Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_obj_types" epoch=0 checkpoint=1402 transaction=1403 timestamp=2023-04-13 13:56:49.078 UTC 2024-10-31T15:16:32.969159Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_obj_types" epoch=0 checkpoint=1451 transaction=1452 timestamp=2023-04-13 13:57:56.351 UTC 2024-10-31T15:16:33.034831Z INFO sui_indexer_alt::pipeline::concurrent::watermark: Watermark pipeline="wal_obj_types" epoch=0 checkpoint=5055 transaction=5056 timestamp=2023-04-13 15:16:38.038 UTC updated=true elapsed_ms=1.0442500000000001 2024-10-31T15:16:33.036266Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_obj_types" epoch=0 checkpoint=1501 transaction=1502 timestamp=2023-04-13 13:58:56.227 UTC ^C2024-10-31T15:16:33.091189Z INFO sui_indexer_alt::pipeline::concurrent::collector: Shutdown received, stopping collector pipeline="wal_obj_types" 2024-10-31T15:16:33.091195Z INFO sui_indexer_alt::pipeline::concurrent::watermark: Shutdown received pipeline="wal_obj_types" 2024-10-31T15:16:33.091201Z INFO sui_indexer_alt::ingestion::regulator: Shutdown received, stopping regulator 2024-10-31T15:16:33.091213Z INFO sui_indexer_alt::metrics: Shutdown received, stopping metrics service 2024-10-31T15:16:33.091225Z INFO sui_indexer_alt::pipeline::concurrent::watermark: Stopping watermark task pipeline="wal_obj_types" watermark=CommitterWatermark { pipeline: "wal_obj_types", epoch_hi_inclusive: 0, checkpoint_hi_inclusive: 5055, tx_hi: 5056, timestamp_ms_hi_inclusive: 1681398998038 } 2024-10-31T15:16:33.091410Z INFO sui_indexer_alt::pipeline::concurrent::committer: Batches done, stopping committer pipeline="wal_obj_types" 2024-10-31T15:16:33.091721Z INFO sui_indexer_alt::pipeline::sequential::committer: Shutdown received pipeline="sum_obj_types" 2024-10-31T15:16:33.091725Z INFO sui_indexer_alt::pipeline::sequential::committer: Stopping committer pipeline="sum_obj_types" watermark=CommitterWatermark { pipeline: "sum_obj_types", epoch_hi_inclusive: 0, checkpoint_hi_inclusive: 1546, tx_hi: 1547, timestamp_ms_hi_inclusive: 1681394391225 } 2024-10-31T15:16:33.093829Z INFO sui_indexer_alt::pipeline::processor: Shutdown received, stopping processor pipeline="wal_obj_types" 2024-10-31T15:16:33.094014Z INFO sui_indexer_alt::pipeline::processor: Shutdown received, stopping processor pipeline="sum_obj_types" 2024-10-31T15:16:33.094406Z INFO sui_indexer_alt::ingestion::broadcaster: Shutdown received, stopping ingestion broadcaster 2024-10-31T15:16:33.094513Z INFO sui_indexer_alt: Indexing pipeline gracefully shut down ``` ## Stack - #20089 - #20114 - #20116 - #20117 - #20119 --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit 4b727cbbfd89db7ebfd09b21e05faa58ac2d8c96 Author: Ashok Menon <[email protected]> Date: Fri Nov 1 00:37:19 2024 +0000 indexer-alt: allow sequential pipeline immediate reset (#20119) ## Description If the sequential pipeline committer can guarantee that it could process more checkpoints by looking at its pending buffer, we now reset the polling interval immediately, so it does not wait to issue the next write. This mimics a similar behaviour in the concurrent pipeline. I made this change after noticing how the pipeline behaves when ingestion is stuck retrying a checkpoint, on my local machine. Usually when running locally, performance is limited by checkpoint download rate, but in a sequential pipeline, if a checkpoint failed to download, it is possible for many checkpoints to end up processed and pending. With the previous implementation, once ingestion had recovered (the checkpoint is fetched), the pending buffer kept growing because it was only able to land `MAX_BATCH_CHECKPOINTS / commit_interval`, so if checkpoints were getting added faster than that, it would never recover. With this change, the pipeline recovers almost instantly, and I expect that in GCP where bandwidth is not the rate limiting factor, this should improve throughput during backfill, and synthetic benchmarks. ## Test plan Run the indexer with a large ingestion buffer and concurrency, wait for ingestion to fail to fetch a checkpoint, and then notice how the situation recovers (instead of getting worse until the pipeline eventually complains that it has too many pending checkpoints): ``` sui$ cargo run -p sui-indexer-alt --release -- \ --database-url "postgres://postgres:postgrespw@localhost:5432/sui_indexer_alt" \ indexer --remote-store-url https://checkpoints.mainnet.sui.io \ --last-checkpoint 1200000 --pipeline sum_packages \ --checkpoint-buffer-size 50000 --ingest-concurrency 20000 ``` ## Stack - #20089 - #20114 - #20116 - #20117 --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit b65f34c6a00801620e11469575523895a8e074ac Author: Ashok Menon <[email protected]> Date: Fri Nov 1 00:21:38 2024 +0000 indexer-alt: wal_coin_balances pipeline (#20117) ## Description `wal_coin_balances` is to `sum_coin_balances` what `wal_obj_types` is to `sum_obj_types`. ## Test plan Run the indexer, and correlate the live object set calculated from the write-ahead log against the one that's already in the summary table: ``` sui$ cargo run -p sui-indexer-alt --release -- \ --database-url "postgres://postgres:postgrespw@localhost:5432/sui_indexer_alt" \ indexer --remote-store-url https://checkpoints.mainnet.sui.io/ \ --last-checkpoint 5000 ``` ``` sui_indexer_alt=# SELECT COUNT(*) FROM sum_coin_balances; count ------- 178 (1 row) sui_indexer_alt=# SELECT COUNT(*) FROM ( SELECT DISTINCT ON (object_id) * FROM wal_coin_balances ORDER BY object_id, object_version DESC ) o WHERE o.owner_id IS NOT NULL; count ------- 178 (1 row) ``` ## Stack - #20089 - #20114 - #20116 --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit 71f1faa1a8557a178e0bf89f20829a8ea503f156 Author: Ashok Menon <[email protected]> Date: Fri Nov 1 00:18:46 2024 +0000 indexer-alt: wal_obj_types pipeline (#20116) ## Description Adds the concurrent pipeline that writes the write-ahead log for `sum_obj_types`. It re-uses the `process` implementation from `sum_obj_types` and then writes it into an append-only table. Note that today: - The pipelines are completely independent, which means the WAL pipeline redoes the processing work of the summary pipeline (this is presumably not an issue because the repeated work is not particularly heavy). - This change does not include the pruning necessary to keep this table's size in check (in practice it should only be a couple of gigs in size). This will come in a follow-up PR. ## Test plan Run the indexer and cross check the live-object set calculated from the write-ahead log with the summary: ``` sui$ cargo run -p sui-indexer-alt --release -- \ --database-url "postgres://postgres:postgrespw@localhost:5432/sui_indexer_alt" \ indexer --remote-store-url https://checkpoints.mainnet.sui.io \ --last-checkpoint 5000 ``` ``` sui_indexer_alt=# SELECT COUNT(*) FROM sum_obj_types; count ------- 592 (1 row) ^ sui_indexer_alt=# SELECT COUNT(*) FROM ( SELECT DISTINCT ON (object_id) * FROM wal_obj_types ORDER BY object_id, object_version DESC ) o WHERE o.owner_kind IS NOT NULL; count ------- 592 (1 row) ``` ## Stack - #20089 - #20114 --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit 7de02417c27844c8a086866b1279b4736ab862ec Author: Ashok Menon <[email protected]> Date: Thu Oct 31 23:22:42 2024 +0000 indexer-alt: obj_versions pipeline (#20114) ## Description Pipeline to fill `obj_versions` table -- this is the table that serves "parent version" (dynamic field) queries. ## Test plan ``` sui$ cargo run -p sui-indexer-alt --release -- \ --database-url "postgres://postgres:postgrespw@localhost:5432/sui_indexer_alt" \ indexer --remote-url "https://checkpoints.mainnet.sui.io" \ --last-checkpoint 5000 ``` ## Stack - #20089 --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit 53eaed5e300a67f0d01cf68119a145410925f9de Author: Ashok Menon <[email protected]> Date: Wed Oct 30 00:06:07 2024 +0000 indexer-alt: delays for sequential pipelines ## Description Add the ability to include an optional delay to sequential pipelines. This can be used to implement behaviour similar to `objects_snapshot` in the new architecture. Sequential pipelines honour this configuration in their committer tasks: - They process checkpoints at the same rate as other pipelines - But hold back data in the committer's "pending" map until the committer witnesses a checkpoint that is at least `checkpoint_lag` checkpoints ahead. This also requires slightly complicating the set-up and tear-down logic: - When setting up, we may need to wait until the pending buffer has been primed enough to set a valid commit upperbound. - If the pipeline runs dry before that point, we need to add an extra check to close down the committer. - When tearing down the committer after its channel has closed, we need to check whether batching would have picked up the first pending checkpoint (whether it is a valid "next checkpoint" and it's not held up because of a lag parameter). ## Test plan Run the indexer with various parameterisations for the consistent range, and note that the watermark for the summary pipelines lag behind the watermarks for other pipelines. ``` sui$ cargo run -p sui-indexer-alt --release -- \ --database-url "postgres://postgres:postgrespw@localhost:5432/sui_indexer_alt" \ indexer --remote-store-url https://checkpoints.mainnet.sui.io \ --pipeline kv_objects \ --pipeline sum_obj_types --pipeline sum_coin_balances \ --last-checkpoint 5000 --consistent-range 1000 [...] 2024-10-30T00:24:45.148259Z INFO sui_indexer_alt::ingestion::regulator: Checkpoints done, stopping regulator 2024-10-30T00:24:45.161682Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_obj_types" epoch=0 checkpoint=3605 transaction=3606 2024-10-30T00:24:45.212394Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_coin_balances" epoch=0 checkpoint=2703 transaction=2704 2024-10-30T00:24:45.222986Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_obj_types" epoch=0 checkpoint=3656 transaction=3657 2024-10-30T00:24:45.274318Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_obj_types" epoch=0 checkpoint=3702 transaction=3703 2024-10-30T00:24:45.340714Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_obj_types" epoch=0 checkpoint=3753 transaction=3754 2024-10-30T00:24:45.355934Z INFO sui_indexer_alt::pipeline::concurrent::watermark: Watermark pipeline="kv_objects" elapsed_ms=0.949083 updated=true epoch=0 checkpoint=4760 transaction=4761 2024-10-30T00:24:45.401262Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_obj_types" epoch=0 checkpoint=3801 transaction=3802 2024-10-30T00:24:45.463134Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_obj_types" epoch=0 checkpoint=3852 transaction=3853 2024-10-30T00:24:45.544933Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_obj_types" epoch=0 checkpoint=3904 transaction=3905 2024-10-30T00:24:45.609068Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_obj_types" epoch=0 checkpoint=3952 transaction=3953 2024-10-30T00:24:45.668560Z INFO sui_indexer_alt::ingestion::broadcaster: Checkpoints done, stopping ingestion broadcaster 2024-10-30T00:24:45.668593Z INFO sui_indexer_alt::pipeline::processor: Checkpoints done, stopping processor pipeline="kv_objects" 2024-10-30T00:24:45.668605Z INFO sui_indexer_alt::pipeline::processor: Checkpoints done, stopping processor pipeline="sum_obj_types" 2024-10-30T00:24:45.668605Z INFO sui_indexer_alt::pipeline::processor: Checkpoints done, stopping processor pipeline="sum_coin_balances" 2024-10-30T00:24:45.674305Z INFO sui_indexer_alt::pipeline::sequential::committer: Processor closed channel, pending rows empty, stopping committer pipeline="sum_obj_types" watermark=CommitterWatermark { pipeline: "sum_obj_types", epoch_hi_inclusive: 0, checkpoint_hi_inclusive: 4000, tx_hi: 4001 } 2024-10-30T00:24:45.713856Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_coin_balances" epoch=0 checkpoint=3003 transaction=3004 2024-10-30T00:24:45.863837Z INFO sui_indexer_alt::pipeline::concurrent::watermark: Watermark pipeline="kv_objects" elapsed_ms=1.83825 updated=true epoch=0 checkpoint=4960 transaction=4961 2024-10-30T00:24:46.120459Z INFO sui_indexer_alt::pipeline::concurrent::collector: Processor closed channel, pending rows empty, stopping collector pipeline="kv_objects" 2024-10-30T00:24:46.131407Z INFO sui_indexer_alt::pipeline::concurrent::committer: Batches done, stopping committer pipeline="kv_objects" 2024-10-30T00:24:46.214267Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_coin_balances" epoch=0 checkpoint=3303 transaction=3304 2024-10-30T00:24:46.356941Z INFO sui_indexer_alt::pipeline::concurrent::watermark: Watermark pipeline="kv_objects" elapsed_ms=0.68 updated=true epoch=0 checkpoint=5000 transaction=5001 2024-10-30T00:24:46.356970Z INFO sui_indexer_alt::pipeline::concurrent::watermark: Committer closed channel, stopping watermark task pipeline="kv_objects" watermark=CommitterWatermark { pipeline: "kv_objects", epoch_hi_inclusive: 0, checkpoint_hi_inclusive: 5000, tx_hi: 5001 } 2024-10-30T00:24:46.714498Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_coin_balances" epoch=0 checkpoint=3603 transaction=3604 2024-10-30T00:24:47.214758Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_coin_balances" epoch=0 checkpoint=3903 transaction=3904 2024-10-30T00:24:47.715841Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_coin_balances" epoch=0 checkpoint=4000 transaction=4001 2024-10-30T00:24:47.715917Z INFO sui_indexer_alt::pipeline::sequential::committer: Processor closed channel, pending rows empty, stopping committer pipeline="sum_coin_balances" watermark=CommitterWatermark { pipeline: "sum_coin_balances", epoch_hi_inclusive: 0, checkpoint_hi_inclusive: 4000, tx_hi: 4001 } 2024-10-30T00:24:47.716485Z INFO sui_indexer_alt: Indexing pipeline gracefully shut down 2024-10-30T00:24:47.716647Z INFO sui_indexer_alt::metrics: Shutdown received, stopping metrics service ``` commit d57326492b62ec8ba7ff8826ad473ec1b5008f11 Author: Ashok Menon <[email protected]> Date: Wed Oct 30 00:01:45 2024 +0000 fix(indexer-alt): remove double logging on pipeline skip ## Description We were logging the pipeline being skipped twice. ## Test plan Check logs while the indexer starts up: ``` sui$ cargo run -p sui-indexer-alt --release -- \ --database-url "postgres://postgres:postgrespw@localhost:5432/sui_indexer_alt" \ indexer --remote-store-url "https://checkpoints.mainnet.sui.io" \ --last-checkpoint 5000 --pipeline sum_obj_types ``` commit ef0d78c638e3ef0d0a3b28787b618ea56a140284 Author: Ge Gao <[email protected]> Date: Thu Oct 31 19:30:51 2024 -0400 add sui-field-count for sui-indexer-alt (#20126) ## Description title, so that we can replace hard-coded field count in pipelines. a crate did very similar things but it's archived, so prob better to have our own impl, also b/c the functionality is relatively simple https://github.com/discosultan/field-count ## Test plan added tests in models/ --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit c92dd1b4f16074ced8c1fe5c56a10e675f68bbe6 Author: phoenix <[email protected]> Date: Thu Oct 31 18:28:34 2024 -0400 [kv store] bigtable. Add support for querying checkpoint by digest (#20122) ## Description Adds support for querying checkpoint by digest --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit 74ace7b6166f10bddfd714568e0ddc20db2ca75d Author: JasonRUAN <[email protected]> Date: Fri Nov 1 06:25:44 2024 +0800 [CLI] fix `sui keytool import` --alias command and default alias does not work (#20111) ## Description fix `sui keytool import` --alias command and default alias does not work Before Fix： <img width="1295" alt="image" src="https://github.com/user-attachments/assets/4cb40d92-e266-4b6e-9c39-c05ac52d7a9d"> After Fix： - **--alias** <img width="1301" alt="image" src="https://github.com/user-attachments/assets/b5c711e8-2a4d-4861-9965-30cde260e93f"> - **default alias** <img width="1295" alt="image" src="https://github.com/user-attachments/assets/4513db65-0e10-4179-b5f0-de188871cf78"> ## Test plan How did you test the new or updated feature? --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [x] CLI: `sui keytool import --alias` is now fixed and correctly records the provided alias. - [ ] Rust SDK: - [ ] REST API: commit e41a759769fdea08f660160a23e945bbad80b337 Author: sui-merge-bot[bot] <114704316+sui-merge-bot[bot]@users.noreply.github.com> Date: Thu Oct 31 23:50:37 2024 +0200 Version Packages (#20127) This PR was opened by the [Changesets release](https://github.com/changesets/action) GitHub action. When you're ready to do a release, you can merge this and publish to npm yourself or [setup this action to publish automatically](https://github.com/changesets/action#with-publishing). If you're not ready to do a release yet, that's fine, whenever you add more changesets to main, this PR will be updated. # Releases ## @mysten/[email protected] ### Patch Changes - 02c9e46: Fix exports on the bundled package Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> commit 02c9e4691b429b4a0ec85b71eace37b72515d055 Author: Manolis Liolios <[email protected]> Date: Thu Oct 31 23:38:15 2024 +0200 Fix build, re-publish @mysten/kms (#20125) ## Description Build will always treat `src` as the base for outputs when building. ## Test plan How did you test the new or updated feature? --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit 16f2683a99e7b8824aa5879d3f20ffef936a7546 Author: William Smith <[email protected]> Date: Thu Oct 31 17:27:16 2024 -0400 [sui-benchmark] Add expected failure payloads (#19831) ## Description Introduce a new payload type into the benchmark tool - `ExpectedFailurePayload`. This payload type is configured with some expected failure type (currently only one implemented - user signature failure - but others can be added with minimal scaffolding) such that `payload.make_transaction()` generates a transaction that will fail in this manner. Note that the failures that this payload type is concerned with are failures to execute the transaction itself, rather than during execution by the MoveVM. In other words, it is expected that the failure mode will not consume gas or produce effects. Note that for this reason, its failure mode is inverted. It will be tallied for metrics purposes as an `error` if it succeeds and tallied as an `expected_error` (which is a success) if it fails. Also note that transaction responses for this type are handled by producing a `NextOp::Retry` (with some additional logging/metrics) since it is functionally equivalent to a retryable error. ## Test plan Ran the following: ``` SIM_STRESS_TEST_QPS=200 SIM_STRESS_TEST_WORKERS=20 RUST_LOG=debug cargo simtest --nocapture test_simulated_load_expected_failure_traffic_control ``` Observed the following in logs ``` 2022-01-03T02:05:47.734507Z INFO node{id=1 name="client"}: simtest::test: crates/sui-benchmark/tests/simtest.rs:1088: end of test BenchmarkStats { duration: 50.000002242s, num_error_txes: 0, num_expected_error_txes: 8049, num_success_txes: 348, num_success_cmds: 1977, total_gas_used: 3215776400, latency_ms: HistogramWrapper { histogram: Histogram { auto_resize: false, highest_trackable_value: 120000, lowest_discernible_value: 1, significant_value_digits: 3, bucket_count: 7, sub_bucket_count: 2048, sub_bucket_half_count: 1024, sub_bucket_half_count_magnitude: 10, sub_bucket_mask: 2047, leading_zero_count_base: 53, unit_magnitude: 0, unit_magnitude_mask: 0, max_value: 14359, min_non_zero_value: 72, total_count: 348, counts: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 2, 8, 12, 9, 20, 25, 22, 19 ... ``` [Also ran in PTN](https://metrics.sui.io/d/adl51ctsvmkg0a/traffic-control-dos-protection-dashboard?from=2024-10-31T19:00:50.548Z&to=2024-10-31T20:20:03.673Z&timezone=browser&var-network=private-testnet&var-host=$__all&var-fullnode=$__all) --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit d95918cea0d8fa0f1a4007273f477e3c12b61302 Author: phoenix <[email protected]> Date: Thu Oct 31 17:18:17 2024 -0400 [data ingestion] add support for bigtable in main workflow binary (#20018) ## Description adding BigTable KV as a variant to main binary that manages internal workflows --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit c33374e735257b2a460a2f882e062619f8958bc5 Author: wlmyng <[email protected]> Date: Thu Oct 31 12:51:47 2024 -0700 [mvr] indexer schema subset and reduced persist tasks (#20100) ## Description Describe the changes or additions included in this PR. ## Test plan How did you test the new or updated feature? --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit 7c517e7e5ce7193d1f51ef7f20a31b56f0e66797 Author: Xun Li <[email protected]> Date: Thu Oct 31 12:49:58 2024 -0700 Add system_invariant_violation macro (#20105) ## Description Introduce a system invariant violation macro. We can then set up alerts whenever the counter value changes. Used it in one place as an example. ## Test plan How did you test the new or updated feature? --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit b248f35879801b3bae9b76c4d204f36540d816f8 Author: Andrew Schran <[email protected]> Date: Thu Oct 31 14:16:48 2024 -0400 Add support for TLS connections with self-signed cert on validator gRPC interface (#19796) Client side use of TLS will be enabled-by-default in a future PR. --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [x] Nodes (Validators and Full nodes): Adds support for TLS on validator gRPC interface. - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit e06610823b985bdc01c40e23c9a9a2f8d5b571d1 Author: sui-merge-bot[bot] <114704316+sui-merge-bot[bot]@users.noreply.github.com> Date: Thu Oct 31 19:58:58 2024 +0200 Version Packages (#20076) This PR was opened by the [Changesets release](https://github.com/changesets/action) GitHub action. When you're ready to do a release, you can merge this and publish to npm yourself or [setup this action to publish automatically](https://github.com/changesets/action#with-publishing). If you're not ready to do a release yet, that's fine, whenever you add more changesets to main, this PR will be updated. # Releases ## @mysten/[email protected] ### Minor Changes - c24814b: Adds a custom header; 'Client-Request-Method' which will contain the method name used in each outgoing jsonrpc request ## @mysten/[email protected] ### Patch Changes - Updated dependencies [c24814b] - @mysten/[email protected] - @mysten/[email protected] ## @mysten/[email protected] ### Patch Changes - Updated dependencies [c24814b] - @mysten/[email protected] - @mysten/[email protected] - @mysten/[email protected] ## @mysten/[email protected] ### Patch Changes - Updated dependencies [c24814b] - @mysten/[email protected] ## @mysten/[email protected] ### Patch Changes - Updated dependencies [c24814b] - @mysten/[email protected] ## @mysten/[email protected] ### Patch Changes - Updated dependencies [c24814b] - @mysten/[email protected] - @mysten/[email protected] ## @mysten/[email protected] ### Patch Changes - Updated dependencies [c24814b] - @mysten/[email protected] ## @mysten/[email protected] ### Patch Changes - 4166d71: Fix doc comment on `getKiosk` command - Updated dependencies [c24814b] - @mysten/[email protected] ## @mysten/[email protected] ### Patch Changes - b3f3925: Introduces @mysten/kms which initially exposes a Sui AWS KMS signer - Updated dependencies [c24814b] - @mysten/[email protected] ## @mysten/[email protected] ### Patch Changes - Updated dependencies [c24814b] - @mysten/[email protected] ## @mysten/[email protected] ### Patch Changes - Updated dependencies [c24814b] - @mysten/[email protected] ## @mysten/[email protected] ### Patch Changes - Updated dependencies [c24814b] - @mysten/[email protected] ## @mysten/[email protected] ### Patch Changes - Updated dependencies [c24814b] - @mysten/[email protected] - @mysten/[email protected] Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> commit 093954a09bce1942a9f9773b845f59bf5124efbf Author: ronny-mysten <[email protected]> Date: Thu Oct 31 11:35:27 2024 -0600 [docs] Rpc update (#20094) ## Description Updates the RPC render to include component schemas. Also updates a few comments. ## Test plan How did you test the new or updated feature? --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit fa66617b7dfa7096fd64a2dc1b002d154223be4a Author: wlmyng <[email protected]> Date: Thu Oct 31 10:18:57 2024 -0700 [mvr] renaming to sui-mvr-indexer and Dockerfile (#20099) ## Description Describe the changes or additions included in this PR. ## Test plan How did you test the new or updated feature? --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit 434206f14a0a21bc62ea08480b4a5c4d36d6cc25 Author: Todd Nowacki <[email protected]> Date: Thu Oct 31 10:18:46 2024 -0700 [move-compiler] Re-organize warning filters (#20103) ## Description - Re-organize warning filters. It should make it a bit cleaner to refactor diagnostic reporting ## Test plan - ran tests --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit 0d16b6f8b7c4ff79f70400e8a94d7486b2b2c3fe Author: wlmyng <[email protected]> Date: Thu Oct 31 09:36:37 2024 -0700 [mvr] new crate for custom mvr-indexer, copied from sui-indexer crate (#20098) ## Description Simple copy-paste of the sui-indexer crate for the custom mvr indexer ## Test plan How did you test the new or updated feature? --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit b3f39253ff0153b69b42bc298d1ecdbb0524e276 Author: John Naulty Jr. <[email protected]> Date: Thu Oct 31 09:21:29 2024 -0700 AWS KMS TS Signer (#16139) ## Description Describe the changes or additions included in this PR. ## Test Plan How did you test the new or updated feature? --- If your changes are not user-facing and do not break anything, you can skip the following section. Otherwise, please briefly describe what has changed under the Release Notes section. ### Type of Change (Check all that apply) - [ ] protocol change - [ ] user-visible impact - [ ] breaking change for a client SDKs - [ ] breaking change for FNs (FN binary must upgrade) - [ ] breaking change for validators or node operators (must upgrade binaries) - [ ] breaking change for on-chain data layout - [ ] necessitate either a data wipe or data migration ### Release notes --------- Co-authored-by: Manolis Liolios <[email protected]> commit f78c7b006c28c4c1afd3f7d42e21e53970902f5d Author: Brandon Williams <[email protected]> Date: Wed Oct 30 19:50:45 2024 -0500 rest: revert client change to requested protobuf and request bcs instead Revert the client change to request protobuf as a response type, and instead request bcs, because the support for protobuf hasn't rolled out to other networks yet. commit 108ad4c8ea53c68bd90d0b3fcdc88fbc75640c61 Author: Brandon Williams <[email protected]> Date: Wed Oct 30 12:50:43 2024 -0500 rest: support protobuf format for committee endpoints commit 718a244bca584382faedde56f8247c6b26225b07 Author: Brandon Williams <[email protected]> Date: Wed Oct 30 11:03:43 2024 -0500 rest: support protobuf format for resolve, simulate, and execute transaction endpoints commit 2987f5967ecffa8ddbec2b8ebf9ee2eec09dad60 Author: Brandon Williams <[email protected]> Date: Wed Oct 30 08:55:51 2024 -0500 rest: support protobuf format for GetTransaction and ListTransactions endpoints commit 672ecbd2a58c7514aa24d139c442316a03d7f89e Author: Brandon Williams <[email protected]> Date: Tue Oct 29 21:30:42 2024 -0500 rest: support protobuf format for GetObject endpoints commit c8472c057523ff88040ae688f1410d34164b0ebc Author: Brandon Williams <[email protected]> Date: Tue Oct 29 19:00:39 2024 -0500 rest: remove ListFullCheckpoint api Remove the ListFullCheckpoint api as its a bit too heavy-weight and instead we'll look to introduce a streaming api in the future. commit 39f1ae21fd5467e71522729519720d2add9468dc Author: Xun Li <[email protected]> Date: Wed Oct 30 22:25:50 2024 -0700 Clean up local execution (#20086) ## Description Describe the changes or additions included in this PR. ## Test plan How did you test the new or updated feature? --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit 65b02e203ad897f438f87d0e5cfe46c56a1d4247 Author: phoenix <[email protected]> Date: Wed Oct 30 21:26:54 2024 -0400 [pruner] enable periodic compaction of individual SST files by default (#20101) ## Description enables periodic compaction of individual SST files by default --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit f2670718f694edd59e8a062bc932d4efd96ba765 Author: Tim Zakian <[email protected]> Date: Wed Oct 30 16:49:01 2024 -0700 [execution] Update to deps-only mode (#20102) ## Description Disallow new modules from being added to packages in deps-only mode. ## Test plan Added new tests to check for both previous and new behavior. --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [X] Protocol: Updated restrictions on deps-only packages to not allow adding new modules to the package. - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit 7e6319fa33b7021a94ff7dcc07e479a4bae92b55 Author: Adam Welc <[email protected]> Date: Wed Oct 30 15:51:37 2024 -0700 [trace-view] Added support for native functions and global locations (#20080) ## Description Adds support for native functions where the `CloseFrame` event happens right after `OpenFrame` event and both need to be skipped to skip over the native function. Also added support for handling global locations - they should be parsed properly but largely ignored in the trace as they cannot directly affect values of local variables ## Test plan All old and new tests must pass commit 21639a51847f1a04b6b221ac77df3875c0430be7 Author: Xun Li <[email protected]> Date: Wed Oct 30 14:30:59 2024 -0700 [RPC] Fix coin metadata api inefficiency (#19794) ## Description This PR fixes a minor inefficiency in the coin metadata api. It does not need to fetch the transaction, as it only needs the effects. This likely avoids a remote KV fetch. ## Test plan CI --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: comm…
carlosbmamoru
pushed a commit
to Mamoru-Foundation/sui
that referenced
this pull request
Nov 19, 2024
commit cd214a3d7e8e9e47ccf31ab8f5315e8c114313be Author: wlmyng <[email protected]> Date: Fri Nov 15 13:21:24 2024 -0800 [mvr] Introduce mvr-mode on production indexer (#20271) (#20288) ## Description introduce mvr-mode flag. If true, enables pruning objects_history, and toggles committer to write only a subset of tables to db. Compatible with the production indexer schema. ## Test plan How did you test the new or updated feature? --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: ## Description Describe the changes or additions included in this PR. ## Test plan How did you test the new or updated feature? --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit ab6863576baf7ff74f0de95ab5abef9040fc8fcd Author: Eugene Boguslavsky <[email protected]> Date: Thu Nov 14 18:41:02 2024 -0800 Sui `v1.38.0` Framework Bytecode Snapshot (#20277) ## Description Sui `v1.38.0` Framework Bytecode Snapshot ## Test plan 👀 commit 5d70ec6f5d4b7ff611649432aeb8c538efb63e80 Author: Sadhan Sood <[email protected]> Date: Thu Nov 14 17:08:27 2024 -0800 feat(analytics): Add package filter to object handler (#20272) ## Description This PR adds a package filter for object handler in analytics for being able to generate an object table specifically for walrus in testnet. ## Test Invoking with `--package-id-filter 0x9f992cc2430a1f442ca7a5ca7638169f5d5c00e0ebc3977a65e9ac6e497fe5ef --starting-checkpoint-seq-num 120132840` to filter for walrus specific objects. Here is the walrus package object: https://testnet.suivision.xyz/package/0x9f992cc2430a1f442ca7a5ca7638169f5d5c00e0ebc3977a65e9ac6e497fe5ef commit b36024d32ce349ba9465b38bd6706b8b2068ee67 Author: stefan-mysten <[email protected]> Date: Thu Nov 14 16:27:12 2024 -0800 [CLI] Add back the warning about version mismatch (#20273) ## Description #20259 removed the warning for the SDK, so this PR adds it back for the CLI. ## Test plan How did you test the new or updated feature? --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit 81f75843464ce1140040403df0a0fa0852f87386 Author: Adam Welc <[email protected]> Date: Thu Nov 14 13:01:50 2024 -0800 [move-compiler] Added function location info to the source map (#20261) ## Description This PR adds additional field to the source map describing the location of the entire function. This is needed to enable better handling of macros in the trace viewer (we need to know which instructions in a given function are outside of a function body to identify those that belong to inlined macro functions). ## Test plan All existing tests must pass commit 8d90acd980b4490a7f5488cee2aad465abe8e663 Author: Tim Zakian <[email protected]> Date: Thu Nov 14 11:53:41 2024 -0800 [json-rpc] Add enums in `getNormalizedMoveModule` (#20231) ## Description Adds enums to the `NormalizedMoveModule` for json-rpc. This fixes #20205 ## Test plan Tested manually on the repro provided in the issue to make sure enums are included in the output. --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [x] JSON-RPC: Fixes an issue where enum types were not included in the output of `getNormalizedMoveModule` and `getNormalizedMoveModuleForPackage` - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit 53dd080a841710c91f492de17f7312a29662235f Author: William Smith <[email protected]> Date: Thu Nov 14 14:03:22 2024 -0500 [TrafficControl] Add salt to TrafficSketch (#20245) ## Description False positives stem from IP collisions during traffic tallying. Because validators tend to see closely correlated transaction execution traffic patterns, it is possible that a false positive resulting in an erroneously blocked IP could be correlated across many validators at the same time. If this happens on a quorum of validators, this would result in an IP being effectively denied service until the TTL expires. Luckily, we can exploit this property to lower the effective false positive rate (as observed by the client) by removing this correlation. This PR adds a salt key to the `SketchKey`. The value is generated in memory at startup. ## Test plan Existing tests --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit 97838b20eb28b5abd12d275f5924c70c97b86e2c Author: sui-merge-bot[bot] <114704316+sui-merge-bot[bot]@users.noreply.github.com> Date: Thu Nov 14 10:25:13 2024 -0800 Version Packages (#20269) This PR was opened by the [Changesets release](https://github.com/changesets/action) GitHub action. When you're ready to do a release, you can merge this and publish to npm yourself or [setup this action to publish automatically](https://github.com/changesets/action#with-publishing). If you're not ready to do a release yet, that's fine, whenever you add more changesets to main, this PR will be updated. # Releases ## @mysten/[email protected] ### Patch Changes - @mysten/[email protected] ## @mysten/[email protected] ### Patch Changes - Updated dependencies [1765780] - @mysten/[email protected] ## @mysten/[email protected] ### Patch Changes - 1765780: fix zksend link network detection Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> commit 176578020ba15ff321d42b487f87af0a650bece3 Author: Pavlos Chrysochoidis <[email protected]> Date: Thu Nov 14 18:10:02 2024 +0000 zksend sdk fix network detection (#20268) ## Description Describe the changes or additions included in this PR. ## Test plan How did you test the new or updated feature? --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit 28e184a694636fc3b5a589c219f47b549250d6c1 Author: xiaodi <[email protected]> Date: Fri Nov 15 00:24:24 2024 +0800 Update regulated.mdx (#20267) ## Description Replaced "Regulated Coin and Deny List" with "Loyalty Tokens" in documentation. ## Test plan How did you test the new or updated feature? --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit 0bd58999ae2be185c6677ebc4a44b3e5dcd15296 Author: sui-merge-bot[bot] <114704316+sui-merge-bot[bot]@users.noreply.github.com> Date: Thu Nov 14 09:18:08 2024 -0500 Version Packages (#20265) This PR was opened by the [Changesets release](https://github.com/changesets/action) GitHub action. When you're ready to do a release, you can merge this and publish to npm yourself or [setup this action to publish automatically](https://github.com/changesets/action#with-publishing). If you're not ready to do a release yet, that's fine, whenever you add more changesets to main, this PR will be updated. # Releases ## @mysten/[email protected] ### Patch Changes - 4a42691: NS token pools added Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> commit 4a426918ef263103020aca25cf05809e4531ec51 Author: Tony Lee <[email protected]> Date: Thu Nov 14 08:26:36 2024 -0500 NS Deepbook Pools (#20256) commit d5d51aed177432e62950385336b79f8395c940f9 Author: stefan-mysten <[email protected]> Date: Wed Nov 13 17:17:13 2024 -0800 [RustSDK] Less noise when rpc version and local CLI version mismatch (#20259) ## Description This PR removes the SDK tracing warning on mismatch local + RPC API versions. ## Test plan How did you test the new or updated feature? --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit c85e7c07c0713c1edf7cdb59eab46372ef29fe24 Author: Tim Zakian <[email protected]> Date: Wed Nov 13 17:16:49 2024 -0800 [docs] Update Sui CLI install instructions to include the tracing feature (#20255) ## Description Update docs for installing the Sui CLI to include `--features tracing` so that they work with the trace viewer/debugger. ## Test plan 👀 commit 9dfd26a723e5efdb5816d6c2492cb0bb27b2f2c5 Author: Eugene Boguslavsky <[email protected]> Date: Wed Nov 13 15:33:46 2024 -0800 Use tracing feature for sui builds (#20260) ## Description Use tracing feature for sui builds ## Test plan 👀 commit 047de09d84bf90b633fd183b0cecef38e9387317 Author: Xun Li <[email protected]> Date: Wed Nov 13 14:30:23 2024 -0800 [indexer-alt] Add processor lag metrics (#20235) ## Description Describe the changes or additions included in this PR. ## Test plan How did you test the new or updated feature? --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit c03132d75e80b569c2fd8447539ac0b7bddc8ecc Author: Andrew Schran <[email protected]> Date: Wed Nov 13 16:51:15 2024 -0500 Add support for TLS connections with self-signed cert on validator gRPC interface (#20238) Client side use of TLS will be enabled-by-default in a future PR. This has corrected TLS upgrade behavior compared to older PR #19796. --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [x] Nodes (Validators and Full nodes): Adds support for TLS on validator gRPC interface. - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit 313bf8b43a1b8b511cb9b012a23d5803444b3ea5 Author: Tim Zakian <[email protected]> Date: Wed Nov 13 13:16:01 2024 -0800 [move] Add Move 2024 (non-beta) edition and make it the default edition (#20242) ## Description Add Move `2024` edition and make it the default edition. ## Test plan Updated existing tests. --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [X] CLI: Added the Move `2024` edition, and set it as the default Move edition both when creating a new Move package and when migrating from an existing package. Existing `2024.beta` and `2024.alpha` editions remain supported. - [ ] Rust SDK: - [ ] REST API: commit 8f21cd702b8ec9f4796f317aa9ec6bfd56330f0e Author: phoenix <[email protected]> Date: Wed Nov 13 15:26:23 2024 -0500 [data ingestion] bigtable: set env credentials parameter (#20240) ## Description Pass filename with credential parameters to the ingestion binary --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit 3c444d5dada018e5b10a929efa6f17e3445367d5 Author: Adam Welc <[email protected]> Date: Wed Nov 13 11:04:35 2024 -0800 [move-ide] Fixed auto-completion on the dot character (#20175) ## Description This PR fixes a certain problem with auto-completion on the `.` character. Previously, once a part of the access chain after the `.` was parsed, auto-completion information for the `.` character location was no longer generated. This is particularly problematic if someone expects auto-completion in the middle of the function code, for example: ```move public struct SomeStruct has drop { some_field: u64, } public fun foo(_param: SomeStruct) {} public fun bar(): SomeStruct { let val = SomeStruct{some_field: 42}; val. SomeStruct {some_field: 42} } ``` It is expected that auto-completion would work correctly after `val.` but previously it did not because this expression was actually parsed as `val.SomeStruct`. This PR fixes this:  Previously, auto-completion for the `.` location would also not work if what follows the `.` is a function call (whether the function being called was resolved correctly or not). It now works correctly in both cases:   ## Test plan All new and (updated) all tests must pass commit 8d647c0f00809bba411079ce765fdb4fbecef00c Author: sui-merge-bot[bot] <114704316+sui-merge-bot[bot]@users.noreply.github.com> Date: Wed Nov 13 18:49:45 2024 +0000 Version Packages (#20252) This PR was opened by the [Changesets release](https://github.com/changesets/action) GitHub action. When you're ready to do a release, you can merge this and publish to npm yourself or [setup this action to publish automatically](https://github.com/changesets/action#with-publishing). If you're not ready to do a release yet, that's fine, whenever you add more changesets to main, this PR will be updated. # Releases ## @mysten/[email protected] ### Minor Changes - 60bdb62: Update graphql query and use stashed host for claim api - 364a93a: Remove decoding spaces to `+` in zksend wallet url hashes - 364a93a: Add claimedBy property that is set when claimedAssets are loaded ## @mysten/[email protected] ### Patch Changes - @mysten/[email protected] ## @mysten/[email protected] ### Patch Changes - Updated dependencies [60bdb62] - Updated dependencies [364a93a] - Updated dependencies [364a93a] - @mysten/[email protected] Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> commit 8117d8393f25fa85390094bca61952ab24c9b595 Author: Tim Zakian <[email protected]> Date: Wed Nov 13 10:38:18 2024 -0800 [move/tracing] Panic unit tests if tracing without being compiled with tracing support (#20253) ## Description Make sure that if the `--trace-execution` flag is passed that the binary was compiled with the `tracing` feature flag set. ## Test plan CI --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit 60bdb62627a821bb54b7c7415a63193b7395a05d Author: hayes-mysten <[email protected]> Date: Wed Nov 13 10:08:56 2024 -0800 [zksend] Update garphql query, and use getstashed.com as claim api host (#20237) ## Description fixes #20222 Describe the changes or additions included in this PR. ## Test plan How did you test the new or updated feature? --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit 364a93a1b10be70aabe0fb8937b3c1af610d1eba Author: hayes-mysten <[email protected]> Date: Wed Nov 13 10:00:16 2024 -0800 [zksend] add claimedBy property when loading assets of claimed link (#20244) ## Description Describe the changes or additions included in this PR. ## Test plan How did you test the new or updated feature? --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit 43d7c3dcfcdf062cad98b480f0c0a62e218026e9 Author: Xun Li <[email protected]> Date: Wed Nov 13 09:14:22 2024 -0800 [indexer-alt] Skip commit if batch is empty (#20124) ## Description If we did not collect anything to the batch, we should not commit to the DB. We should also avoid reporting any metrics. ## Test plan CI --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: --------- Co-authored-by: Ashok Menon <[email protected]> commit f04e4cddfdde35a5d5cba9c7fd7c0f9482d4193b Author: sui-merge-bot[bot] <114704316+sui-merge-bot[bot]@users.noreply.github.com> Date: Wed Nov 13 11:07:19 2024 -0500 Version Packages (#20239) This PR was opened by the [Changesets release](https://github.com/changesets/action) GitHub action. When you're ready to do a release, you can merge this and publish to npm yourself or [setup this action to publish automatically](https://github.com/changesets/action#with-publishing). If you're not ready to do a release yet, that's fine, whenever you add more changesets to main, this PR will be updated. # Releases ## @mysten/[email protected] ### Patch Changes - Updated dependencies [d5a23d7] - @mysten/[email protected] - @mysten/[email protected] ## @mysten/[email protected] ### Patch Changes - Updated dependencies [d5a23d7] - @mysten/[email protected] - @mysten/[email protected] - @mysten/[email protected] ## @mysten/[email protected] ### Patch Changes - Updated dependencies [d5a23d7] - @mysten/[email protected] ## @mysten/[email protected] ### Patch Changes - a43ae92: NS Token - Updated dependencies [d5a23d7] - @mysten/[email protected] ## @mysten/[email protected] ### Patch Changes - Updated dependencies [d5a23d7] - @mysten/[email protected] - @mysten/[email protected] ## @mysten/[email protected] ### Patch Changes - Updated dependencies [d5a23d7] - @mysten/[email protected] ## @mysten/[email protected] ### Patch Changes - Updated dependencies [d5a23d7] - @mysten/[email protected] ## @mysten/[email protected] ### Patch Changes - Updated dependencies [d5a23d7] - @mysten/[email protected] ## @mysten/[email protected] ### Patch Changes - Updated dependencies [d5a23d7] - @mysten/[email protected] ## @mysten/[email protected] ### Patch Changes - d5a23d7: Add tx.object.option for creatnig object options in transaction builder ## @mysten/[email protected] ### Patch Changes - Updated dependencies [d5a23d7] - @mysten/[email protected] ## @mysten/[email protected] ### Patch Changes - Updated dependencies [d5a23d7] - @mysten/[email protected] ## @mysten/[email protected] ### Patch Changes - Updated dependencies [d5a23d7] - @mysten/[email protected] - @mysten/[email protected] Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> commit a43ae921386d1e8406f81d73d2c317ad92bb211e Author: Tony Lee <[email protected]> Date: Wed Nov 13 10:38:16 2024 -0500 NS token in deepbook sdk (#20251) ## Description NS token in deepbook sdk ## Test plan How did you test the new or updated feature? Testnet and local ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit 91dec7c02d776f3e68a5a1e5b86851c7d289c206 Author: Ashok Menon <[email protected]> Date: Wed Nov 13 14:00:52 2024 +0000 indexer-alt: remove watermarks.epoch_lo (#20218) ## Description This field turns out not to be used by the new pruner implementation, because it is entirely based on checkpoints. ## Test plan CI ## Stack - #20149 - #20150 - #20166 - #20216 - #20217 --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit c5f275ea9fab4a099e998a7b4f65926fe1ae8e8e Author: Ashok Menon <[email protected]> Date: Mon Nov 11 02:43:06 2024 +0000 indexer-alt: pruner task ## Description Add the task that actually deletes data, based on the reader low watermark. ## Test plan Run the indexer and note the following: - Metrics related to deleted rows by the pruner (from `localhost:9184/metrics`) - The contents of the `watermarks` table. ``` sui$ cargo run -p sui-indexer-alt --release -- \ --database-url "postgres://postgres:postgrespw@localhost:5432/sui_indexer_alt" \ indexer --remote-store-url https://checkpoints.mainnet.sui.io \ --last-checkpoint 10000 \ --consistent-range 100 --consistent-pruning-interval 10 \ --pipeline sum_obj_types --pipeline wal_obj_types ``` Also tested running the indexer for an extended period of time (1M checkpoints over roughly half an hour in local testing), and noted how the pruner behaves. When configured as it would be in production (roughly one hour of consistent range, and a 5 minute pruning interval and a 2 minute pruning delay): - Many rows accumulated during backfill -- by the end of the 1M checkpoints, the pruner had only pruned up to between checkpoint 500K and checkpoint 700K depending on the pipeline. This should not be an issue under normal operation where the indexer will run for long enough for pruning to stabilise at the tip of the network (and it would be recommended practice to start from formal snapshot and therefore only need to run pruning from that point forward). - Because the reader watermark task and the pruner task use the same interval, it can take up to two ticks of that interval for the pruner to act on a change to its upperbound -- again, it should be okay, as the pruner's interval should be at least an order of magnitude smaller than its retention period. commit 3a2e332c68cfe9491eb56d09f087ff8cc559e6e4 Author: Ashok Menon <[email protected]> Date: Mon Nov 11 02:37:06 2024 +0000 fix(indexer-alt): overly chatty loud watermark updates ## Description The loud watermark update logic always bumped the previous update watermark by the same interval. When the indexer was running well ahead of the loud watermark update rate, this would cause many updates to be issued successively. This change makes it so that the next update is always `LOUD_WATERMARK_UPDATE_INTERVAL` away from the last loud update. ## Test plan Run the indexer. Previously updates -- particularly from summary tables that gathered changes up and write them out in big batches -- would come in bursts, and this behaviour is no longer apparent after the change: ``` sui$ cargo run -p sui-indexer --release -- \ --database-url "postgres://postgres:postgrespw@localhost:5432/sui_indexer_alt" \ indexer --remote-store-url https://checkpoints.mainnet.sui.io \ --last-checkpoint 1000000 --consistent-range 3600 ``` commit 6d31be68d0ef82f90aa0447e13fd4c82eb40f732 Author: Ashok Menon <[email protected]> Date: Sun Nov 10 17:19:53 2024 +0000 indexer-alt: reader watermark task ## Description The reader watermark task is one half of the pruning implementation. It applies only to concurrent pipelines, and tracks the lowest checkpoint sequence number the reader is guaranteed to get data for, and the timestamp (from the DB), that this threshold was set at. This task does not do any actual pruning, but it sets the limit that the pruner can go to, and a time at which it can start pruning to that point. A later PR will introduce the actual pruning task. Unlike the existing pruner implementation, this system does not support pruning by epoch, only by checkpoint. This was for two reasons: - Pruning no longer works by dropping partitions, so we can keep the implementation simple by only supporting the form of pruning we need (by checkpoint sequence number). - Pruning is part of the indexing framework, rather than a specific part of our indexer implementation, so pruning by epoch would require that the indexing framework kept track of a mapping from epoch to checkpoint sequence number. Along with the reader watermark, the following changes were made: - The existing watermark task was renamed to the `committer_watermark` to distinguish it. - The interface between pipelines and the indexing framework has been simplified to return just one `JoinHandle`. This was because clippy complained that returning 6 handles was too complex, and additionally, because the reader watermark task is not connected to the rest of the tasks for a pipeline, it needs a more complicated tear down logic, where firstly the committer tasks are wound down, and then the pruner is explicitly cancelled, before its tasks are awaited as well. The change also sets up pruning configs for write-ahead log tables, so that we can test reader watermark behaviour. ## Test plan Run the indexer with just one of the consistent read summary and write-ahead logs, and then query the watermarks table: ``` cargo run -p sui-indexer-alt --release -- \ --database-url "postgres://postgres:postgrespw@localhost:5432/sui_indexer_alt" \ indexer --remote-store-url https://checkpoints.mainnet.sui.io \ --last-checkpoint 10000 \ --consistent-range 100 --consistent-pruning-interval 10 \ --pipeline sum_obj_types --pipeline wal_obj_types ``` ``` sui_indexer_alt=# SELECT * FROM watermarks; pipeline | epoch_hi_inclusive | checkpoint_hi_inclusive | tx_hi | timestamp_ms_hi_inclusive | epoch_lo | reader_lo | pruner_timestamp | pruner_hi ---------------+--------------------+-------------------------+-------+---------------------------+----------+-----------+----------------------------+----------- sum_obj_types | 1 | 9900 | 9902 | 1681405380769 | 0 | 0 | 1970-01-01 00:00:00 | 0 wal_obj_types | 1 | 10000 | 10002 | 1681405504878 | 0 | 6321 | 2024-11-10 17:33:11.459593 | 0 (2 rows) ``` Note that `reader_lo` is not guaranteed to be as advanced as it could be, because this task checks in relatively infrequently, and the indexer as a whole might shut down efore it has a chance to catch up. However, on restart, the indexer should fast-forward the watermark (i.e. if the steps above are repeated, `reader_lo` will be updated): ``` sui_indexer_alt=# SELECT * FROM watermarks; pipeline | epoch_hi_inclusive | checkpoint_hi_inclusive | tx_hi | timestamp_ms_hi_inclusive | epoch_lo | reader_lo | pruner_timestamp | pruner_hi ---------------+--------------------+-------------------------+-------+---------------------------+----------+-----------+----------------------------+----------- sum_obj_types | 1 | 9900 | 9902 | 1681405380769 | 0 | 0 | 1970-01-01 00:00:00 | 0 wal_obj_types | 1 | 10000 | 10002 | 1681405504878 | 0 | 9801 | 2024-11-10 17:33:49.913129 | 0 (2 rows) ``` commit 81474ef2fb2bc2e1ae2a39bbf599e2493504ab1d Author: Ashok Menon <[email protected]> Date: Sun Nov 10 17:15:47 2024 +0000 indexer-alt: watermarks.pruner_timestamp is a TIMESTAMP ## Description Change `watermark.pruner_timestamp_ms BIGINT` to `watermark.pruner_timestamp TIMESTAMP`. This is because unlike the other timestamps in this table, which are read from checkpoint data, this timestamp is read from the database, and so it is easiest to represent it using the database's native type for timestamps. ## Test plan CI commit 1d568438c5ca3806946d93900e1561b31d06daa3 Author: Ashok Menon <[email protected]> Date: Wed Nov 13 13:54:12 2024 +0000 chore(indexer-alt): rename tx_calls_fun -> tx_calls (#20166) ## Description I partially simplified the naming of `tx_calls_fun` to `tx_calls` but did not do a complete job -- this change completes it. ## Test plan CI ## Stack - #20118 - #20132 - #20147 - #20148 - #20149 - #20150 --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit 85c0f810e907cabd503a28c2e134e8af282351e8 Author: Ashok Menon <[email protected]> Date: Wed Nov 13 13:53:31 2024 +0000 indexer-alt: epochs pipelines (#20150) ## Description Adds two tables and pipelines: `kv_epoch_starts` and `kv_epoch_ends`, to index epoch information. This pipeline is different from epoch indexing in `sui-indexer` in a number of ways: - It is an append-only pipeline. The columns that are written at the start and end of the epoch are split into two separate tables that can be written to concurrently (by separate pipelines). - The first row of `kv_epoch_starts` is written by the bootstrap process which seeds the `kv_genesis` table as well, this avoids having to condition on whether the checkpoint being processed is the genesis checkpoint in the main pipeline. - Instead of indexing the number of transactions in the epoch, it tracks the transaction high watermark -- readers will need to read the records to calculate the number of total transactions (this avoids having to read the last epoch's total transactions in the write path). - We index the `SuiSystemState` object as BCS, rather than the summary structure. - We explicitly record whether the epoch advancement at the end of the epoch triggered safe mode (the system state object also tracks whether the epoch was started in safe mode). - Fields related to information that came from `SystemEpochInfoEvent` have all been consolidated in `kv_epoch_ends`, and they are all optional, in case of safe mode. It's worth elaborating on the last bullet point, because this is quite a subtle, but large change: - Today, `total_stake` and `storage_fund_balance` are written at the start of an epoch based on the fields of the `SystemEpochInfoEvent` emitted from the previous epoch, and are `NOT NULL`. - The remaining fields were nullable, but only because they would be written to later, once the epoch was over. This was awkward to work with in a couple of ways: - It meant that for the genesis epoch, we needed to make some numbers up (all zeroes) because we did not have an event to read from. - We had to do something similar if we hit safe mode. - When indexing the start and end of epochs separately, it meant that we had to duplicate work (finding the system epoch info event). By making the fields nullable, and consolidating them in `kv_epoch_ends`, we can simplify the pipelines: - `kv_epoch_starts` and the bootstrapping logic can work purely based on the system state object. - `kv_epoch_ends` can work purely based on the `SystemEpochInfoEvent`, and can leave fields `NULL` if we are in safe mode. In the case of `kv_epoch_starts` we could also have cut down fields to just `epoch`, `cp_lo` and `system_state`. I chose not to do this because the system state is actually quite a large object, and it is beneficial to avoid having to deserialize to answer simpler queries. ## Test plan Ran the indexer on the first 1M checkpoints, and correlated the resulting info in the respective tables from the data that the current indexer produced: ``` sui_indexer_alt=# SELECT epoch, protocol_version, cp_lo, TO_TIMESTAMP(start_timestamp_ms / 1000), reference_gas_price FROM kv_epoch_starts; epoch | protocol_version | cp_lo | to_timestamp | reference_gas_price -------+------------------+--------+------------------------+--------------------- 0 | 4 | 0 | 2023-04-12 18:00:00+01 | 1000 1 | 4 | 9770 | 2023-04-13 18:00:02+01 | 1000 2 | 4 | 85169 | 2023-04-14 18:00:04+01 | 1000 3 | 4 | 161192 | 2023-04-15 18:00:08+01 | 1000 4 | 4 | 237074 | 2023-04-16 18:00:11+01 | 1000 5 | 4 | 314160 | 2023-04-17 18:00:15+01 | 1000 6 | 4 | 391107 | 2023-04-18 18:00:18+01 | 1000 7 | 4 | 467716 | 2023-04-19 18:00:21+01 | 1000 8 | 4 | 544978 | 2023-04-20 18:00:26+01 | 1000 9 | 5 | 621933 | 2023-04-21 18:00:28+01 | 1000 10 | 6 | 699410 | 2023-04-22 18:00:31+01 | 1000 11 | 6 | 777074 | 2023-04-23 18:00:34+01 | 1000 12 | 6 | 855530 | 2023-04-24 18:00:36+01 | 1000 13 | 6 | 933559 | 2023-04-25 18:00:39+01 | 1000 (14 rows) sui_indexer_alt=# SELECT epoch, cp_hi, tx_hi, TO_TIMESTAMP(end_timestamp_ms / 1000), safe_mode, storage_fund_balance, storage_fund_reinvestment, storage_charge, storage_rebate, stake_subsidy_amount, total_gas_fees, total_stake_rewards_distributed, leftover_storage_fund_inflow FROM kv_epoch_ends ORDER BY epoch ASC; epoch | cp_hi | tx_hi | to_timestamp | safe_mode | storage_fund_balance | storage_fund_reinvestment | storage_charge | storage_rebate | stake_subsidy_amount | total_gas_fees | total_stake_rewards_distributed | leftover_storage_fund_inflow -------+--------+--------+------------------------+-----------+----------------------+---------------------------+----------------+----------------+----------------------+----------------+---------------------------------+------------------------------ 0 | 9770 | 9771 | 2023-04-13 18:00:02+01 | f | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 1 | 85169 | 85174 | 2023-04-14 18:00:04+01 | f | 2973880 | 0 | 3952000 | 978120 | 0 | 102000000 | 102000000 | 0 2 | 161192 | 161199 | 2023-04-15 18:00:08+01 | f | 717398960 | 0 | 715403200 | 978120 | 0 | 1000000 | 1000000 | 0 3 | 237074 | 237084 | 2023-04-16 18:00:11+01 | f | 733657184 | 0 | 1430198400 | 1413940176 | 0 | 2000000 | 2000000 | 0 4 | 314160 | 314171 | 2023-04-17 18:00:15+01 | f | 733657184 | 0 | 0 | 0 | 0 | 0 | 0 | 0 5 | 391107 | 391119 | 2023-04-18 18:00:18+01 | f | 733657184 | 0 | 0 | 0 | 0 | 0 | 0 | 0 6 | 467716 | 467730 | 2023-04-19 18:00:21+01 | f | 735633184 | 0 | 1976000 | 0 | 0 | 1000000 | 1000000 | 0 7 | 544978 | 544994 | 2023-04-20 18:00:26+01 | f | 729859616 | 0 | 702475600 | 708249168 | 0 | 1000000 | 1000000 | 0 8 | 621933 | 621950 | 2023-04-21 18:00:28+01 | f | 729859616 | 0 | 0 | 0 | 0 | 0 | 0 | 0 9 | 699410 | 699428 | 2023-04-22 18:00:31+01 | f | 729859616 | 0 | 0 | 0 | 0 | 0 | 0 | 0 10 | 777074 | 777093 | 2023-04-23 18:00:34+01 | f | 729859616 | 0 | 0 | 0 | 0 | 0 | 0 | 0 11 | 855530 | 855550 | 2023-04-24 18:00:36+01 | f | 729859616 | 0 | 0 | 0 | 0 | 0 | 0 | 0 12 | 933559 | 933586 | 2023-04-25 18:00:39+01 | f | 735866656 | 0 | 13832000 | 7824960 | 0 | 6000000 | 6000000 | 0 (13 rows) ``` ## Stack - #20118 - #20132 - #20147 - #20148 - #20149 --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit e96fca283cf8f00fd09291fdbdcd401c2063b71b Author: Ashok Menon <[email protected]> Date: Sat Nov 2 02:05:09 2024 +0000 indexer-alt: kv_feature_flags pipeline ## Description Like `kv_protocol_configs`, but for feature flags. ## Test plan Run the indexer over the first 700K checkpoints -- this will include two protocol config upgrades, then check the database for feature flags. ``` sui$ cargo run -p sui-indexer-alt --release -- \ --database-url "postgres://postgres:postgrespw@localhost:5432/sui_indexer_alt" \ indexer --remote-store-url https://checkpoints.mainnet.sui.io \ --last-checkpoint 700000 ``` ``` sui_indexer_alt=# SELECT * FROM kv_feature_flags ORDER BY protocol_version DESC, flag_name; protocol_version | flag_name | flag_value ------------------+-------------------------------------------------------------+------------ 6 | accept_zklogin_in_multisig | f 6 | advance_epoch_start_time_in_safe_mode | t 6 | advance_to_highest_supported_protocol_version | f 6 | allow_receiving_object_id | f 6 | authority_capabilities_v2 | f [...] ``` commit 1aab0cf2b23b6d682446eae86937542f9f698e3c Author: Ashok Menon <[email protected]> Date: Sat Nov 2 01:40:35 2024 +0000 indexer-alt: kv_protocol_configs pipeline ## Description Add a pipeline for indexing protocol configs. ## Test plan Index the first ~1M checkpoints, which should pick up the first protocol version upgrade (from 4 to 5 for epoch 10) and then query `kv_protocol_configs` results after that: ``` sui$ cargo run -p sui-indexer-alt -- \ --database-url postgres://postgres:postgrespw@localhost:5432/sui_indexer_alt \ --remote-store-url https://checkpoints.mainnet.sui.io \ --last-checkpoint 500000 ``` ``` sui_indexer_alt=# SELECT * FROM kv_protocol_configs; ``` commit 97932677f340404a8843ac8fb46ae56ea0b7501c Author: Ashok Menon <[email protected]> Date: Sat Nov 2 01:18:58 2024 +0000 refactor(indexer-alt): configurable processor ## Description Change the pipeline/handler/processor interface to accept a value representing the handler/processor. This allows the processor to include configuration. This will be used by upcoming pipelines for processing protocol configs and feature flags. ## Test plan CI commit ae6275f6957ac936caffe04e82b7762ba192c4f8 Author: Anastasios Kichidis <[email protected]> Date: Wed Nov 13 12:38:22 2024 +0000 [Consensus] resubmit GC'ed transactions (#19572) ## Description Transactions of GC'ed blocks should be retried to remain compliant with the so far system assumptions. Refactored the consensus side to signal back the submitter about the status of the block that the submitted transaction has been included to. Practically two possible outcomes: 1. Sequenced (when the block has been committed and consequently the tx as well) 2. GarbageCollected (when the tx's block has not been committed and passed gc_round) also took the opportunity for a small refactoring around the `SubmitToConsensus` trait and the `ConsensusClient` as intentions started diverging. ## Test plan CI/PT --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit bb0288b9b8de0307690a434eaf62a930cf1280e8 Author: benr-ml <[email protected]> Date: Wed Nov 13 14:35:19 2024 +0200 Enable G1Uncompressed in testnet (#20226) ## Description Describe the changes or additions included in this PR. ## Test plan How did you test the new or updated feature? --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit 6ea6bc9b112bf34a3dcfb136430726924b03e0d0 Author: Mark Logan <[email protected]> Date: Tue Nov 12 23:08:08 2024 -0800 Fixes for overload protection (#20248) - Only check overload queues for mutable shared inputs - Increase max tx queue age to 500ms ## Description Describe the changes or additions included in this PR. ## Test plan How did you test the new or updated feature? --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit 297913c0adcb677eb369d520e9de7c80696fbde3 Author: Mark Logan <[email protected]> Date: Tue Nov 12 21:21:06 2024 -0800 Fix bug with object_by_id inserts. (#20236) The previous version of this code was based on a misunderstanding of the Moka Cache API. It assumed that: let entry = cache.entry(k).or_insert_with(|| v); entry.is_fresh() could be used to detect whether there was a race with cache.insert(k, v) In fact this is not the case! Both threads must use `or_insert_with` in order to handle the race correctly. The bug is instantly reproducible with the following code: use moka::sync::Cache; use std::sync::{Arc, Mutex}; fn monotonic_update(cache: Arc<Cache<u64, Arc<Mutex<u64>>>>, key: u64, value: u64) { let entry = cache .entry(key) .or_insert_with(|| Arc::new(Mutex::new(value))); if !entry.is_fresh() { let mut entry = entry.value().lock().unwrap(); // only update if the new value is greater than the current value if value > *entry { *entry = value; } } } fn blind_write(cache: Arc<Cache<u64, Arc<Mutex<u64>>>>, key: u64, value: u64) { cache.insert(key, Arc::new(Mutex::new(value))); } fn main() { for _ in 0..1000 { let cache = Arc::new(Cache::new(1000)); let cache1 = cache.clone(); let cache2 = cache.clone(); let handle1 = std::thread::spawn(move || { monotonic_update(cache1, 1, 1); }); let handle2 = std::thread::spawn(move || { // Correct way to update the value // monotonic_update(cache2, 1, 2); // Incorrect way to update the value blind_write(cache2, 1, 2); }); handle1.join().unwrap(); handle2.join().unwrap(); let entry = cache.get(&1).unwrap(); let value = entry.lock().unwrap(); assert_eq!(*value, 2); } } ## Description Describe the changes or additions included in this PR. ## Test plan How did you test the new or updated feature? --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit ec6d01642ad68b1dea024a3f34d5966359a3617e Author: Todd Nowacki <[email protected]> Date: Tue Nov 12 15:51:12 2024 -0800 [move-compiler] Improve interactions with Sui mode and interface files (#20196) ## Description - Sui mode used only named addresses for type comparison. This is not sufficient for interface files that might be using only the numerical address. - Switched `is` to use only the numerical address. We might want to evaluate differently if we start using it more outside of Sui mode ## Test plan - Ran tests --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit 7b61e1ff70f4b2fbfd67085db2f795bcaa291105 Author: Avrora <[email protected]> Date: Wed Nov 13 01:42:06 2024 +0200 Docs: Fix typo (#20225) Fix typo ## Description Describe the changes or additions included in this PR. ## Test plan How did you test the new or updated feature? --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: --------- Co-authored-by: ronny-mysten <[email protected]> commit f716589e9b7d966cea77dbe8145ed892fe4fd0cd Author: ronny-mysten <[email protected]> Date: Tue Nov 12 16:41:39 2024 -0700 [docs] Removed orphaned page (#20228) ## Description Removed orphaned page and updates the link that pointed to it. ## Test plan How did you test the new or updated feature? --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit d5a23d78a9b30ec1c5ae5ac3cafdbe2817d68df6 Author: hayes-mysten <[email protected]> Date: Tue Nov 12 12:49:03 2024 -0800 [ts-sdk] Add tx.object.option for creatnig object options in transact… (#20194) …ion builder ## Description Describe the changes or additions included in this PR. ## Test plan How did you test the new or updated feature? --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit de2c7de6f6b5d643560c0ca343b9009c96e7fefd Author: benr-ml <[email protected]> Date: Tue Nov 12 21:44:54 2024 +0200 Update fastcrypto, including moving some structs between dkg->dkg_v1 (#20224) ## Description Describe the changes or additions included in this PR. ## Test plan How did you test the new or updated feature? --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit 44fe9eb64fed3a10ad5da8d3ad0d542b2f06b281 Author: Andrew Schran <[email protected]> Date: Tue Nov 12 14:44:34 2024 -0500 Revert "Add support for TLS connections with self-signed cert on validator gRPC interface" (#20232) Reverts MystenLabs/sui#19796 due to suspicion it's causing performance issues commit 73dbd005bb0f828a64eb4e811d4f8e2b3c80bdfe Author: sui-merge-bot[bot] <114704316+sui-merge-bot[bot]@users.noreply.github.com> Date: Tue Nov 12 13:38:32 2024 -0500 Version Packages (#20229) This PR was opened by the [Changesets release](https://github.com/changesets/action) GitHub action. When you're ready to do a release, you can merge this and publish to npm yourself or [setup this action to publish automatically](https://github.com/changesets/action#with-publishing). If you're not ready to do a release yet, that's fine, whenever you add more changesets to main, this PR will be updated. # Releases ## @mysten/[email protected] ### Patch Changes - a24d43d: getOrders and decodeOrderId support Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> commit c4afb26067e9a4e9eea75a706032f1764c4c3ded Author: Xun Li <[email protected]> Date: Tue Nov 12 10:17:08 2024 -0800 [indexer-alt] Add checkpoint ingestion lag metrics (#20191) ## Description This PR adds two metrics, one that tracks the ingestion lag on the latest fetched checkpoint, one that tracks histogram. These two combined could give us the full picture on the lag. ## Test plan CI. --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit 4ba4b29a0ca9c473e89a1ba5b01caf71c31c5c21 Author: phoenix <[email protected]> Date: Tue Nov 12 12:47:34 2024 -0500 [kv store] add timeout to bigtable ingestion + backfill mode (#20193) ## Description adds timeouts to the Bigtable ingestion pipeline and introduces a backfill mode that skips most progress updates to DynamoDB during backfilling --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit a24d43d709e1c05f98b092d50c2ac87b5c172489 Author: Tony Lee <[email protected]> Date: Tue Nov 12 12:38:38 2024 -0500 Deepbook SDK Examples (#20227) ## Description Getting balance for balance manager, getting live orders for a balance manager, and price/quantity map for a balance manager. Balance: ``` Manager: MANAGER_1 { coinType: '0x0000000000000000000000000000000000000000000000000000000000000002::sui::SUI', balance: 374527.5686 } { coinType: '0xdba34672e30cb065b1f93e3ab55318768fd6fef66c15942c9f7cb846e2f900e7::usdc::USDC', balance: 1492902.476423 } ``` Live Orders: ``` Manager: MANAGER_1 SUI_USDC [ '170141183460526324404595434778137174870', '170141183460526490425292098164101718871', '170141183460526490425292098164101718872', '56723756473400944927347700', '56502395544516430307955699', '55819866013789176898163698', '56723756473400944927347696' ] DEEP_SUI [ '413576002151014891304428578578', '421323634661972902983148578577', '170141183888249226801011805690925112362', '414498339354700368885228578576' ] ``` Price/Quantity Map: ``` Manager: MANAGER_1 SUI_USDC bid orders: [ [ 3.046, 6.6 ], [ 3.035, 21418.1 ], [ 3.027, 26427.4 ], [ 2.995, 46747.6 ] ] SUI_USDC ask orders: [ [ 3.049, 6.6 ], [ 3.065, 21207.2 ], [ 3.073, 26034.1 ], [ 3.096, 45223.3 ] ] ``` ## Test plan How did you test the new or updated feature? Locally tested using mainnet ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading …
carlosbmamoru
pushed a commit
to Mamoru-Foundation/sui
that referenced
this pull request
Nov 19, 2024
commit cd214a3d7e8e9e47ccf31ab8f5315e8c114313be Author: wlmyng <[email protected]> Date: Fri Nov 15 13:21:24 2024 -0800 [mvr] Introduce mvr-mode on production indexer (#20271) (#20288) ## Description introduce mvr-mode flag. If true, enables pruning objects_history, and toggles committer to write only a subset of tables to db. Compatible with the production indexer schema. ## Test plan How did you test the new or updated feature? --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: ## Description Describe the changes or additions included in this PR. ## Test plan How did you test the new or updated feature? --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit ab6863576baf7ff74f0de95ab5abef9040fc8fcd Author: Eugene Boguslavsky <[email protected]> Date: Thu Nov 14 18:41:02 2024 -0800 Sui `v1.38.0` Framework Bytecode Snapshot (#20277) ## Description Sui `v1.38.0` Framework Bytecode Snapshot ## Test plan 👀 commit 5d70ec6f5d4b7ff611649432aeb8c538efb63e80 Author: Sadhan Sood <[email protected]> Date: Thu Nov 14 17:08:27 2024 -0800 feat(analytics): Add package filter to object handler (#20272) ## Description This PR adds a package filter for object handler in analytics for being able to generate an object table specifically for walrus in testnet. ## Test Invoking with `--package-id-filter 0x9f992cc2430a1f442ca7a5ca7638169f5d5c00e0ebc3977a65e9ac6e497fe5ef --starting-checkpoint-seq-num 120132840` to filter for walrus specific objects. Here is the walrus package object: https://testnet.suivision.xyz/package/0x9f992cc2430a1f442ca7a5ca7638169f5d5c00e0ebc3977a65e9ac6e497fe5ef commit b36024d32ce349ba9465b38bd6706b8b2068ee67 Author: stefan-mysten <[email protected]> Date: Thu Nov 14 16:27:12 2024 -0800 [CLI] Add back the warning about version mismatch (#20273) ## Description #20259 removed the warning for the SDK, so this PR adds it back for the CLI. ## Test plan How did you test the new or updated feature? --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit 81f75843464ce1140040403df0a0fa0852f87386 Author: Adam Welc <[email protected]> Date: Thu Nov 14 13:01:50 2024 -0800 [move-compiler] Added function location info to the source map (#20261) ## Description This PR adds additional field to the source map describing the location of the entire function. This is needed to enable better handling of macros in the trace viewer (we need to know which instructions in a given function are outside of a function body to identify those that belong to inlined macro functions). ## Test plan All existing tests must pass commit 8d90acd980b4490a7f5488cee2aad465abe8e663 Author: Tim Zakian <[email protected]> Date: Thu Nov 14 11:53:41 2024 -0800 [json-rpc] Add enums in `getNormalizedMoveModule` (#20231) ## Description Adds enums to the `NormalizedMoveModule` for json-rpc. This fixes #20205 ## Test plan Tested manually on the repro provided in the issue to make sure enums are included in the output. --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [x] JSON-RPC: Fixes an issue where enum types were not included in the output of `getNormalizedMoveModule` and `getNormalizedMoveModuleForPackage` - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit 53dd080a841710c91f492de17f7312a29662235f Author: William Smith <[email protected]> Date: Thu Nov 14 14:03:22 2024 -0500 [TrafficControl] Add salt to TrafficSketch (#20245) ## Description False positives stem from IP collisions during traffic tallying. Because validators tend to see closely correlated transaction execution traffic patterns, it is possible that a false positive resulting in an erroneously blocked IP could be correlated across many validators at the same time. If this happens on a quorum of validators, this would result in an IP being effectively denied service until the TTL expires. Luckily, we can exploit this property to lower the effective false positive rate (as observed by the client) by removing this correlation. This PR adds a salt key to the `SketchKey`. The value is generated in memory at startup. ## Test plan Existing tests --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit 97838b20eb28b5abd12d275f5924c70c97b86e2c Author: sui-merge-bot[bot] <114704316+sui-merge-bot[bot]@users.noreply.github.com> Date: Thu Nov 14 10:25:13 2024 -0800 Version Packages (#20269) This PR was opened by the [Changesets release](https://github.com/changesets/action) GitHub action. When you're ready to do a release, you can merge this and publish to npm yourself or [setup this action to publish automatically](https://github.com/changesets/action#with-publishing). If you're not ready to do a release yet, that's fine, whenever you add more changesets to main, this PR will be updated. # Releases ## @mysten/[email protected] ### Patch Changes - @mysten/[email protected] ## @mysten/[email protected] ### Patch Changes - Updated dependencies [1765780] - @mysten/[email protected] ## @mysten/[email protected] ### Patch Changes - 1765780: fix zksend link network detection Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> commit 176578020ba15ff321d42b487f87af0a650bece3 Author: Pavlos Chrysochoidis <[email protected]> Date: Thu Nov 14 18:10:02 2024 +0000 zksend sdk fix network detection (#20268) ## Description Describe the changes or additions included in this PR. ## Test plan How did you test the new or updated feature? --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit 28e184a694636fc3b5a589c219f47b549250d6c1 Author: xiaodi <[email protected]> Date: Fri Nov 15 00:24:24 2024 +0800 Update regulated.mdx (#20267) ## Description Replaced "Regulated Coin and Deny List" with "Loyalty Tokens" in documentation. ## Test plan How did you test the new or updated feature? --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit 0bd58999ae2be185c6677ebc4a44b3e5dcd15296 Author: sui-merge-bot[bot] <114704316+sui-merge-bot[bot]@users.noreply.github.com> Date: Thu Nov 14 09:18:08 2024 -0500 Version Packages (#20265) This PR was opened by the [Changesets release](https://github.com/changesets/action) GitHub action. When you're ready to do a release, you can merge this and publish to npm yourself or [setup this action to publish automatically](https://github.com/changesets/action#with-publishing). If you're not ready to do a release yet, that's fine, whenever you add more changesets to main, this PR will be updated. # Releases ## @mysten/[email protected] ### Patch Changes - 4a42691: NS token pools added Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> commit 4a426918ef263103020aca25cf05809e4531ec51 Author: Tony Lee <[email protected]> Date: Thu Nov 14 08:26:36 2024 -0500 NS Deepbook Pools (#20256) commit d5d51aed177432e62950385336b79f8395c940f9 Author: stefan-mysten <[email protected]> Date: Wed Nov 13 17:17:13 2024 -0800 [RustSDK] Less noise when rpc version and local CLI version mismatch (#20259) ## Description This PR removes the SDK tracing warning on mismatch local + RPC API versions. ## Test plan How did you test the new or updated feature? --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit c85e7c07c0713c1edf7cdb59eab46372ef29fe24 Author: Tim Zakian <[email protected]> Date: Wed Nov 13 17:16:49 2024 -0800 [docs] Update Sui CLI install instructions to include the tracing feature (#20255) ## Description Update docs for installing the Sui CLI to include `--features tracing` so that they work with the trace viewer/debugger. ## Test plan 👀 commit 9dfd26a723e5efdb5816d6c2492cb0bb27b2f2c5 Author: Eugene Boguslavsky <[email protected]> Date: Wed Nov 13 15:33:46 2024 -0800 Use tracing feature for sui builds (#20260) ## Description Use tracing feature for sui builds ## Test plan 👀 commit 047de09d84bf90b633fd183b0cecef38e9387317 Author: Xun Li <[email protected]> Date: Wed Nov 13 14:30:23 2024 -0800 [indexer-alt] Add processor lag metrics (#20235) ## Description Describe the changes or additions included in this PR. ## Test plan How did you test the new or updated feature? --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit c03132d75e80b569c2fd8447539ac0b7bddc8ecc Author: Andrew Schran <[email protected]> Date: Wed Nov 13 16:51:15 2024 -0500 Add support for TLS connections with self-signed cert on validator gRPC interface (#20238) Client side use of TLS will be enabled-by-default in a future PR. This has corrected TLS upgrade behavior compared to older PR #19796. --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [x] Nodes (Validators and Full nodes): Adds support for TLS on validator gRPC interface. - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit 313bf8b43a1b8b511cb9b012a23d5803444b3ea5 Author: Tim Zakian <[email protected]> Date: Wed Nov 13 13:16:01 2024 -0800 [move] Add Move 2024 (non-beta) edition and make it the default edition (#20242) ## Description Add Move `2024` edition and make it the default edition. ## Test plan Updated existing tests. --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [X] CLI: Added the Move `2024` edition, and set it as the default Move edition both when creating a new Move package and when migrating from an existing package. Existing `2024.beta` and `2024.alpha` editions remain supported. - [ ] Rust SDK: - [ ] REST API: commit 8f21cd702b8ec9f4796f317aa9ec6bfd56330f0e Author: phoenix <[email protected]> Date: Wed Nov 13 15:26:23 2024 -0500 [data ingestion] bigtable: set env credentials parameter (#20240) ## Description Pass filename with credential parameters to the ingestion binary --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit 3c444d5dada018e5b10a929efa6f17e3445367d5 Author: Adam Welc <[email protected]> Date: Wed Nov 13 11:04:35 2024 -0800 [move-ide] Fixed auto-completion on the dot character (#20175) ## Description This PR fixes a certain problem with auto-completion on the `.` character. Previously, once a part of the access chain after the `.` was parsed, auto-completion information for the `.` character location was no longer generated. This is particularly problematic if someone expects auto-completion in the middle of the function code, for example: ```move public struct SomeStruct has drop { some_field: u64, } public fun foo(_param: SomeStruct) {} public fun bar(): SomeStruct { let val = SomeStruct{some_field: 42}; val. SomeStruct {some_field: 42} } ``` It is expected that auto-completion would work correctly after `val.` but previously it did not because this expression was actually parsed as `val.SomeStruct`. This PR fixes this:  Previously, auto-completion for the `.` location would also not work if what follows the `.` is a function call (whether the function being called was resolved correctly or not). It now works correctly in both cases:   ## Test plan All new and (updated) all tests must pass commit 8d647c0f00809bba411079ce765fdb4fbecef00c Author: sui-merge-bot[bot] <114704316+sui-merge-bot[bot]@users.noreply.github.com> Date: Wed Nov 13 18:49:45 2024 +0000 Version Packages (#20252) This PR was opened by the [Changesets release](https://github.com/changesets/action) GitHub action. When you're ready to do a release, you can merge this and publish to npm yourself or [setup this action to publish automatically](https://github.com/changesets/action#with-publishing). If you're not ready to do a release yet, that's fine, whenever you add more changesets to main, this PR will be updated. # Releases ## @mysten/[email protected] ### Minor Changes - 60bdb62: Update graphql query and use stashed host for claim api - 364a93a: Remove decoding spaces to `+` in zksend wallet url hashes - 364a93a: Add claimedBy property that is set when claimedAssets are loaded ## @mysten/[email protected] ### Patch Changes - @mysten/[email protected] ## @mysten/[email protected] ### Patch Changes - Updated dependencies [60bdb62] - Updated dependencies [364a93a] - Updated dependencies [364a93a] - @mysten/[email protected] Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> commit 8117d8393f25fa85390094bca61952ab24c9b595 Author: Tim Zakian <[email protected]> Date: Wed Nov 13 10:38:18 2024 -0800 [move/tracing] Panic unit tests if tracing without being compiled with tracing support (#20253) ## Description Make sure that if the `--trace-execution` flag is passed that the binary was compiled with the `tracing` feature flag set. ## Test plan CI --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit 60bdb62627a821bb54b7c7415a63193b7395a05d Author: hayes-mysten <[email protected]> Date: Wed Nov 13 10:08:56 2024 -0800 [zksend] Update garphql query, and use getstashed.com as claim api host (#20237) ## Description fixes #20222 Describe the changes or additions included in this PR. ## Test plan How did you test the new or updated feature? --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit 364a93a1b10be70aabe0fb8937b3c1af610d1eba Author: hayes-mysten <[email protected]> Date: Wed Nov 13 10:00:16 2024 -0800 [zksend] add claimedBy property when loading assets of claimed link (#20244) ## Description Describe the changes or additions included in this PR. ## Test plan How did you test the new or updated feature? --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit 43d7c3dcfcdf062cad98b480f0c0a62e218026e9 Author: Xun Li <[email protected]> Date: Wed Nov 13 09:14:22 2024 -0800 [indexer-alt] Skip commit if batch is empty (#20124) ## Description If we did not collect anything to the batch, we should not commit to the DB. We should also avoid reporting any metrics. ## Test plan CI --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: --------- Co-authored-by: Ashok Menon <[email protected]> commit f04e4cddfdde35a5d5cba9c7fd7c0f9482d4193b Author: sui-merge-bot[bot] <114704316+sui-merge-bot[bot]@users.noreply.github.com> Date: Wed Nov 13 11:07:19 2024 -0500 Version Packages (#20239) This PR was opened by the [Changesets release](https://github.com/changesets/action) GitHub action. When you're ready to do a release, you can merge this and publish to npm yourself or [setup this action to publish automatically](https://github.com/changesets/action#with-publishing). If you're not ready to do a release yet, that's fine, whenever you add more changesets to main, this PR will be updated. # Releases ## @mysten/[email protected] ### Patch Changes - Updated dependencies [d5a23d7] - @mysten/[email protected] - @mysten/[email protected] ## @mysten/[email protected] ### Patch Changes - Updated dependencies [d5a23d7] - @mysten/[email protected] - @mysten/[email protected] - @mysten/[email protected] ## @mysten/[email protected] ### Patch Changes - Updated dependencies [d5a23d7] - @mysten/[email protected] ## @mysten/[email protected] ### Patch Changes - a43ae92: NS Token - Updated dependencies [d5a23d7] - @mysten/[email protected] ## @mysten/[email protected] ### Patch Changes - Updated dependencies [d5a23d7] - @mysten/[email protected] - @mysten/[email protected] ## @mysten/[email protected] ### Patch Changes - Updated dependencies [d5a23d7] - @mysten/[email protected] ## @mysten/[email protected] ### Patch Changes - Updated dependencies [d5a23d7] - @mysten/[email protected] ## @mysten/[email protected] ### Patch Changes - Updated dependencies [d5a23d7] - @mysten/[email protected] ## @mysten/[email protected] ### Patch Changes - Updated dependencies [d5a23d7] - @mysten/[email protected] ## @mysten/[email protected] ### Patch Changes - d5a23d7: Add tx.object.option for creatnig object options in transaction builder ## @mysten/[email protected] ### Patch Changes - Updated dependencies [d5a23d7] - @mysten/[email protected] ## @mysten/[email protected] ### Patch Changes - Updated dependencies [d5a23d7] - @mysten/[email protected] ## @mysten/[email protected] ### Patch Changes - Updated dependencies [d5a23d7] - @mysten/[email protected] - @mysten/[email protected] Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> commit a43ae921386d1e8406f81d73d2c317ad92bb211e Author: Tony Lee <[email protected]> Date: Wed Nov 13 10:38:16 2024 -0500 NS token in deepbook sdk (#20251) ## Description NS token in deepbook sdk ## Test plan How did you test the new or updated feature? Testnet and local ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit 91dec7c02d776f3e68a5a1e5b86851c7d289c206 Author: Ashok Menon <[email protected]> Date: Wed Nov 13 14:00:52 2024 +0000 indexer-alt: remove watermarks.epoch_lo (#20218) ## Description This field turns out not to be used by the new pruner implementation, because it is entirely based on checkpoints. ## Test plan CI ## Stack - #20149 - #20150 - #20166 - #20216 - #20217 --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit c5f275ea9fab4a099e998a7b4f65926fe1ae8e8e Author: Ashok Menon <[email protected]> Date: Mon Nov 11 02:43:06 2024 +0000 indexer-alt: pruner task ## Description Add the task that actually deletes data, based on the reader low watermark. ## Test plan Run the indexer and note the following: - Metrics related to deleted rows by the pruner (from `localhost:9184/metrics`) - The contents of the `watermarks` table. ``` sui$ cargo run -p sui-indexer-alt --release -- \ --database-url "postgres://postgres:postgrespw@localhost:5432/sui_indexer_alt" \ indexer --remote-store-url https://checkpoints.mainnet.sui.io \ --last-checkpoint 10000 \ --consistent-range 100 --consistent-pruning-interval 10 \ --pipeline sum_obj_types --pipeline wal_obj_types ``` Also tested running the indexer for an extended period of time (1M checkpoints over roughly half an hour in local testing), and noted how the pruner behaves. When configured as it would be in production (roughly one hour of consistent range, and a 5 minute pruning interval and a 2 minute pruning delay): - Many rows accumulated during backfill -- by the end of the 1M checkpoints, the pruner had only pruned up to between checkpoint 500K and checkpoint 700K depending on the pipeline. This should not be an issue under normal operation where the indexer will run for long enough for pruning to stabilise at the tip of the network (and it would be recommended practice to start from formal snapshot and therefore only need to run pruning from that point forward). - Because the reader watermark task and the pruner task use the same interval, it can take up to two ticks of that interval for the pruner to act on a change to its upperbound -- again, it should be okay, as the pruner's interval should be at least an order of magnitude smaller than its retention period. commit 3a2e332c68cfe9491eb56d09f087ff8cc559e6e4 Author: Ashok Menon <[email protected]> Date: Mon Nov 11 02:37:06 2024 +0000 fix(indexer-alt): overly chatty loud watermark updates ## Description The loud watermark update logic always bumped the previous update watermark by the same interval. When the indexer was running well ahead of the loud watermark update rate, this would cause many updates to be issued successively. This change makes it so that the next update is always `LOUD_WATERMARK_UPDATE_INTERVAL` away from the last loud update. ## Test plan Run the indexer. Previously updates -- particularly from summary tables that gathered changes up and write them out in big batches -- would come in bursts, and this behaviour is no longer apparent after the change: ``` sui$ cargo run -p sui-indexer --release -- \ --database-url "postgres://postgres:postgrespw@localhost:5432/sui_indexer_alt" \ indexer --remote-store-url https://checkpoints.mainnet.sui.io \ --last-checkpoint 1000000 --consistent-range 3600 ``` commit 6d31be68d0ef82f90aa0447e13fd4c82eb40f732 Author: Ashok Menon <[email protected]> Date: Sun Nov 10 17:19:53 2024 +0000 indexer-alt: reader watermark task ## Description The reader watermark task is one half of the pruning implementation. It applies only to concurrent pipelines, and tracks the lowest checkpoint sequence number the reader is guaranteed to get data for, and the timestamp (from the DB), that this threshold was set at. This task does not do any actual pruning, but it sets the limit that the pruner can go to, and a time at which it can start pruning to that point. A later PR will introduce the actual pruning task. Unlike the existing pruner implementation, this system does not support pruning by epoch, only by checkpoint. This was for two reasons: - Pruning no longer works by dropping partitions, so we can keep the implementation simple by only supporting the form of pruning we need (by checkpoint sequence number). - Pruning is part of the indexing framework, rather than a specific part of our indexer implementation, so pruning by epoch would require that the indexing framework kept track of a mapping from epoch to checkpoint sequence number. Along with the reader watermark, the following changes were made: - The existing watermark task was renamed to the `committer_watermark` to distinguish it. - The interface between pipelines and the indexing framework has been simplified to return just one `JoinHandle`. This was because clippy complained that returning 6 handles was too complex, and additionally, because the reader watermark task is not connected to the rest of the tasks for a pipeline, it needs a more complicated tear down logic, where firstly the committer tasks are wound down, and then the pruner is explicitly cancelled, before its tasks are awaited as well. The change also sets up pruning configs for write-ahead log tables, so that we can test reader watermark behaviour. ## Test plan Run the indexer with just one of the consistent read summary and write-ahead logs, and then query the watermarks table: ``` cargo run -p sui-indexer-alt --release -- \ --database-url "postgres://postgres:postgrespw@localhost:5432/sui_indexer_alt" \ indexer --remote-store-url https://checkpoints.mainnet.sui.io \ --last-checkpoint 10000 \ --consistent-range 100 --consistent-pruning-interval 10 \ --pipeline sum_obj_types --pipeline wal_obj_types ``` ``` sui_indexer_alt=# SELECT * FROM watermarks; pipeline | epoch_hi_inclusive | checkpoint_hi_inclusive | tx_hi | timestamp_ms_hi_inclusive | epoch_lo | reader_lo | pruner_timestamp | pruner_hi ---------------+--------------------+-------------------------+-------+---------------------------+----------+-----------+----------------------------+----------- sum_obj_types | 1 | 9900 | 9902 | 1681405380769 | 0 | 0 | 1970-01-01 00:00:00 | 0 wal_obj_types | 1 | 10000 | 10002 | 1681405504878 | 0 | 6321 | 2024-11-10 17:33:11.459593 | 0 (2 rows) ``` Note that `reader_lo` is not guaranteed to be as advanced as it could be, because this task checks in relatively infrequently, and the indexer as a whole might shut down efore it has a chance to catch up. However, on restart, the indexer should fast-forward the watermark (i.e. if the steps above are repeated, `reader_lo` will be updated): ``` sui_indexer_alt=# SELECT * FROM watermarks; pipeline | epoch_hi_inclusive | checkpoint_hi_inclusive | tx_hi | timestamp_ms_hi_inclusive | epoch_lo | reader_lo | pruner_timestamp | pruner_hi ---------------+--------------------+-------------------------+-------+---------------------------+----------+-----------+----------------------------+----------- sum_obj_types | 1 | 9900 | 9902 | 1681405380769 | 0 | 0 | 1970-01-01 00:00:00 | 0 wal_obj_types | 1 | 10000 | 10002 | 1681405504878 | 0 | 9801 | 2024-11-10 17:33:49.913129 | 0 (2 rows) ``` commit 81474ef2fb2bc2e1ae2a39bbf599e2493504ab1d Author: Ashok Menon <[email protected]> Date: Sun Nov 10 17:15:47 2024 +0000 indexer-alt: watermarks.pruner_timestamp is a TIMESTAMP ## Description Change `watermark.pruner_timestamp_ms BIGINT` to `watermark.pruner_timestamp TIMESTAMP`. This is because unlike the other timestamps in this table, which are read from checkpoint data, this timestamp is read from the database, and so it is easiest to represent it using the database's native type for timestamps. ## Test plan CI commit 1d568438c5ca3806946d93900e1561b31d06daa3 Author: Ashok Menon <[email protected]> Date: Wed Nov 13 13:54:12 2024 +0000 chore(indexer-alt): rename tx_calls_fun -> tx_calls (#20166) ## Description I partially simplified the naming of `tx_calls_fun` to `tx_calls` but did not do a complete job -- this change completes it. ## Test plan CI ## Stack - #20118 - #20132 - #20147 - #20148 - #20149 - #20150 --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit 85c0f810e907cabd503a28c2e134e8af282351e8 Author: Ashok Menon <[email protected]> Date: Wed Nov 13 13:53:31 2024 +0000 indexer-alt: epochs pipelines (#20150) ## Description Adds two tables and pipelines: `kv_epoch_starts` and `kv_epoch_ends`, to index epoch information. This pipeline is different from epoch indexing in `sui-indexer` in a number of ways: - It is an append-only pipeline. The columns that are written at the start and end of the epoch are split into two separate tables that can be written to concurrently (by separate pipelines). - The first row of `kv_epoch_starts` is written by the bootstrap process which seeds the `kv_genesis` table as well, this avoids having to condition on whether the checkpoint being processed is the genesis checkpoint in the main pipeline. - Instead of indexing the number of transactions in the epoch, it tracks the transaction high watermark -- readers will need to read the records to calculate the number of total transactions (this avoids having to read the last epoch's total transactions in the write path). - We index the `SuiSystemState` object as BCS, rather than the summary structure. - We explicitly record whether the epoch advancement at the end of the epoch triggered safe mode (the system state object also tracks whether the epoch was started in safe mode). - Fields related to information that came from `SystemEpochInfoEvent` have all been consolidated in `kv_epoch_ends`, and they are all optional, in case of safe mode. It's worth elaborating on the last bullet point, because this is quite a subtle, but large change: - Today, `total_stake` and `storage_fund_balance` are written at the start of an epoch based on the fields of the `SystemEpochInfoEvent` emitted from the previous epoch, and are `NOT NULL`. - The remaining fields were nullable, but only because they would be written to later, once the epoch was over. This was awkward to work with in a couple of ways: - It meant that for the genesis epoch, we needed to make some numbers up (all zeroes) because we did not have an event to read from. - We had to do something similar if we hit safe mode. - When indexing the start and end of epochs separately, it meant that we had to duplicate work (finding the system epoch info event). By making the fields nullable, and consolidating them in `kv_epoch_ends`, we can simplify the pipelines: - `kv_epoch_starts` and the bootstrapping logic can work purely based on the system state object. - `kv_epoch_ends` can work purely based on the `SystemEpochInfoEvent`, and can leave fields `NULL` if we are in safe mode. In the case of `kv_epoch_starts` we could also have cut down fields to just `epoch`, `cp_lo` and `system_state`. I chose not to do this because the system state is actually quite a large object, and it is beneficial to avoid having to deserialize to answer simpler queries. ## Test plan Ran the indexer on the first 1M checkpoints, and correlated the resulting info in the respective tables from the data that the current indexer produced: ``` sui_indexer_alt=# SELECT epoch, protocol_version, cp_lo, TO_TIMESTAMP(start_timestamp_ms / 1000), reference_gas_price FROM kv_epoch_starts; epoch | protocol_version | cp_lo | to_timestamp | reference_gas_price -------+------------------+--------+------------------------+--------------------- 0 | 4 | 0 | 2023-04-12 18:00:00+01 | 1000 1 | 4 | 9770 | 2023-04-13 18:00:02+01 | 1000 2 | 4 | 85169 | 2023-04-14 18:00:04+01 | 1000 3 | 4 | 161192 | 2023-04-15 18:00:08+01 | 1000 4 | 4 | 237074 | 2023-04-16 18:00:11+01 | 1000 5 | 4 | 314160 | 2023-04-17 18:00:15+01 | 1000 6 | 4 | 391107 | 2023-04-18 18:00:18+01 | 1000 7 | 4 | 467716 | 2023-04-19 18:00:21+01 | 1000 8 | 4 | 544978 | 2023-04-20 18:00:26+01 | 1000 9 | 5 | 621933 | 2023-04-21 18:00:28+01 | 1000 10 | 6 | 699410 | 2023-04-22 18:00:31+01 | 1000 11 | 6 | 777074 | 2023-04-23 18:00:34+01 | 1000 12 | 6 | 855530 | 2023-04-24 18:00:36+01 | 1000 13 | 6 | 933559 | 2023-04-25 18:00:39+01 | 1000 (14 rows) sui_indexer_alt=# SELECT epoch, cp_hi, tx_hi, TO_TIMESTAMP(end_timestamp_ms / 1000), safe_mode, storage_fund_balance, storage_fund_reinvestment, storage_charge, storage_rebate, stake_subsidy_amount, total_gas_fees, total_stake_rewards_distributed, leftover_storage_fund_inflow FROM kv_epoch_ends ORDER BY epoch ASC; epoch | cp_hi | tx_hi | to_timestamp | safe_mode | storage_fund_balance | storage_fund_reinvestment | storage_charge | storage_rebate | stake_subsidy_amount | total_gas_fees | total_stake_rewards_distributed | leftover_storage_fund_inflow -------+--------+--------+------------------------+-----------+----------------------+---------------------------+----------------+----------------+----------------------+----------------+---------------------------------+------------------------------ 0 | 9770 | 9771 | 2023-04-13 18:00:02+01 | f | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 1 | 85169 | 85174 | 2023-04-14 18:00:04+01 | f | 2973880 | 0 | 3952000 | 978120 | 0 | 102000000 | 102000000 | 0 2 | 161192 | 161199 | 2023-04-15 18:00:08+01 | f | 717398960 | 0 | 715403200 | 978120 | 0 | 1000000 | 1000000 | 0 3 | 237074 | 237084 | 2023-04-16 18:00:11+01 | f | 733657184 | 0 | 1430198400 | 1413940176 | 0 | 2000000 | 2000000 | 0 4 | 314160 | 314171 | 2023-04-17 18:00:15+01 | f | 733657184 | 0 | 0 | 0 | 0 | 0 | 0 | 0 5 | 391107 | 391119 | 2023-04-18 18:00:18+01 | f | 733657184 | 0 | 0 | 0 | 0 | 0 | 0 | 0 6 | 467716 | 467730 | 2023-04-19 18:00:21+01 | f | 735633184 | 0 | 1976000 | 0 | 0 | 1000000 | 1000000 | 0 7 | 544978 | 544994 | 2023-04-20 18:00:26+01 | f | 729859616 | 0 | 702475600 | 708249168 | 0 | 1000000 | 1000000 | 0 8 | 621933 | 621950 | 2023-04-21 18:00:28+01 | f | 729859616 | 0 | 0 | 0 | 0 | 0 | 0 | 0 9 | 699410 | 699428 | 2023-04-22 18:00:31+01 | f | 729859616 | 0 | 0 | 0 | 0 | 0 | 0 | 0 10 | 777074 | 777093 | 2023-04-23 18:00:34+01 | f | 729859616 | 0 | 0 | 0 | 0 | 0 | 0 | 0 11 | 855530 | 855550 | 2023-04-24 18:00:36+01 | f | 729859616 | 0 | 0 | 0 | 0 | 0 | 0 | 0 12 | 933559 | 933586 | 2023-04-25 18:00:39+01 | f | 735866656 | 0 | 13832000 | 7824960 | 0 | 6000000 | 6000000 | 0 (13 rows) ``` ## Stack - #20118 - #20132 - #20147 - #20148 - #20149 --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit e96fca283cf8f00fd09291fdbdcd401c2063b71b Author: Ashok Menon <[email protected]> Date: Sat Nov 2 02:05:09 2024 +0000 indexer-alt: kv_feature_flags pipeline ## Description Like `kv_protocol_configs`, but for feature flags. ## Test plan Run the indexer over the first 700K checkpoints -- this will include two protocol config upgrades, then check the database for feature flags. ``` sui$ cargo run -p sui-indexer-alt --release -- \ --database-url "postgres://postgres:postgrespw@localhost:5432/sui_indexer_alt" \ indexer --remote-store-url https://checkpoints.mainnet.sui.io \ --last-checkpoint 700000 ``` ``` sui_indexer_alt=# SELECT * FROM kv_feature_flags ORDER BY protocol_version DESC, flag_name; protocol_version | flag_name | flag_value ------------------+-------------------------------------------------------------+------------ 6 | accept_zklogin_in_multisig | f 6 | advance_epoch_start_time_in_safe_mode | t 6 | advance_to_highest_supported_protocol_version | f 6 | allow_receiving_object_id | f 6 | authority_capabilities_v2 | f [...] ``` commit 1aab0cf2b23b6d682446eae86937542f9f698e3c Author: Ashok Menon <[email protected]> Date: Sat Nov 2 01:40:35 2024 +0000 indexer-alt: kv_protocol_configs pipeline ## Description Add a pipeline for indexing protocol configs. ## Test plan Index the first ~1M checkpoints, which should pick up the first protocol version upgrade (from 4 to 5 for epoch 10) and then query `kv_protocol_configs` results after that: ``` sui$ cargo run -p sui-indexer-alt -- \ --database-url postgres://postgres:postgrespw@localhost:5432/sui_indexer_alt \ --remote-store-url https://checkpoints.mainnet.sui.io \ --last-checkpoint 500000 ``` ``` sui_indexer_alt=# SELECT * FROM kv_protocol_configs; ``` commit 97932677f340404a8843ac8fb46ae56ea0b7501c Author: Ashok Menon <[email protected]> Date: Sat Nov 2 01:18:58 2024 +0000 refactor(indexer-alt): configurable processor ## Description Change the pipeline/handler/processor interface to accept a value representing the handler/processor. This allows the processor to include configuration. This will be used by upcoming pipelines for processing protocol configs and feature flags. ## Test plan CI commit ae6275f6957ac936caffe04e82b7762ba192c4f8 Author: Anastasios Kichidis <[email protected]> Date: Wed Nov 13 12:38:22 2024 +0000 [Consensus] resubmit GC'ed transactions (#19572) ## Description Transactions of GC'ed blocks should be retried to remain compliant with the so far system assumptions. Refactored the consensus side to signal back the submitter about the status of the block that the submitted transaction has been included to. Practically two possible outcomes: 1. Sequenced (when the block has been committed and consequently the tx as well) 2. GarbageCollected (when the tx's block has not been committed and passed gc_round) also took the opportunity for a small refactoring around the `SubmitToConsensus` trait and the `ConsensusClient` as intentions started diverging. ## Test plan CI/PT --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit bb0288b9b8de0307690a434eaf62a930cf1280e8 Author: benr-ml <[email protected]> Date: Wed Nov 13 14:35:19 2024 +0200 Enable G1Uncompressed in testnet (#20226) ## Description Describe the changes or additions included in this PR. ## Test plan How did you test the new or updated feature? --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit 6ea6bc9b112bf34a3dcfb136430726924b03e0d0 Author: Mark Logan <[email protected]> Date: Tue Nov 12 23:08:08 2024 -0800 Fixes for overload protection (#20248) - Only check overload queues for mutable shared inputs - Increase max tx queue age to 500ms ## Description Describe the changes or additions included in this PR. ## Test plan How did you test the new or updated feature? --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit 297913c0adcb677eb369d520e9de7c80696fbde3 Author: Mark Logan <[email protected]> Date: Tue Nov 12 21:21:06 2024 -0800 Fix bug with object_by_id inserts. (#20236) The previous version of this code was based on a misunderstanding of the Moka Cache API. It assumed that: let entry = cache.entry(k).or_insert_with(|| v); entry.is_fresh() could be used to detect whether there was a race with cache.insert(k, v) In fact this is not the case! Both threads must use `or_insert_with` in order to handle the race correctly. The bug is instantly reproducible with the following code: use moka::sync::Cache; use std::sync::{Arc, Mutex}; fn monotonic_update(cache: Arc<Cache<u64, Arc<Mutex<u64>>>>, key: u64, value: u64) { let entry = cache .entry(key) .or_insert_with(|| Arc::new(Mutex::new(value))); if !entry.is_fresh() { let mut entry = entry.value().lock().unwrap(); // only update if the new value is greater than the current value if value > *entry { *entry = value; } } } fn blind_write(cache: Arc<Cache<u64, Arc<Mutex<u64>>>>, key: u64, value: u64) { cache.insert(key, Arc::new(Mutex::new(value))); } fn main() { for _ in 0..1000 { let cache = Arc::new(Cache::new(1000)); let cache1 = cache.clone(); let cache2 = cache.clone(); let handle1 = std::thread::spawn(move || { monotonic_update(cache1, 1, 1); }); let handle2 = std::thread::spawn(move || { // Correct way to update the value // monotonic_update(cache2, 1, 2); // Incorrect way to update the value blind_write(cache2, 1, 2); }); handle1.join().unwrap(); handle2.join().unwrap(); let entry = cache.get(&1).unwrap(); let value = entry.lock().unwrap(); assert_eq!(*value, 2); } } ## Description Describe the changes or additions included in this PR. ## Test plan How did you test the new or updated feature? --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit ec6d01642ad68b1dea024a3f34d5966359a3617e Author: Todd Nowacki <[email protected]> Date: Tue Nov 12 15:51:12 2024 -0800 [move-compiler] Improve interactions with Sui mode and interface files (#20196) ## Description - Sui mode used only named addresses for type comparison. This is not sufficient for interface files that might be using only the numerical address. - Switched `is` to use only the numerical address. We might want to evaluate differently if we start using it more outside of Sui mode ## Test plan - Ran tests --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit 7b61e1ff70f4b2fbfd67085db2f795bcaa291105 Author: Avrora <[email protected]> Date: Wed Nov 13 01:42:06 2024 +0200 Docs: Fix typo (#20225) Fix typo ## Description Describe the changes or additions included in this PR. ## Test plan How did you test the new or updated feature? --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: --------- Co-authored-by: ronny-mysten <[email protected]> commit f716589e9b7d966cea77dbe8145ed892fe4fd0cd Author: ronny-mysten <[email protected]> Date: Tue Nov 12 16:41:39 2024 -0700 [docs] Removed orphaned page (#20228) ## Description Removed orphaned page and updates the link that pointed to it. ## Test plan How did you test the new or updated feature? --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit d5a23d78a9b30ec1c5ae5ac3cafdbe2817d68df6 Author: hayes-mysten <[email protected]> Date: Tue Nov 12 12:49:03 2024 -0800 [ts-sdk] Add tx.object.option for creatnig object options in transact… (#20194) …ion builder ## Description Describe the changes or additions included in this PR. ## Test plan How did you test the new or updated feature? --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit de2c7de6f6b5d643560c0ca343b9009c96e7fefd Author: benr-ml <[email protected]> Date: Tue Nov 12 21:44:54 2024 +0200 Update fastcrypto, including moving some structs between dkg->dkg_v1 (#20224) ## Description Describe the changes or additions included in this PR. ## Test plan How did you test the new or updated feature? --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit 44fe9eb64fed3a10ad5da8d3ad0d542b2f06b281 Author: Andrew Schran <[email protected]> Date: Tue Nov 12 14:44:34 2024 -0500 Revert "Add support for TLS connections with self-signed cert on validator gRPC interface" (#20232) Reverts MystenLabs/sui#19796 due to suspicion it's causing performance issues commit 73dbd005bb0f828a64eb4e811d4f8e2b3c80bdfe Author: sui-merge-bot[bot] <114704316+sui-merge-bot[bot]@users.noreply.github.com> Date: Tue Nov 12 13:38:32 2024 -0500 Version Packages (#20229) This PR was opened by the [Changesets release](https://github.com/changesets/action) GitHub action. When you're ready to do a release, you can merge this and publish to npm yourself or [setup this action to publish automatically](https://github.com/changesets/action#with-publishing). If you're not ready to do a release yet, that's fine, whenever you add more changesets to main, this PR will be updated. # Releases ## @mysten/[email protected] ### Patch Changes - a24d43d: getOrders and decodeOrderId support Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> commit c4afb26067e9a4e9eea75a706032f1764c4c3ded Author: Xun Li <[email protected]> Date: Tue Nov 12 10:17:08 2024 -0800 [indexer-alt] Add checkpoint ingestion lag metrics (#20191) ## Description This PR adds two metrics, one that tracks the ingestion lag on the latest fetched checkpoint, one that tracks histogram. These two combined could give us the full picture on the lag. ## Test plan CI. --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit 4ba4b29a0ca9c473e89a1ba5b01caf71c31c5c21 Author: phoenix <[email protected]> Date: Tue Nov 12 12:47:34 2024 -0500 [kv store] add timeout to bigtable ingestion + backfill mode (#20193) ## Description adds timeouts to the Bigtable ingestion pipeline and introduces a backfill mode that skips most progress updates to DynamoDB during backfilling --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit a24d43d709e1c05f98b092d50c2ac87b5c172489 Author: Tony Lee <[email protected]> Date: Tue Nov 12 12:38:38 2024 -0500 Deepbook SDK Examples (#20227) ## Description Getting balance for balance manager, getting live orders for a balance manager, and price/quantity map for a balance manager. Balance: ``` Manager: MANAGER_1 { coinType: '0x0000000000000000000000000000000000000000000000000000000000000002::sui::SUI', balance: 374527.5686 } { coinType: '0xdba34672e30cb065b1f93e3ab55318768fd6fef66c15942c9f7cb846e2f900e7::usdc::USDC', balance: 1492902.476423 } ``` Live Orders: ``` Manager: MANAGER_1 SUI_USDC [ '170141183460526324404595434778137174870', '170141183460526490425292098164101718871', '170141183460526490425292098164101718872', '56723756473400944927347700', '56502395544516430307955699', '55819866013789176898163698', '56723756473400944927347696' ] DEEP_SUI [ '413576002151014891304428578578', '421323634661972902983148578577', '170141183888249226801011805690925112362', '414498339354700368885228578576' ] ``` Price/Quantity Map: ``` Manager: MANAGER_1 SUI_USDC bid orders: [ [ 3.046, 6.6 ], [ 3.035, 21418.1 ], [ 3.027, 26427.4 ], [ 2.995, 46747.6 ] ] SUI_USDC ask orders: [ [ 3.049, 6.6 ], [ 3.065, 21207.2 ], [ 3.073, 26034.1 ], [ 3.096, 45223.3 ] ] ``` ## Test plan How did you test the new or updated feature? Locally tested using mainnet ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading …
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Reverts #19796 due to suspicion it's causing performance issues