Revert "Add support for TLS connections with self-signed cert on validator gRPC interface" (#20232) #20234
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
|
aschran
temporarily deployed
to
sui-typescript-aws-kms-test-env
GitHub Actions
Inactive
arun-koshy
approved these changes
Nov 12, 2024
ebmifa
approved these changes
Nov 12, 2024
carlosbmamoru
pushed a commit
to Mamoru-Foundation/sui
that referenced
this pull request
Nov 14, 2024
commit dc117c586a01d6dbef41eafcaf3d0be2e37f2d12 Merge: 32514d1c36 b8eb8920ae Author: Carlos Baez <[email protected]> Date: Thu Nov 14 11:36:36 2024 +0100 Merge tag 'testnet-v1.37.3' into testnet4-wit-v1.37.3 Sui testnet v1.37.3 release commit b8eb8920aeca592d0cf15fbddc7f64ed4aad7202 Author: Andrew Schran <[email protected]> Date: Tue Nov 12 15:34:20 2024 -0500 Revert "Add support for TLS connections with self-signed cert on validator gRPC interface" (#20232) (#20234) Reverts MystenLabs/sui#19796 due to suspicion it's causing performance issues --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit e205b302bd5c5742d21fbc18811b705980714c81 Author: Eugene Boguslavsky <[email protected]> Date: Tue Nov 12 16:56:05 2024 +0000 Sui v1.37.3 Version Bump commit f77489d1cf177b6bacda47c3c3ec6f120d1fed45 Author: Emma Zhong <[email protected]> Date: Mon Nov 11 19:22:47 2024 -0800 [cherrypick][jsonrpc] fix estimated rewards during safe mode (#20182) (#20187) ## Description Currently if the exchange rate for an epoch is not found, we would assign it the default exchange rate while we should've used the exchange rate from the previous existing epoch instead. This PR fixes that. ## Test plan Added some unit tests. --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [x] JSON-RPC: fixed reward estimation in `getStakes` API so that we don't overestimate stake rewards of stakes activated during safe mode epochs. - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: ## Description Describe the changes or additions included in this PR. ## Test plan How did you test the new or updated feature? --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit 102280ae7591ec1cf187a433624443b8798e5605 Author: Eugene Boguslavsky <[email protected]> Date: Thu Nov 7 19:25:00 2024 +0000 Sui v1.37.2 Version Bump commit b15a076619641002381f4be001f28be5dce7a3e9 Author: wlmyng <[email protected]> Date: Tue Nov 5 14:18:50 2024 -0800 [graphql][cherry-pick] need to add object_status=0 to use partial indexes (#20176) (#20181) ## Description cherry pick ## Test plan How did you test the new or updated feature? --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit 7839b9501066108cb2322ba9039120a41781a1b0 Author: John Martin <[email protected]> Date: Mon Nov 4 14:39:56 2024 -0800 add client-request-method to CORS allowed headers (#20160) (#20168) ## Description adds `client-request-method` to allowed cors methods ## Test plan --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: ## Description Describe the changes or additions included in this PR. ## Test plan How did you test the new or updated feature? --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit ecffa454e86b994b79dfc6fb9032d02ee95a0697 Author: John Martin <[email protected]> Date: Mon Nov 4 13:13:22 2024 -0800 fix: remove high cardinality address label from safe_client_latency (… (#20169) …#20165) ## Description safe_client_latency is the highest cardinality metric across our observability system due to the `address` label + being a histogram: ``` root@sui-node-mysten-rpc-0:/sui# curl -s localhost:9184/metrics | grep safe_client_latency_bucket | wc -l 8512 ``` --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: ## Description Describe the changes or additions included in this PR. ## Test plan How did you test the new or updated feature? --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit 1493483238672e1d5c22dbff15eb670b1d57883a Author: Eugene Boguslavsky <[email protected]> Date: Mon Nov 4 20:00:02 2024 +0000 Sui v1.37.1 Version Bump commit 6646b75dd68808b7bedb679c41706b12ae79a5a9 Author: Eugene Boguslavsky <[email protected]> Date: Thu Oct 31 18:46:36 2024 -0700 Sui `v1.37.0` Framework Bytecode Snapshot (#20131) ## Description Sui `v1.37.0` Framework Bytecode Snapshot ## Test plan `cargo run --bin sui-framework-snapshot` commit fa5d48fcb3e06a11677dc14e90f12cac59e8f2fc Author: Ashok Menon <[email protected]> Date: Fri Nov 1 01:03:44 2024 +0000 chore(indexer-alt): alphabetize Cargo.toml commit e116dfa6547e96608bee6767a483e080bb0a1e99 Author: Ashok Menon <[email protected]> Date: Fri Nov 1 00:39:20 2024 +0000 indexer-alt: watermark timestamps and other logging improvements (#20120) ## Description Track the timestamp associated with a watermark, include it in tracing messages, and introduce gauges to track the timestamps of the latest checkpoint gathered and written to the DB. This change also introduces a couple of other timestamp/watermark related tracing improvements: - Measure watermarks gathered and in DB for the sequential pipeline (previously just tracked the watermark after they were written to the DB). These will usually be the exact same, but may differ if the pipeline needs to retry a write. - Gets rid of the ordering and equality impls for `CommitterWatermark` -- these are no longer required. - Standardise the ordering of watermark traces between the sequential and concurrent pipelines, so that they are easier to compare with each other (they now share a common prefix). - Standardised tracing messages during committer/watermark teardown: Now in each exit edge, we log the reason for the exit, and then a message with the final watermark. ## Test plan Run `sum_obj_types` and `wal_obj_types` with a consistent range of `3600` and note that the difference in their timestamps is roughly an hour (to begin with, it's about 70 minutes): ``` sui$ cargo run -p sui-indexer-alt --release -- \ --database-url 'postgres://postgres:postgrespw@localhost:5432/sui_indexer_alt' \ indexer --remote-store-url 'https://checkpoints.mainnet.sui.io' \ --consistent-range 3600 --pipeline wal_obj_types --pipeline sum_obj_types` Running `/Users/ashokmenon/sui/idx-poc/target/release/sui-indexer-alt --database-url 'postgres://postgres:postgrespw@localhost:5432/sui_indexer_alt' indexer --remote-store-url 'https://checkpoints.mainnet.sui.io' --consistent-range 3600 --pipeline wal_obj_types --pipeline sum_obj_types` 2024-10-31T15:16:26.867221Z INFO sui_indexer_alt::db: Running migrations ... 2024-10-31T15:16:26.889145Z INFO sui_indexer_alt::db: Migrations complete. 2024-10-31T15:16:27.024406Z INFO sui_indexer_alt: Skipping pipeline ev_emit_mod 2024-10-31T15:16:27.024413Z INFO sui_indexer_alt: Skipping pipeline ev_struct_inst 2024-10-31T15:16:27.024414Z INFO sui_indexer_alt: Skipping pipeline kv_checkpoints 2024-10-31T15:16:27.024414Z INFO sui_indexer_alt: Skipping pipeline kv_objects 2024-10-31T15:16:27.024415Z INFO sui_indexer_alt: Skipping pipeline kv_transactions 2024-10-31T15:16:27.024416Z INFO sui_indexer_alt: Skipping pipeline obj_versions 2024-10-31T15:16:27.024416Z INFO sui_indexer_alt: Skipping pipeline tx_affected_objects 2024-10-31T15:16:27.024417Z INFO sui_indexer_alt: Skipping pipeline tx_balance_changes 2024-10-31T15:16:27.024417Z INFO sui_indexer_alt: Skipping pipeline wal_coin_balances 2024-10-31T15:16:27.033550Z INFO sui_indexer_alt: Skipping pipeline sum_coin_balances 2024-10-31T15:16:27.033607Z INFO sui_indexer_alt::pipeline::processor: Starting processor pipeline="wal_obj_types" 2024-10-31T15:16:27.033856Z INFO sui_indexer_alt::pipeline::concurrent::collector: Starting collector pipeline="wal_obj_types" 2024-10-31T15:16:27.033869Z INFO sui_indexer_alt::pipeline::concurrent::committer: Starting committer pipeline="wal_obj_types" 2024-10-31T15:16:27.033983Z INFO sui_indexer_alt: Skipping pipeline sum_packages 2024-10-31T15:16:27.034011Z INFO sui_indexer_alt::pipeline::processor: Starting processor pipeline="sum_obj_types" 2024-10-31T15:16:27.034183Z INFO sui_indexer_alt::pipeline::concurrent::watermark: Starting watermark pipeline="wal_obj_types" watermark=CommitterWatermark { pipeline: "wal_obj_types", epoch_hi_inclusive: 0, checkpoint_hi_inclusive: 0, tx_hi: 0, timestamp_ms_hi_inclusive: 0 } 2024-10-31T15:16:27.034260Z INFO sui_indexer_alt::pipeline::sequential::committer: Starting committer pipeline="sum_obj_types" watermark=CommitterWatermark { pipeline: "sum_obj_types", epoch_hi_inclusive: 0, checkpoint_hi_inclusive: 0, tx_hi: 0, timestamp_ms_hi_inclusive: 0 } 2024-10-31T15:16:27.034305Z INFO sui_indexer_alt: Ingestion range first_checkpoint=0 last_checkpoint=None 2024-10-31T15:16:27.034339Z INFO sui_indexer_alt::metrics: Starting metrics service on 0.0.0.0:9184 2024-10-31T15:16:27.034351Z INFO sui_indexer_alt::ingestion::regulator: Starting ingestion regulator 2024-10-31T15:16:27.034355Z INFO sui_indexer_alt::ingestion::broadcaster: Starting ingestion broadcaster 2024-10-31T15:16:27.541400Z INFO sui_indexer_alt::pipeline::concurrent::watermark: Watermark pipeline="wal_obj_types" epoch=0 checkpoint=212 transaction=213 timestamp=2023-04-13 13:28:25.836 UTC updated=true elapsed_ms=5.486208 2024-10-31T15:16:28.043533Z INFO sui_indexer_alt::pipeline::concurrent::watermark: Watermark pipeline="wal_obj_types" epoch=0 checkpoint=515 transaction=516 timestamp=2023-04-13 13:36:17.240 UTC updated=true elapsed_ms=5.512666 2024-10-31T15:16:28.538421Z INFO sui_indexer_alt::pipeline::concurrent::watermark: Watermark pipeline="wal_obj_types" epoch=0 checkpoint=1017 transaction=1018 timestamp=2023-04-13 13:48:04.584 UTC updated=true elapsed_ms=3.468209 2024-10-31T15:16:29.041788Z INFO sui_indexer_alt::pipeline::concurrent::watermark: Watermark pipeline="wal_obj_types" epoch=0 checkpoint=1422 transaction=1423 timestamp=2023-04-13 13:57:11.577 UTC updated=true elapsed_ms=7.184292 2024-10-31T15:16:29.540675Z INFO sui_indexer_alt::pipeline::concurrent::watermark: Watermark pipeline="wal_obj_types" epoch=0 checkpoint=1824 transaction=1825 timestamp=2023-04-13 14:05:59.154 UTC updated=true elapsed_ms=5.7144580000000005 2024-10-31T15:16:30.040987Z INFO sui_indexer_alt::pipeline::concurrent::watermark: Watermark pipeline="wal_obj_types" epoch=0 checkpoint=2226 transaction=2227 timestamp=2023-04-13 14:14:54.721 UTC updated=true elapsed_ms=5.955416 2024-10-31T15:16:30.539394Z INFO sui_indexer_alt::pipeline::concurrent::watermark: Watermark pipeline="wal_obj_types" epoch=0 checkpoint=2630 transaction=2631 timestamp=2023-04-13 14:23:21.636 UTC updated=true elapsed_ms=5.0003340000000005 2024-10-31T15:16:31.035944Z INFO sui_indexer_alt::pipeline::concurrent::watermark: Watermark pipeline="wal_obj_types" epoch=0 checkpoint=3142 transaction=3143 timestamp=2023-04-13 14:34:00.192 UTC updated=true elapsed_ms=1.368042 2024-10-31T15:16:31.532346Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_obj_types" epoch=0 checkpoint=71 transaction=72 timestamp=2023-04-13 13:24:36.662 UTC 2024-10-31T15:16:31.535237Z INFO sui_indexer_alt::pipeline::concurrent::watermark: Watermark pipeline="wal_obj_types" epoch=0 checkpoint=3646 transaction=3647 timestamp=2023-04-13 14:45:19.227 UTC updated=true elapsed_ms=0.7080409999999999 2024-10-31T15:16:31.556459Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_obj_types" epoch=0 checkpoint=103 transaction=104 timestamp=2023-04-13 13:25:35.491 UTC 2024-10-31T15:16:31.605247Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_obj_types" epoch=0 checkpoint=151 transaction=152 timestamp=2023-04-13 13:27:01.087 UTC 2024-10-31T15:16:31.658676Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_obj_types" epoch=0 checkpoint=203 transaction=204 timestamp=2023-04-13 13:28:14.431 UTC 2024-10-31T15:16:31.709093Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_obj_types" epoch=0 checkpoint=251 transaction=252 timestamp=2023-04-13 13:29:41.350 UTC 2024-10-31T15:16:31.774101Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_obj_types" epoch=0 checkpoint=304 transaction=305 timestamp=2023-04-13 13:31:05.962 UTC 2024-10-31T15:16:31.823847Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_obj_types" epoch=0 checkpoint=351 transaction=352 timestamp=2023-04-13 13:32:06.748 UTC 2024-10-31T15:16:31.875613Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_obj_types" epoch=0 checkpoint=404 transaction=405 timestamp=2023-04-13 13:33:26.122 UTC 2024-10-31T15:16:31.923673Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_obj_types" epoch=0 checkpoint=452 transaction=453 timestamp=2023-04-13 13:34:41.672 UTC 2024-10-31T15:16:31.973523Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_obj_types" epoch=0 checkpoint=502 transaction=503 timestamp=2023-04-13 13:35:54.245 UTC 2024-10-31T15:16:32.020968Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_obj_types" epoch=0 checkpoint=553 transaction=554 timestamp=2023-04-13 13:37:12.723 UTC 2024-10-31T15:16:32.034336Z INFO sui_indexer_alt::pipeline::concurrent::watermark: Watermark pipeline="wal_obj_types" epoch=0 checkpoint=4147 transaction=4148 timestamp=2023-04-13 14:56:30.470 UTC updated=true elapsed_ms=0.478916 2024-10-31T15:16:32.081115Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_obj_types" epoch=0 checkpoint=603 transaction=604 timestamp=2023-04-13 13:38:26.226 UTC 2024-10-31T15:16:32.143228Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_obj_types" epoch=0 checkpoint=657 transaction=658 timestamp=2023-04-13 13:39:32.583 UTC 2024-10-31T15:16:32.201323Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_obj_types" epoch=0 checkpoint=710 transaction=711 timestamp=2023-04-13 13:40:44.461 UTC 2024-10-31T15:16:32.239715Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_obj_types" epoch=0 checkpoint=751 transaction=752 timestamp=2023-04-13 13:41:40.955 UTC 2024-10-31T15:16:32.306828Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_obj_types" epoch=0 checkpoint=803 transaction=804 timestamp=2023-04-13 13:43:00.433 UTC 2024-10-31T15:16:32.351995Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_obj_types" epoch=0 checkpoint=854 transaction=855 timestamp=2023-04-13 13:44:21.487 UTC 2024-10-31T15:16:32.408044Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_obj_types" epoch=0 checkpoint=903 transaction=904 timestamp=2023-04-13 13:45:33.468 UTC 2024-10-31T15:16:32.478623Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_obj_types" epoch=0 checkpoint=963 transaction=964 timestamp=2023-04-13 13:46:48.531 UTC 2024-10-31T15:16:32.517822Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_obj_types" epoch=0 checkpoint=1004 transaction=1005 timestamp=2023-04-13 13:47:43.997 UTC 2024-10-31T15:16:32.540701Z INFO sui_indexer_alt::pipeline::concurrent::watermark: Watermark pipeline="wal_obj_types" epoch=0 checkpoint=4550 transaction=4551 timestamp=2023-04-13 15:05:30.690 UTC updated=true elapsed_ms=5.776375 2024-10-31T15:16:32.574600Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_obj_types" epoch=0 checkpoint=1052 transaction=1053 timestamp=2023-04-13 13:48:55.253 UTC 2024-10-31T15:16:32.625726Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_obj_types" epoch=0 checkpoint=1102 transaction=1103 timestamp=2023-04-13 13:49:57.288 UTC 2024-10-31T15:16:32.679135Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_obj_types" epoch=0 checkpoint=1159 transaction=1160 timestamp=2023-04-13 13:51:06.819 UTC 2024-10-31T15:16:32.729858Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_obj_types" epoch=0 checkpoint=1203 transaction=1204 timestamp=2023-04-13 13:52:11.201 UTC 2024-10-31T15:16:32.777044Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_obj_types" epoch=0 checkpoint=1257 transaction=1258 timestamp=2023-04-13 13:53:26.750 UTC 2024-10-31T15:16:32.819603Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_obj_types" epoch=0 checkpoint=1301 transaction=1302 timestamp=2023-04-13 13:54:17.123 UTC 2024-10-31T15:16:32.870028Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_obj_types" epoch=0 checkpoint=1351 transaction=1352 timestamp=2023-04-13 13:55:24.124 UTC 2024-10-31T15:16:32.923059Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_obj_types" epoch=0 checkpoint=1402 transaction=1403 timestamp=2023-04-13 13:56:49.078 UTC 2024-10-31T15:16:32.969159Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_obj_types" epoch=0 checkpoint=1451 transaction=1452 timestamp=2023-04-13 13:57:56.351 UTC 2024-10-31T15:16:33.034831Z INFO sui_indexer_alt::pipeline::concurrent::watermark: Watermark pipeline="wal_obj_types" epoch=0 checkpoint=5055 transaction=5056 timestamp=2023-04-13 15:16:38.038 UTC updated=true elapsed_ms=1.0442500000000001 2024-10-31T15:16:33.036266Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_obj_types" epoch=0 checkpoint=1501 transaction=1502 timestamp=2023-04-13 13:58:56.227 UTC ^C2024-10-31T15:16:33.091189Z INFO sui_indexer_alt::pipeline::concurrent::collector: Shutdown received, stopping collector pipeline="wal_obj_types" 2024-10-31T15:16:33.091195Z INFO sui_indexer_alt::pipeline::concurrent::watermark: Shutdown received pipeline="wal_obj_types" 2024-10-31T15:16:33.091201Z INFO sui_indexer_alt::ingestion::regulator: Shutdown received, stopping regulator 2024-10-31T15:16:33.091213Z INFO sui_indexer_alt::metrics: Shutdown received, stopping metrics service 2024-10-31T15:16:33.091225Z INFO sui_indexer_alt::pipeline::concurrent::watermark: Stopping watermark task pipeline="wal_obj_types" watermark=CommitterWatermark { pipeline: "wal_obj_types", epoch_hi_inclusive: 0, checkpoint_hi_inclusive: 5055, tx_hi: 5056, timestamp_ms_hi_inclusive: 1681398998038 } 2024-10-31T15:16:33.091410Z INFO sui_indexer_alt::pipeline::concurrent::committer: Batches done, stopping committer pipeline="wal_obj_types" 2024-10-31T15:16:33.091721Z INFO sui_indexer_alt::pipeline::sequential::committer: Shutdown received pipeline="sum_obj_types" 2024-10-31T15:16:33.091725Z INFO sui_indexer_alt::pipeline::sequential::committer: Stopping committer pipeline="sum_obj_types" watermark=CommitterWatermark { pipeline: "sum_obj_types", epoch_hi_inclusive: 0, checkpoint_hi_inclusive: 1546, tx_hi: 1547, timestamp_ms_hi_inclusive: 1681394391225 } 2024-10-31T15:16:33.093829Z INFO sui_indexer_alt::pipeline::processor: Shutdown received, stopping processor pipeline="wal_obj_types" 2024-10-31T15:16:33.094014Z INFO sui_indexer_alt::pipeline::processor: Shutdown received, stopping processor pipeline="sum_obj_types" 2024-10-31T15:16:33.094406Z INFO sui_indexer_alt::ingestion::broadcaster: Shutdown received, stopping ingestion broadcaster 2024-10-31T15:16:33.094513Z INFO sui_indexer_alt: Indexing pipeline gracefully shut down ``` ## Stack - #20089 - #20114 - #20116 - #20117 - #20119 --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit 4b727cbbfd89db7ebfd09b21e05faa58ac2d8c96 Author: Ashok Menon <[email protected]> Date: Fri Nov 1 00:37:19 2024 +0000 indexer-alt: allow sequential pipeline immediate reset (#20119) ## Description If the sequential pipeline committer can guarantee that it could process more checkpoints by looking at its pending buffer, we now reset the polling interval immediately, so it does not wait to issue the next write. This mimics a similar behaviour in the concurrent pipeline. I made this change after noticing how the pipeline behaves when ingestion is stuck retrying a checkpoint, on my local machine. Usually when running locally, performance is limited by checkpoint download rate, but in a sequential pipeline, if a checkpoint failed to download, it is possible for many checkpoints to end up processed and pending. With the previous implementation, once ingestion had recovered (the checkpoint is fetched), the pending buffer kept growing because it was only able to land `MAX_BATCH_CHECKPOINTS / commit_interval`, so if checkpoints were getting added faster than that, it would never recover. With this change, the pipeline recovers almost instantly, and I expect that in GCP where bandwidth is not the rate limiting factor, this should improve throughput during backfill, and synthetic benchmarks. ## Test plan Run the indexer with a large ingestion buffer and concurrency, wait for ingestion to fail to fetch a checkpoint, and then notice how the situation recovers (instead of getting worse until the pipeline eventually complains that it has too many pending checkpoints): ``` sui$ cargo run -p sui-indexer-alt --release -- \ --database-url "postgres://postgres:postgrespw@localhost:5432/sui_indexer_alt" \ indexer --remote-store-url https://checkpoints.mainnet.sui.io \ --last-checkpoint 1200000 --pipeline sum_packages \ --checkpoint-buffer-size 50000 --ingest-concurrency 20000 ``` ## Stack - #20089 - #20114 - #20116 - #20117 --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit b65f34c6a00801620e11469575523895a8e074ac Author: Ashok Menon <[email protected]> Date: Fri Nov 1 00:21:38 2024 +0000 indexer-alt: wal_coin_balances pipeline (#20117) ## Description `wal_coin_balances` is to `sum_coin_balances` what `wal_obj_types` is to `sum_obj_types`. ## Test plan Run the indexer, and correlate the live object set calculated from the write-ahead log against the one that's already in the summary table: ``` sui$ cargo run -p sui-indexer-alt --release -- \ --database-url "postgres://postgres:postgrespw@localhost:5432/sui_indexer_alt" \ indexer --remote-store-url https://checkpoints.mainnet.sui.io/ \ --last-checkpoint 5000 ``` ``` sui_indexer_alt=# SELECT COUNT(*) FROM sum_coin_balances; count ------- 178 (1 row) sui_indexer_alt=# SELECT COUNT(*) FROM ( SELECT DISTINCT ON (object_id) * FROM wal_coin_balances ORDER BY object_id, object_version DESC ) o WHERE o.owner_id IS NOT NULL; count ------- 178 (1 row) ``` ## Stack - #20089 - #20114 - #20116 --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit 71f1faa1a8557a178e0bf89f20829a8ea503f156 Author: Ashok Menon <[email protected]> Date: Fri Nov 1 00:18:46 2024 +0000 indexer-alt: wal_obj_types pipeline (#20116) ## Description Adds the concurrent pipeline that writes the write-ahead log for `sum_obj_types`. It re-uses the `process` implementation from `sum_obj_types` and then writes it into an append-only table. Note that today: - The pipelines are completely independent, which means the WAL pipeline redoes the processing work of the summary pipeline (this is presumably not an issue because the repeated work is not particularly heavy). - This change does not include the pruning necessary to keep this table's size in check (in practice it should only be a couple of gigs in size). This will come in a follow-up PR. ## Test plan Run the indexer and cross check the live-object set calculated from the write-ahead log with the summary: ``` sui$ cargo run -p sui-indexer-alt --release -- \ --database-url "postgres://postgres:postgrespw@localhost:5432/sui_indexer_alt" \ indexer --remote-store-url https://checkpoints.mainnet.sui.io \ --last-checkpoint 5000 ``` ``` sui_indexer_alt=# SELECT COUNT(*) FROM sum_obj_types; count ------- 592 (1 row) ^ sui_indexer_alt=# SELECT COUNT(*) FROM ( SELECT DISTINCT ON (object_id) * FROM wal_obj_types ORDER BY object_id, object_version DESC ) o WHERE o.owner_kind IS NOT NULL; count ------- 592 (1 row) ``` ## Stack - #20089 - #20114 --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit 7de02417c27844c8a086866b1279b4736ab862ec Author: Ashok Menon <[email protected]> Date: Thu Oct 31 23:22:42 2024 +0000 indexer-alt: obj_versions pipeline (#20114) ## Description Pipeline to fill `obj_versions` table -- this is the table that serves "parent version" (dynamic field) queries. ## Test plan ``` sui$ cargo run -p sui-indexer-alt --release -- \ --database-url "postgres://postgres:postgrespw@localhost:5432/sui_indexer_alt" \ indexer --remote-url "https://checkpoints.mainnet.sui.io" \ --last-checkpoint 5000 ``` ## Stack - #20089 --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit 53eaed5e300a67f0d01cf68119a145410925f9de Author: Ashok Menon <[email protected]> Date: Wed Oct 30 00:06:07 2024 +0000 indexer-alt: delays for sequential pipelines ## Description Add the ability to include an optional delay to sequential pipelines. This can be used to implement behaviour similar to `objects_snapshot` in the new architecture. Sequential pipelines honour this configuration in their committer tasks: - They process checkpoints at the same rate as other pipelines - But hold back data in the committer's "pending" map until the committer witnesses a checkpoint that is at least `checkpoint_lag` checkpoints ahead. This also requires slightly complicating the set-up and tear-down logic: - When setting up, we may need to wait until the pending buffer has been primed enough to set a valid commit upperbound. - If the pipeline runs dry before that point, we need to add an extra check to close down the committer. - When tearing down the committer after its channel has closed, we need to check whether batching would have picked up the first pending checkpoint (whether it is a valid "next checkpoint" and it's not held up because of a lag parameter). ## Test plan Run the indexer with various parameterisations for the consistent range, and note that the watermark for the summary pipelines lag behind the watermarks for other pipelines. ``` sui$ cargo run -p sui-indexer-alt --release -- \ --database-url "postgres://postgres:postgrespw@localhost:5432/sui_indexer_alt" \ indexer --remote-store-url https://checkpoints.mainnet.sui.io \ --pipeline kv_objects \ --pipeline sum_obj_types --pipeline sum_coin_balances \ --last-checkpoint 5000 --consistent-range 1000 [...] 2024-10-30T00:24:45.148259Z INFO sui_indexer_alt::ingestion::regulator: Checkpoints done, stopping regulator 2024-10-30T00:24:45.161682Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_obj_types" epoch=0 checkpoint=3605 transaction=3606 2024-10-30T00:24:45.212394Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_coin_balances" epoch=0 checkpoint=2703 transaction=2704 2024-10-30T00:24:45.222986Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_obj_types" epoch=0 checkpoint=3656 transaction=3657 2024-10-30T00:24:45.274318Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_obj_types" epoch=0 checkpoint=3702 transaction=3703 2024-10-30T00:24:45.340714Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_obj_types" epoch=0 checkpoint=3753 transaction=3754 2024-10-30T00:24:45.355934Z INFO sui_indexer_alt::pipeline::concurrent::watermark: Watermark pipeline="kv_objects" elapsed_ms=0.949083 updated=true epoch=0 checkpoint=4760 transaction=4761 2024-10-30T00:24:45.401262Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_obj_types" epoch=0 checkpoint=3801 transaction=3802 2024-10-30T00:24:45.463134Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_obj_types" epoch=0 checkpoint=3852 transaction=3853 2024-10-30T00:24:45.544933Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_obj_types" epoch=0 checkpoint=3904 transaction=3905 2024-10-30T00:24:45.609068Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_obj_types" epoch=0 checkpoint=3952 transaction=3953 2024-10-30T00:24:45.668560Z INFO sui_indexer_alt::ingestion::broadcaster: Checkpoints done, stopping ingestion broadcaster 2024-10-30T00:24:45.668593Z INFO sui_indexer_alt::pipeline::processor: Checkpoints done, stopping processor pipeline="kv_objects" 2024-10-30T00:24:45.668605Z INFO sui_indexer_alt::pipeline::processor: Checkpoints done, stopping processor pipeline="sum_obj_types" 2024-10-30T00:24:45.668605Z INFO sui_indexer_alt::pipeline::processor: Checkpoints done, stopping processor pipeline="sum_coin_balances" 2024-10-30T00:24:45.674305Z INFO sui_indexer_alt::pipeline::sequential::committer: Processor closed channel, pending rows empty, stopping committer pipeline="sum_obj_types" watermark=CommitterWatermark { pipeline: "sum_obj_types", epoch_hi_inclusive: 0, checkpoint_hi_inclusive: 4000, tx_hi: 4001 } 2024-10-30T00:24:45.713856Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_coin_balances" epoch=0 checkpoint=3003 transaction=3004 2024-10-30T00:24:45.863837Z INFO sui_indexer_alt::pipeline::concurrent::watermark: Watermark pipeline="kv_objects" elapsed_ms=1.83825 updated=true epoch=0 checkpoint=4960 transaction=4961 2024-10-30T00:24:46.120459Z INFO sui_indexer_alt::pipeline::concurrent::collector: Processor closed channel, pending rows empty, stopping collector pipeline="kv_objects" 2024-10-30T00:24:46.131407Z INFO sui_indexer_alt::pipeline::concurrent::committer: Batches done, stopping committer pipeline="kv_objects" 2024-10-30T00:24:46.214267Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_coin_balances" epoch=0 checkpoint=3303 transaction=3304 2024-10-30T00:24:46.356941Z INFO sui_indexer_alt::pipeline::concurrent::watermark: Watermark pipeline="kv_objects" elapsed_ms=0.68 updated=true epoch=0 checkpoint=5000 transaction=5001 2024-10-30T00:24:46.356970Z INFO sui_indexer_alt::pipeline::concurrent::watermark: Committer closed channel, stopping watermark task pipeline="kv_objects" watermark=CommitterWatermark { pipeline: "kv_objects", epoch_hi_inclusive: 0, checkpoint_hi_inclusive: 5000, tx_hi: 5001 } 2024-10-30T00:24:46.714498Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_coin_balances" epoch=0 checkpoint=3603 transaction=3604 2024-10-30T00:24:47.214758Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_coin_balances" epoch=0 checkpoint=3903 transaction=3904 2024-10-30T00:24:47.715841Z INFO sui_indexer_alt::pipeline::sequential::committer: Watermark pipeline="sum_coin_balances" epoch=0 checkpoint=4000 transaction=4001 2024-10-30T00:24:47.715917Z INFO sui_indexer_alt::pipeline::sequential::committer: Processor closed channel, pending rows empty, stopping committer pipeline="sum_coin_balances" watermark=CommitterWatermark { pipeline: "sum_coin_balances", epoch_hi_inclusive: 0, checkpoint_hi_inclusive: 4000, tx_hi: 4001 } 2024-10-30T00:24:47.716485Z INFO sui_indexer_alt: Indexing pipeline gracefully shut down 2024-10-30T00:24:47.716647Z INFO sui_indexer_alt::metrics: Shutdown received, stopping metrics service ``` commit d57326492b62ec8ba7ff8826ad473ec1b5008f11 Author: Ashok Menon <[email protected]> Date: Wed Oct 30 00:01:45 2024 +0000 fix(indexer-alt): remove double logging on pipeline skip ## Description We were logging the pipeline being skipped twice. ## Test plan Check logs while the indexer starts up: ``` sui$ cargo run -p sui-indexer-alt --release -- \ --database-url "postgres://postgres:postgrespw@localhost:5432/sui_indexer_alt" \ indexer --remote-store-url "https://checkpoints.mainnet.sui.io" \ --last-checkpoint 5000 --pipeline sum_obj_types ``` commit ef0d78c638e3ef0d0a3b28787b618ea56a140284 Author: Ge Gao <[email protected]> Date: Thu Oct 31 19:30:51 2024 -0400 add sui-field-count for sui-indexer-alt (#20126) ## Description title, so that we can replace hard-coded field count in pipelines. a crate did very similar things but it's archived, so prob better to have our own impl, also b/c the functionality is relatively simple https://github.com/discosultan/field-count ## Test plan added tests in models/ --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit c92dd1b4f16074ced8c1fe5c56a10e675f68bbe6 Author: phoenix <[email protected]> Date: Thu Oct 31 18:28:34 2024 -0400 [kv store] bigtable. Add support for querying checkpoint by digest (#20122) ## Description Adds support for querying checkpoint by digest --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit 74ace7b6166f10bddfd714568e0ddc20db2ca75d Author: JasonRUAN <[email protected]> Date: Fri Nov 1 06:25:44 2024 +0800 [CLI] fix `sui keytool import` --alias command and default alias does not work (#20111) ## Description fix `sui keytool import` --alias command and default alias does not work Before Fix： <img width="1295" alt="image" src="https://github.com/user-attachments/assets/4cb40d92-e266-4b6e-9c39-c05ac52d7a9d"> After Fix： - **--alias** <img width="1301" alt="image" src="https://github.com/user-attachments/assets/b5c711e8-2a4d-4861-9965-30cde260e93f"> - **default alias** <img width="1295" alt="image" src="https://github.com/user-attachments/assets/4513db65-0e10-4179-b5f0-de188871cf78"> ## Test plan How did you test the new or updated feature? --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [x] CLI: `sui keytool import --alias` is now fixed and correctly records the provided alias. - [ ] Rust SDK: - [ ] REST API: commit e41a759769fdea08f660160a23e945bbad80b337 Author: sui-merge-bot[bot] <114704316+sui-merge-bot[bot]@users.noreply.github.com> Date: Thu Oct 31 23:50:37 2024 +0200 Version Packages (#20127) This PR was opened by the [Changesets release](https://github.com/changesets/action) GitHub action. When you're ready to do a release, you can merge this and publish to npm yourself or [setup this action to publish automatically](https://github.com/changesets/action#with-publishing). If you're not ready to do a release yet, that's fine, whenever you add more changesets to main, this PR will be updated. # Releases ## @mysten/[email protected] ### Patch Changes - 02c9e46: Fix exports on the bundled package Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> commit 02c9e4691b429b4a0ec85b71eace37b72515d055 Author: Manolis Liolios <[email protected]> Date: Thu Oct 31 23:38:15 2024 +0200 Fix build, re-publish @mysten/kms (#20125) ## Description Build will always treat `src` as the base for outputs when building. ## Test plan How did you test the new or updated feature? --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit 16f2683a99e7b8824aa5879d3f20ffef936a7546 Author: William Smith <[email protected]> Date: Thu Oct 31 17:27:16 2024 -0400 [sui-benchmark] Add expected failure payloads (#19831) ## Description Introduce a new payload type into the benchmark tool - `ExpectedFailurePayload`. This payload type is configured with some expected failure type (currently only one implemented - user signature failure - but others can be added with minimal scaffolding) such that `payload.make_transaction()` generates a transaction that will fail in this manner. Note that the failures that this payload type is concerned with are failures to execute the transaction itself, rather than during execution by the MoveVM. In other words, it is expected that the failure mode will not consume gas or produce effects. Note that for this reason, its failure mode is inverted. It will be tallied for metrics purposes as an `error` if it succeeds and tallied as an `expected_error` (which is a success) if it fails. Also note that transaction responses for this type are handled by producing a `NextOp::Retry` (with some additional logging/metrics) since it is functionally equivalent to a retryable error. ## Test plan Ran the following: ``` SIM_STRESS_TEST_QPS=200 SIM_STRESS_TEST_WORKERS=20 RUST_LOG=debug cargo simtest --nocapture test_simulated_load_expected_failure_traffic_control ``` Observed the following in logs ``` 2022-01-03T02:05:47.734507Z INFO node{id=1 name="client"}: simtest::test: crates/sui-benchmark/tests/simtest.rs:1088: end of test BenchmarkStats { duration: 50.000002242s, num_error_txes: 0, num_expected_error_txes: 8049, num_success_txes: 348, num_success_cmds: 1977, total_gas_used: 3215776400, latency_ms: HistogramWrapper { histogram: Histogram { auto_resize: false, highest_trackable_value: 120000, lowest_discernible_value: 1, significant_value_digits: 3, bucket_count: 7, sub_bucket_count: 2048, sub_bucket_half_count: 1024, sub_bucket_half_count_magnitude: 10, sub_bucket_mask: 2047, leading_zero_count_base: 53, unit_magnitude: 0, unit_magnitude_mask: 0, max_value: 14359, min_non_zero_value: 72, total_count: 348, counts: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 2, 8, 12, 9, 20, 25, 22, 19 ... ``` [Also ran in PTN](https://metrics.sui.io/d/adl51ctsvmkg0a/traffic-control-dos-protection-dashboard?from=2024-10-31T19:00:50.548Z&to=2024-10-31T20:20:03.673Z&timezone=browser&var-network=private-testnet&var-host=$__all&var-fullnode=$__all) --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit d95918cea0d8fa0f1a4007273f477e3c12b61302 Author: phoenix <[email protected]> Date: Thu Oct 31 17:18:17 2024 -0400 [data ingestion] add support for bigtable in main workflow binary (#20018) ## Description adding BigTable KV as a variant to main binary that manages internal workflows --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit c33374e735257b2a460a2f882e062619f8958bc5 Author: wlmyng <[email protected]> Date: Thu Oct 31 12:51:47 2024 -0700 [mvr] indexer schema subset and reduced persist tasks (#20100) ## Description Describe the changes or additions included in this PR. ## Test plan How did you test the new or updated feature? --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit 7c517e7e5ce7193d1f51ef7f20a31b56f0e66797 Author: Xun Li <[email protected]> Date: Thu Oct 31 12:49:58 2024 -0700 Add system_invariant_violation macro (#20105) ## Description Introduce a system invariant violation macro. We can then set up alerts whenever the counter value changes. Used it in one place as an example. ## Test plan How did you test the new or updated feature? --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit b248f35879801b3bae9b76c4d204f36540d816f8 Author: Andrew Schran <[email protected]> Date: Thu Oct 31 14:16:48 2024 -0400 Add support for TLS connections with self-signed cert on validator gRPC interface (#19796) Client side use of TLS will be enabled-by-default in a future PR. --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [x] Nodes (Validators and Full nodes): Adds support for TLS on validator gRPC interface. - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit e06610823b985bdc01c40e23c9a9a2f8d5b571d1 Author: sui-merge-bot[bot] <114704316+sui-merge-bot[bot]@users.noreply.github.com> Date: Thu Oct 31 19:58:58 2024 +0200 Version Packages (#20076) This PR was opened by the [Changesets release](https://github.com/changesets/action) GitHub action. When you're ready to do a release, you can merge this and publish to npm yourself or [setup this action to publish automatically](https://github.com/changesets/action#with-publishing). If you're not ready to do a release yet, that's fine, whenever you add more changesets to main, this PR will be updated. # Releases ## @mysten/[email protected] ### Minor Changes - c24814b: Adds a custom header; 'Client-Request-Method' which will contain the method name used in each outgoing jsonrpc request ## @mysten/[email protected] ### Patch Changes - Updated dependencies [c24814b] - @mysten/[email protected] - @mysten/[email protected] ## @mysten/[email protected] ### Patch Changes - Updated dependencies [c24814b] - @mysten/[email protected] - @mysten/[email protected] - @mysten/[email protected] ## @mysten/[email protected] ### Patch Changes - Updated dependencies [c24814b] - @mysten/[email protected] ## @mysten/[email protected] ### Patch Changes - Updated dependencies [c24814b] - @mysten/[email protected] ## @mysten/[email protected] ### Patch Changes - Updated dependencies [c24814b] - @mysten/[email protected] - @mysten/[email protected] ## @mysten/[email protected] ### Patch Changes - Updated dependencies [c24814b] - @mysten/[email protected] ## @mysten/[email protected] ### Patch Changes - 4166d71: Fix doc comment on `getKiosk` command - Updated dependencies [c24814b] - @mysten/[email protected] ## @mysten/[email protected] ### Patch Changes - b3f3925: Introduces @mysten/kms which initially exposes a Sui AWS KMS signer - Updated dependencies [c24814b] - @mysten/[email protected] ## @mysten/[email protected] ### Patch Changes - Updated dependencies [c24814b] - @mysten/[email protected] ## @mysten/[email protected] ### Patch Changes - Updated dependencies [c24814b] - @mysten/[email protected] ## @mysten/[email protected] ### Patch Changes - Updated dependencies [c24814b] - @mysten/[email protected] ## @mysten/[email protected] ### Patch Changes - Updated dependencies [c24814b] - @mysten/[email protected] - @mysten/[email protected] Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> commit 093954a09bce1942a9f9773b845f59bf5124efbf Author: ronny-mysten <[email protected]> Date: Thu Oct 31 11:35:27 2024 -0600 [docs] Rpc update (#20094) ## Description Updates the RPC render to include component schemas. Also updates a few comments. ## Test plan How did you test the new or updated feature? --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit fa66617b7dfa7096fd64a2dc1b002d154223be4a Author: wlmyng <[email protected]> Date: Thu Oct 31 10:18:57 2024 -0700 [mvr] renaming to sui-mvr-indexer and Dockerfile (#20099) ## Description Describe the changes or additions included in this PR. ## Test plan How did you test the new or updated feature? --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit 434206f14a0a21bc62ea08480b4a5c4d36d6cc25 Author: Todd Nowacki <[email protected]> Date: Thu Oct 31 10:18:46 2024 -0700 [move-compiler] Re-organize warning filters (#20103) ## Description - Re-organize warning filters. It should make it a bit cleaner to refactor diagnostic reporting ## Test plan - ran tests --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit 0d16b6f8b7c4ff79f70400e8a94d7486b2b2c3fe Author: wlmyng <[email protected]> Date: Thu Oct 31 09:36:37 2024 -0700 [mvr] new crate for custom mvr-indexer, copied from sui-indexer crate (#20098) ## Description Simple copy-paste of the sui-indexer crate for the custom mvr indexer ## Test plan How did you test the new or updated feature? --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit b3f39253ff0153b69b42bc298d1ecdbb0524e276 Author: John Naulty Jr. <[email protected]> Date: Thu Oct 31 09:21:29 2024 -0700 AWS KMS TS Signer (#16139) ## Description Describe the changes or additions included in this PR. ## Test Plan How did you test the new or updated feature? --- If your changes are not user-facing and do not break anything, you can skip the following section. Otherwise, please briefly describe what has changed under the Release Notes section. ### Type of Change (Check all that apply) - [ ] protocol change - [ ] user-visible impact - [ ] breaking change for a client SDKs - [ ] breaking change for FNs (FN binary must upgrade) - [ ] breaking change for validators or node operators (must upgrade binaries) - [ ] breaking change for on-chain data layout - [ ] necessitate either a data wipe or data migration ### Release notes --------- Co-authored-by: Manolis Liolios <[email protected]> commit f78c7b006c28c4c1afd3f7d42e21e53970902f5d Author: Brandon Williams <[email protected]> Date: Wed Oct 30 19:50:45 2024 -0500 rest: revert client change to requested protobuf and request bcs instead Revert the client change to request protobuf as a response type, and instead request bcs, because the support for protobuf hasn't rolled out to other networks yet. commit 108ad4c8ea53c68bd90d0b3fcdc88fbc75640c61 Author: Brandon Williams <[email protected]> Date: Wed Oct 30 12:50:43 2024 -0500 rest: support protobuf format for committee endpoints commit 718a244bca584382faedde56f8247c6b26225b07 Author: Brandon Williams <[email protected]> Date: Wed Oct 30 11:03:43 2024 -0500 rest: support protobuf format for resolve, simulate, and execute transaction endpoints commit 2987f5967ecffa8ddbec2b8ebf9ee2eec09dad60 Author: Brandon Williams <[email protected]> Date: Wed Oct 30 08:55:51 2024 -0500 rest: support protobuf format for GetTransaction and ListTransactions endpoints commit 672ecbd2a58c7514aa24d139c442316a03d7f89e Author: Brandon Williams <[email protected]> Date: Tue Oct 29 21:30:42 2024 -0500 rest: support protobuf format for GetObject endpoints commit c8472c057523ff88040ae688f1410d34164b0ebc Author: Brandon Williams <[email protected]> Date: Tue Oct 29 19:00:39 2024 -0500 rest: remove ListFullCheckpoint api Remove the ListFullCheckpoint api as its a bit too heavy-weight and instead we'll look to introduce a streaming api in the future. commit 39f1ae21fd5467e71522729519720d2add9468dc Author: Xun Li <[email protected]> Date: Wed Oct 30 22:25:50 2024 -0700 Clean up local execution (#20086) ## Description Describe the changes or additions included in this PR. ## Test plan How did you test the new or updated feature? --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit 65b02e203ad897f438f87d0e5cfe46c56a1d4247 Author: phoenix <[email protected]> Date: Wed Oct 30 21:26:54 2024 -0400 [pruner] enable periodic compaction of individual SST files by default (#20101) ## Description enables periodic compaction of individual SST files by default --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit f2670718f694edd59e8a062bc932d4efd96ba765 Author: Tim Zakian <[email protected]> Date: Wed Oct 30 16:49:01 2024 -0700 [execution] Update to deps-only mode (#20102) ## Description Disallow new modules from being added to packages in deps-only mode. ## Test plan Added new tests to check for both previous and new behavior. --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [X] Protocol: Updated restrictions on deps-only packages to not allow adding new modules to the package. - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: commit 7e6319fa33b7021a94ff7dcc07e479a4bae92b55 Author: Adam Welc <[email protected]> Date: Wed Oct 30 15:51:37 2024 -0700 [trace-view] Added support for native functions and global locations (#20080) ## Description Adds support for native functions where the `CloseFrame` event happens right after `OpenFrame` event and both need to be skipped to skip over the native function. Also added support for handling global locations - they should be parsed properly but largely ignored in the trace as they cannot directly affect values of local variables ## Test plan All old and new tests must pass commit 21639a51847f1a04b6b221ac77df3875c0430be7 Author: Xun Li <[email protected]> Date: Wed Oct 30 14:30:59 2024 -0700 [RPC] Fix coin metadata api inefficiency (#19794) ## Description This PR fixes a minor inefficiency in the coin metadata api. It does not need to fetch the transaction, as it only needs the effects. This likely avoids a remote KV fetch. ## Test plan CI --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API: comm…
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Reverts #19796 due to suspicion it's causing performance issues
Release notes
Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required.
For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates.