Soroban settings upgrade contract and script #3843

sisuresh · 2023-07-19T22:33:41Z

Description

This is still a work in progress.

What it can do -

Retrieve settings through Soroban RPC -

python3 SorobanSettingsUpgrade.py getSettings -id 10
<LedgerEntryData [type=8, config_setting=<ConfigSettingEntry [config_setting_id=10, state_expiration_settings=<StateExpirationSettings [max_entry_expiration=<Uint32 [uint32=6312000]>, min_temp_entry_expiration=<Uint32 [uint32=16]>, min_persistent_entry_expiration=<Uint32 [uint32=86400]>, auto_bump_ledgers=<Uint32 [uint32=0]>, persistent_rent_rate_denominator=<Int64 [int64=0]>, temp_rent_rate_denominator=<Int64 [int64=0]>, max_entries_to_expire=<Uint32 [uint32=0]>, bucket_list_size_window_sample_size=<Uint32 [uint32=0]>, eviction_scan_size=<Uint64 [uint64=0]>]>]>]>

Setup an upgrade that is hardcoded in the script through Soroban RPC -

python3 SorobanSettingsUpgrade.py setupUpgrade
...
...
url encoded upgrade: /SW4vX5dqlA1GNLaAPt%2BtIiLqZjGIOpJ1lrBoV4pA8tvy3TMBHuWLuVigdUG2XA/k1KxRH6RQ8WUKprvSRfm4A%3D%3D

What we can't do yet -

Specify which settings to upgrade with a better UX than updating the hardcoded setting in the script. Maybe a JSON file with the upgrades will work well.
Update the network used (futurenet vs standalone vs pubnet) on the command line.
Directly submit the setup transactions to stellar-core through the tx endpoint to reduce dependencies during emergencies.
Fund the source account with friendbot if it doesn't exist.
Add an endpoint in stellar-core that takes a proposed ConfigUpgradeSetKey and returns what the upgrade diff will be.

Checklist

Reviewed the contributing document
Rebased on top of master (no merge commits)
Ran clang-format v8.0.0 (via make format or the Visual Studio extension)
Compiles
Ran all tests
If change impacts performance, include supporting evidence per the performance document

anupsdf · 2023-07-26T22:12:20Z

scripts/README.md

+- Description - A python script that can setup a setting upgrade or retrieve
+  current settings for Futurenet through Soroban RPC. The next step is to submit all transactions directly to stellar-core's `tx` endpoint. Note that the actual upgrade command will have to be
+  submitted manually on the core nodes. 
+- Prerequisites - run `make build` under `soroban-settings/write_upgrade_bytes`


make build wanted me to install the target wasm32-unknown-unknown so we can add a pre-requisite to run rustup target add wasm32-unknown-unknown.

anupsdf · 2023-07-26T22:18:49Z

scripts/README.md

+  submitted manually on the core nodes. 
+- Prerequisites - run `make build` under `soroban-settings/write_upgrade_bytes`
+  to the build WASM contract used to write the proposed upgrade.
+- Usage - Ex. `SorobanSettingsUpgrade.py getSettings -id 10` to print out the


nit: Maybe mention to install python

I added python3 to the commands.

anupsdf · 2023-07-26T23:40:06Z

scripts/soroban-settings/SorobanSettingsUpgrade.py

+from stellar_sdk.soroban import SorobanServer
+from stellar_sdk.soroban.types import Address, Int128, Bytes
+from stellar_sdk.soroban.soroban_rpc import GetTransactionStatus
+from stellar_sdk.xdr import TransactionMeta, LedgerKey, ConfigUpgradeSet, ConfigSettingContractExecutionLanesV0, ConfigUpgradeSetKey, ConfigSettingEntry, StateExpirationSettings, Uint32, Uint64, Int64, Hash, LedgerKeyConfigSetting, ConfigSettingID


How about adding ConfigSettingContractLedgerCostV0 since that has read/write bytes and ledger settings there? We can throttle those using this script.

Or we can add this in next iteration of the script.

Ah! I see what the id argument corresponds to ConfigSettingEntry XDR union. So passing 9 would fetch us contract_data_entry_size_bytes.

Shaptic

quick drive-by review 🏎️💨

Shaptic · 2023-07-20T18:48:21Z

scripts/soroban-settings/SorobanSettingsUpgrade.py

+from stellar_sdk.soroban import SorobanServer
+from stellar_sdk.soroban.types import Address, Int128, Bytes
+from stellar_sdk.soroban.soroban_rpc import GetTransactionStatus
+from stellar_sdk.xdr import TransactionMeta, LedgerKey, ConfigUpgradeSet, ConfigSettingContractExecutionLanesV0, ConfigUpgradeSetKey, ConfigSettingEntry, StateExpirationSettings, Uint32, Uint64, Int64, Hash, LedgerKeyConfigSetting, ConfigSettingID


why don't you from stellar_sdk.xdr import * and then you don't need that weird alias at the top heh

Good point. Fixed

Shaptic · 2023-07-20T18:48:46Z

scripts/soroban-settings/SorobanSettingsUpgrade.py

+import argparse
+import time
+import sys
+sys.path.append("/Users/siddharthsuresh/dev/py-stellar-base")


this has no effect if it's after the imports

Good point. This wasn't necessary.

Shaptic · 2023-07-20T18:49:44Z

scripts/soroban-settings/SorobanSettingsUpgrade.py

+
+        assert wasm_id, "wasm id should not be empty"
+
+    print("creating contract...")


you're gonna wanna check the GetTransactionStatus.FAILED case here heh

Good catch. Updated to assert success.

MonsieurNicolas · 2023-07-27T04:43:50Z

Not sure if you want to do this in this PR, but there is a section on network upgrades https://github.com/stellar/stellar-core/blob/master/docs/software/admin.md#network-configuration that needs to updated for Soroban (it's the operator user guide)

sisuresh · 2023-07-27T19:56:24Z

Not sure if you want to do this in this PR, but there is a section on network upgrades https://github.com/stellar/stellar-core/blob/master/docs/software/admin.md#network-configuration that needs to updated for Soroban (it's the operator user guide)

Updated.

Shaptic · 2023-07-28T20:03:30Z

scripts/soroban-settings/SorobanSettingsUpgrade.py

+import sys
+
+# The soroban branch of py-stellar-base hasn't been merged into main and released yet, so we have to install it locally.
+sys.path.append("/Users/dev/py-stellar-base")


you can install from the branch via pip, if you want to add that to the instructions

pip install git+https://github.com/StellarCN/py-stellar-base.git@soroban

Wow I'm a python noob. This is good to know. Thanks!

I removed this line here and added instructions yesterday for installing from source but the PR isn't updating for some reason.

Heh this is thanks to @overcat, from here: stellar/sorobanathon#20. I do see it's gone from the branch, though 👍

Shaptic · 2023-07-28T20:05:30Z

scripts/soroban-settings/SorobanSettingsUpgrade.py

+    # Get upgrade xdr
+
+    upgrade = get_upgrade_set()
+    upgrade_xdr = upgrade.to_xdr()


nit: unused

Shaptic · 2023-07-28T20:10:20Z

scripts/soroban-settings/SorobanSettingsUpgrade.py

+    # Get upgrade
+    upgrade_key = get_upgrade_key(contract_id_hash, upgrade_hash).to_xdr()
+    url_encoded_key = urllib.parse.quote(upgrade_key)
+    print(f"url encoded upgrade: {url_encoded_key}")


Just to be sure, Core's http server can handle the /s in the encoded key and won't treat them as endpoint paths? You can pass safe='' to quote if not

Yeah core handles it

scripts/soroban-settings/write_upgrade_bytes/src/lib.rs

dmkozh · 2023-07-31T19:58:43Z

r+ a8ef9cc

Soroban settings upgrade contract and script

9c75aee

sisuresh force-pushed the upgrade-script branch from bb096cb to 9c75aee Compare July 26, 2023 21:44

anupsdf reviewed Jul 26, 2023

View reviewed changes

Add command to README.py

e45f08d

sisuresh force-pushed the upgrade-script branch from 0966ada to e45f08d Compare July 26, 2023 23:11

anupsdf reviewed Jul 26, 2023

View reviewed changes

Add patch to local py-stellar-base

27733fd

Shaptic reviewed Jul 27, 2023

View reviewed changes

sisuresh added 2 commits July 27, 2023 12:34

cleanup

283c360

Update docs

d2d7574

Shaptic reviewed Jul 28, 2023

View reviewed changes

dmkozh reviewed Jul 28, 2023

View reviewed changes

scripts/soroban-settings/write_upgrade_bytes/src/lib.rs Show resolved Hide resolved

dmkozh reviewed Jul 28, 2023

View reviewed changes

scripts/soroban-settings/write_upgrade_bytes/src/lib.rs Outdated Show resolved Hide resolved

More cleanup

a8ef9cc

latobarita merged commit 22416d9 into stellar:master Jul 31, 2023
8 checks passed

anupsdf mentioned this pull request Jul 31, 2023

Soroban launch criteria for Testnet release stellar/rs-soroban-env#971

Closed

sisuresh mentioned this pull request Jul 31, 2023

Create the necessary components and workflow for upgrading network configuration #3795

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Soroban settings upgrade contract and script #3843

Soroban settings upgrade contract and script #3843

sisuresh commented Jul 19, 2023

anupsdf Jul 26, 2023

sisuresh Jul 26, 2023

anupsdf Jul 26, 2023

sisuresh Jul 26, 2023

anupsdf Jul 26, 2023

anupsdf Jul 26, 2023

anupsdf Jul 26, 2023

Shaptic left a comment

Shaptic Jul 20, 2023

sisuresh Jul 27, 2023

Shaptic Jul 20, 2023

sisuresh Jul 27, 2023

Shaptic Jul 20, 2023

sisuresh Jul 27, 2023

MonsieurNicolas commented Jul 27, 2023

sisuresh commented Jul 27, 2023

Shaptic Jul 28, 2023

sisuresh Jul 28, 2023

Shaptic Jul 28, 2023

Shaptic Jul 28, 2023

Shaptic Jul 28, 2023 •

edited

Loading

sisuresh Jul 28, 2023

dmkozh commented Jul 31, 2023


		assert wasm_id, "wasm id should not be empty"

		print("creating contract...")

Soroban settings upgrade contract and script #3843

Soroban settings upgrade contract and script #3843

Conversation

sisuresh commented Jul 19, 2023

Description

Checklist

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Shaptic left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

MonsieurNicolas commented Jul 27, 2023

sisuresh commented Jul 27, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Shaptic Jul 28, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dmkozh commented Jul 31, 2023

Shaptic Jul 28, 2023 •

edited

Loading