diff --git a/README.md b/README.md index 8e18e9b2..ed19dccd 100644 --- a/README.md +++ b/README.md @@ -11,13 +11,11 @@ This repo is your entrypoint to the world of IPC. In this repo you will find: >💡 **We've prepared a [quick start guide](/docs/quickstart-calibration.md) that will have you running and validating on your own subnet quickly, at the cost of detailed explanations.** See: -- [docs/subnet.md](docs/subnet.md) for instructions on how to deploy a new subnet and the required architecture -- [docs/usage.md](docs/usage.md) for instructions on how to use the IPC Agent to interact with subnets -- [docs/deploying-hierarchy.md](docs/deploying-hierarchy.md) for instructions on how to deploy your own IPC root contract and hierarchy - [docs/contracts.md](docs/contracts.md) for instructions on how to deploy FEVM actors on subnets -- [docs/troubleshooting.md](docs/troubleshooting.md) for answers to some common questions - -For a detailed overview of the entire IPC stack design, please check the up-to-date **[IPC Design Reference](https://github.com/consensus-shipyard/IPC-design-reference-spec/blob/main/main.pdf)** doc. +- __NEEDS UPDATE/REMOVE:__ [docs/subnet.md](docs/subnet.md) for instructions on how to deploy a new subnet and the required architecture) +- __NEEDS UPDATE/REMOVE:__ [docs/usage.md](docs/usage.md) for instructions on how to use the IPC Agent to interact with subnets +- __NEEDS UPDATE/REMOVE:__ [docs/deploying-hierarchy.md](docs/deploying-hierarchy.md) for instructions on how to deploy your own IPC root contract and hierarchy +- __NEEDS UPDATE/REMOVE:__ [docs/troubleshooting.md](docs/troubleshooting.md) for answers to some common questions ## Branching Strategy diff --git a/docs/contracts.md b/docs/contracts.md index 6a93ab2c..4f17de46 100644 --- a/docs/contracts.md +++ b/docs/contracts.md @@ -12,13 +12,10 @@ In order to connect the Ethereum tooling to your subnet, you'll need to get the # Sample command $ ./bin/ipc-agent subnet rpc --subnet /r31415926/t2xwzbdu7z5sam6hc57xxwkctciuaz7oe5omipwbq -[2023-05-17T15:10:57Z INFO ipc_agent::cli::commands::subnet::rpc] rpc endpoint for subnet /r31415926/t2xwzbdu7z5sam6hc57xxwkctciuaz7oe5omipwbq: http://127.0.0.1:1240/rpc/v1 -[2023-05-17T15:10:57Z INFO ipc_agent::cli::commands::subnet::rpc] chainID for subnet /r31415926/t2xwzbdu7z5sam6hc57xxwkctciuaz7oe5omipwbq: 31415926 +rpc: http://127.0.0.1:1240/rpc/v1 +chainID: 31415926 ``` -You can also inspect the `json_rpcapi_http` field of your subnet on your config directly to get the RPC endpoint for your subnet. -This RPC endpoint and `chainID will be the ones needed to configure any EVM tooling to connect to your subnet. - ### Example: Connect Metamask to your subnet @@ -49,9 +46,9 @@ It is important to note that the IPC agent doesn't understand Ethereum addresses ./bin/ipc-agent util eth-to-f4-addr --addr $ ./bin/ipc-agent util eth-to-f4-addr --addr 0x6BE1Ccf648c74800380d0520D797a170c808b624 -[2023-05-17T13:37:37Z INFO ipc_agent::cli::commands::util::f4] f4 address: t410fnpq4z5siy5eaaoanauqnpf5bodearnren5fxyoi +t410fnpq4z5siy5eaaoanauqnpf5bodearnren5fxyoi ``` >💡 For more information about the relationship between `f4` and Ethereum addresses refer to [this page](https://docs.filecoin.io/smart-contracts/filecoin-evm-runtime/address-types/). -From there on, you should be able to follow the same steps currently used to deploy EVM contract in the Filecoin mainnet. You can find here the steps to [deploy an ERC20 contract using Remix](https://docs.filecoin.io/smart-contracts/fundamentals/erc-20-quickstart/). \ No newline at end of file +From there on, you should be able to follow the same steps currently used to deploy EVM contract in the Filecoin mainnet. You can find here the steps to [deploy an ERC20 contract using Remix](https://docs.filecoin.io/smart-contracts/fundamentals/erc-20-quickstart/). diff --git a/docs/quickstart-calibration.md b/docs/quickstart-calibration.md index 12ef5e1d..d2085826 100644 --- a/docs/quickstart-calibration.md +++ b/docs/quickstart-calibration.md @@ -2,13 +2,13 @@ >💡 Background and detailed are available in the [README](/README.md). -Ready to test the waters with your first subnet? This guide will deploy a subnet with three local validators orchestrated by the same IPC agent. This subnet will be anchored to the public [Calibration testnet](https://docs.filecoin.io/networks/calibration/details/). This will be a minimal example and may not work on all systems. The full documentation provides more details on each step. +Ready to test the waters with your first subnet? This guide will deploy a subnet with three local validators orchestrated by `ipc-cli`. This subnet will be anchored to the public [Calibration testnet](https://docs.filecoin.io/networks/calibration/details/). This will be a minimal example and may not work on all systems. The full documentation provides more details on each step. Several steps in this guide involve running long-lived processes. In each of these cases, the guide advises starting a new *session*. Depending on your set-up, you may do this using tools like `screen` or `tmux`, or, if using a graphical environment, by opening a new terminal tab, pane, or window. ->💡A video walkthrough of this guide is current being prepared. We still encourage you to try it for yourself! + ->💡If you're only looking to connect to an existing subnet, please see the [README](deploying-hierarchy.md) instead. + ## Step 0: Prepare your system @@ -41,238 +41,195 @@ sudo usermod -aG docker $USER && newgrp docker ## Step 1: Build the IPC stack -Next, we'll download and build the different components (IPC agent, docker images, and eudico). +Next, we'll download and build the different components (mainly, `ipc-cli` and Fendermint). -* Pick a folder where to build the IPC stack. In this example, we'll go with `~/ipc/`. +* Pick a folder where to build the IPC stack. +* Download and compile the `ipc-cli`. ```bash -mkdir -p ~/ipc/ && cd ~/ipc/ +git clone https://github.com/consensus-shipyard/ipc.git +(cd ipc && make build && make install-infra) ``` -* Download and compile the IPC Agent (might take a while) -```bash -git clone https://github.com/consensus-shipyard/ipc-agent.git -(cd ipc-agent && make build && make install-infra) -``` - -## Step 2: Initialise and start the IPC Agent +## Step 2: Initialise your config * Initialise the config ```bash -./ipc-agent/bin/ipc-agent config init -nano ~/.ipc-agent/config.toml +./bin/ipc-cli config init ``` -* Replace the content of `config.toml` with the text below, including the reference contract on Calibration. +This should have populated an default config file with all the parameters required to connect to calibration at `~/.ipc/config.toml`. Feel free to update this configuration to fit your needs. You may need to replace the content of the config to reflect the address of the up-to-date contracts in Calibration. + + +* You can run `nano ~/.ipc/config.toml` to double-check that the config file has been populated with the following content: ```toml -[server] -json_rpc_address = "0.0.0.0:3030" +keystore_path = "~/.ipc" [[subnets]] id = "/r314159" -network_name = "calibration" [subnets.config] -gateway_addr = "0x5fBdA31a37E05D8cceF146f7704f4fCe33e2F96F" +gateway_addr = "0x56948d2CFaa2EF355B8C08Ac925202db212146D1" network_type = "fevm" provider_http = "https://api.calibration.node.glif.io/rpc/v1" -registry_addr = "0xb505eD453138A782b5c51f45952E067798F4777d" -``` +registry_addr = "0x6A4884D2B6A597792dC68014D4B7C117cca5668e" -* [**In a new session**] Start your IPC Agent -```bash -./ipc-agent/bin/ipc-agent daemon -``` +# Subnet template - uncomment and adjust before using +# [[subnets]] +# id = "/r314159/" +# [subnets.config] +# gateway_addr = "t064" +# jsonrpc_api_http = "http://127.0.0.1:1251/rpc/v1" +# auth_token = "" +# network_type = "fvm" +``` -## Step 3: Set up your owner wallets +## Step 3: Set up your wallets -You'll need to create a set of owner wallets. Please make a note of the addresses as you go along. +You'll need to create a set of wallets to spawn and interact of the subnet. Please make a note of the addresses as you go along, it may make your life easier. -* Create the owner wallets for each validator (OWNER_1, OWNER_2, and OWNER_3) +* Create the three different wallets ```bash -./ipc-agent/bin/ipc-agent wallet new -w evm -./ipc-agent/bin/ipc-agent wallet new -w evm -./ipc-agent/bin/ipc-agent wallet new -w evm +./bin/ipc-cli wallet new -w evm +./bin/ipc-cli wallet new -w evm +./bin/ipc-cli wallet new -w evm ``` -* Copy your new wallet addresses into `~/.ipc-agent/config.toml` -```toml -... -accounts = ["", "", ""] -... -``` - -* Reload the config to apply the changes. +* You can optionally set one of the wallets as your default so you don't have to use the `--from` flag explicitly in some of the commands: ```bash -./ipc-agent/bin/ipc-agent config reload +./bin/ipc-cli wallet set-default --address -w evm ``` -* Convert the 0x addresses to f4 addresses for later usage (OWNER_1_F4, OWNER_2_F4, and OWNER_3_F4) -```bash -./ipc-agent/bin/ipc-agent util eth-to-f4-addr --addr -./ipc-agent/bin/ipc-agent util eth-to-f4-addr --addr -./ipc-agent/bin/ipc-agent util eth-to-f4-addr --addr -``` + + + + + + * Go to the [Calibration faucet](https://faucet.calibration.fildev.network/) and get some funds sent to each of your addresses ->💡 In case you'd like to import an EVM account into Metamask, you can use export the private key using `./ipc-agent/bin/ipc-agent wallet export -w evm -a
`. More information is available in the [EVM IPC agent support docs](./usage.md#key-management). +>💡 In case you'd like to import an EVM account into Metamask, you can use export the private key using `./bin/ipc-cli wallet export -w evm -a
`. More information is available in the [EVM IPC agent support docs](./usage.md#key-management). >💡 Note that you may hit faucet rate limits. In that case, wait a few minutes or continue with the guide and come back to this before step 9. Alternatively, you can send funds from your primary wallet to your owner wallets. -## Step 4: Set up your validator worker wallets +## Step 4: Create a child subnet -Mir validators do not support the use of EVM addresses to create new blocks. Therefore, we'll need to create separate worker wallets for each validator. - -* First, create a worker wallet for each validator (WORKER_1, WORKER_2, and WORKER_3) +* The next step is to create a subnet under `/r314159` in calibration. Remember to set a default wallet or explicitly specifying the wallet from which you want to perform the action with the `--from` flag. ```bash -./ipc-agent/bin/ipc-agent wallet new -w fvm --key-type secp256k1 -./ipc-agent/bin/ipc-agent wallet new -w fvm --key-type secp256k1 -./ipc-agent/bin/ipc-agent wallet new -w fvm --key-type secp256k1 +./bin/ipc-cli subnet create --parent /r314159 --min-validators 3 --min-validator-stake 1 --bottomup-check-period 30 ``` -* Export each wallet by substituting their addresses below -```bash -./ipc-agent/bin/ipc-agent wallet export -w fvm --address --output ~/.ipc-agent/worker-wallet1.key -./ipc-agent/bin/ipc-agent wallet export -w fvm --address --output ~/.ipc-agent/worker-wallet2.key -./ipc-agent/bin/ipc-agent wallet export -w fvm --address --output ~/.ipc-agent/worker-wallet3.key -``` +* Make a note of the address of the subnet you created (`/r314159/`) +## Step 5: Join the subnet -## Step 5: Create a child subnet +Before we deploy the infrastructure for the subnet, we will have to bootstrap the subnet and join from our validators, putting some initial collateral into the subnet. For this, we need to send a `join` command from each of our validators from their validator owner addresses providing their corresponding public key. -* The next step is to create a subnet under `/r314159` in calibration +* Get the public key for all of your wallets and note it down. This is the public key that each of your validators will use to sign blocks in the subnet. ```bash -./ipc-agent/bin/ipc-agent subnet create --parent /r314159 --name andromeda --min-validator-stake 10 --min-validators 2 --bottomup-check-period 30 --topdown-check-period 30 +./bin/ipc-cli wallet pub-key -w evm --address +./bin/ipc-cli wallet pub-key -w evm --address +./bin/ipc-cli wallet pub-key -w evm --address ``` -* Make a note of the address of the subnet you created (`/r314159/`) - - -## Step 6: Deploy the infrastructure - -We can deploy the subnet nodes. Note that each node should be importing a different worker wallet key for their validator, and should be exposing different ports. If these ports are unavailable in your system, please pick different ones. - -* Deploy and run a container for each validator, importing the corresponding wallet keys +* Join the subnet with each validator ```bash -./ipc-agent/bin/ipc-infra/run-subnet-docker.sh 1251 1351 /r314159/ ~/.ipc-agent/worker-wallet1.key -./ipc-agent/bin/ipc-infra/run-subnet-docker.sh 1252 1352 /r314159/ ~/.ipc-agent/worker-wallet2.key -./ipc-agent/bin/ipc-infra/run-subnet-docker.sh 1253 1353 /r314159/ ~/.ipc-agent/worker-wallet3.key +./bin/ipc-cli subnet join --from= --subnet=/r314159/ --collateral=10 --public-key= +./bin/ipc-cli subnet join --from= --subnet=/r314159/ --collateral=10 --public-key= +./bin/ipc-cli subnet join --from= --subnet=/r314159/ --collateral=10 --public-key= ``` -* If the deployment is successful, each of these nodes should return the following output at the end of their logs. Save the information for the next step. -``` ->>> Subnet /r314159/ daemon running in container: (friendly name: ) ->>> Token to /r314159/ daemon: ->>> Default wallet: ->>> Subnet validator info: - ->>> API listening in host port ->>> Validator listening in host port -``` +## Step 6: Deploy the infrastructure +With the collateral and number of minimum validators fulfilled, the subnet is bootstrapped in teh parent, and we can deploy the infrastructure. -## Step 7: Interconnect the validators +### Deploying a bootstrap node +Before running our validators, at least one bootstrap needs to be deployed and advertised in the network. Bootstrap nodes allow validators discover other peers and validators in the network. In the current implementation of IPC, only validators are allowed to advertise bootstrap nodes. -* Establish pairwise peer connections between all validators +* We can deploy a new bootstrap node in the subnet by running: ```bash -docker exec -it eudico net connect `docker exec -it eudico net listen | head -n 1 | tr -d '\r'` -docker exec -it eudico net connect `docker exec -it eudico net listen | head -n 1 | tr -d '\r'` -docker exec -it eudico net connect `docker exec -it eudico net listen | head -n 1 | tr -d '\r'` +cargo make --makefile bin/ipc-infra/Makefile.toml bootstrap ``` -## Step 8: Update the IPC Agent configuration +At the end of the output, this command should return the ID of your new bootstrap node: +```console -* Edit the IPC agent configuration `config.toml` -```bash -nano ~/.ipc-agent/config.toml +[cargo-make] INFO - Running Task: cometbft-wait +[cargo-make] INFO - Running Task: cometbft-node-id +2b23b8298dff7711819172252f9df3c84531b1d9@172.26.0.2:26650 +[cargo-make] INFO - Build Done in 13.38 seconds. ``` +Remember the address of your bootstrap for the next step. This address has the following format `id@ip:port`, and by default shows its Docker address. Feel free to adjust the `ip` to use a reachable IP for your deployment so other nodes can contact it (in our case our localhost IP, `127.0.0.1`). -* Append the new subnet to the configuration -```toml -[[subnets]] -id = "/r314159/" -network_name = "andromeda" - -[subnets.config] -gateway_addr = "t064" -accounts = ["", "", ""] -jsonrpc_api_http = "http://127.0.0.1:1251/rpc/v1" -auth_token = "" -network_type = "fvm" +* To advertise the endpoint to the rest of nodes in the network we need to run: +```bash +# Example of BOOTSTRAP_ENDPOINT = 2b23b8298dff7711819172252f9df3c84531b1d9@172.26.0.2:26650 +./bin/ipc-cli subnet add-bootstrap --subnet= --endpoint= ``` -* Reload the config -```bash -./ipc-agent/bin/ipc-agent config reload +* The bootstrap nodes currently deployed in the network can be queried through the following command: +```bash +./bin/ipc-cli subnet list-bootstraps --subnet= ``` +### Deploying the validator infrastructure +With the bootstrap node deployed and advertised to the network, we are now ready to deploy the validators that will run the subnet. -## Step 9: Join the subnet - -All the infrastructure for the subnet is now deployed, and we can join our validators to the subnet. For this, we need to send a `join` command from each of our validators from their validator owner addresses providing the validators multiaddress. - -* Join the subnet with each validator +* First we need to export the private keys of our validators from the addresses that we created with our `ipc-cli wallet` to a known path so they can be picked by Fendermint to sign blocks. We can use the default repo of IPC for this, `~/.ipc`. ```bash -./ipc-agent/bin/ipc-agent subnet join --subnet /r314159/ --collateral 10 --from --validator-net-addr --worker-addr -./ipc-agent/bin/ipc-agent subnet join --subnet /r314159/ --collateral 10 --from --validator-net-addr --worker-addr -./ipc-agent/bin/ipc-agent subnet join --subnet /r314159/ --collateral 10 --from --validator-net-addr --worker-addr +./bin/ipc-cli wallet export -w evm -a --fendermint -o ~/.ipc/ +./bin/ipc-cli wallet export -w evm -a --fendermint -o ~/.ipc/ +./bin/ipc-cli wallet export -w evm -a --fendermint -o ~/.ipc/ ``` ->💡 Make sure to use the f4 addresses for the owner wallets +* Now we have all that we need to deploy the three validators using the following command (configured for each of the validators, i.e. replace the arguments with `<..-n>` to fit that of the specific validator). - -## Step 10: Start validating! - -We have everything in place now to start validating. Run the following script for each of the validators [**each in a new session**], passing the container names: ```bash -./ipc-agent/bin/ipc-infra/mine-subnet.sh -./ipc-agent/bin/ipc-infra/mine-subnet.sh -./ipc-agent/bin/ipc-infra/mine-subnet.sh +cargo make --makefile /bin/ipc-infra/Makefile.toml \ + -e NODE_NAME=validator- \ + -e VALIDATOR_PRIV_KEY= \ + -e CHAIN_NAME= + -e CMT_HOST_PORT= -e ETHAPI_HOST_PORT= \ + -e COMMA_SEPARATED_BOOTSTRAPS= + -e PARENT_REGISTRY= \ + -e PARENT_GATEAY= \ + child-validator ``` +`PARENT_REGISTRY` and `PARENT_GATEWAY` are the contract addresses of the IPC contracts in Calibration. This command also uses the calibration endpoint as default. Finally, you'll need to choose a different `NODE_NAME`, `CMT_HOST_PORT`, `ETHAPI_HOST_PORT` for each of the validators. ->💡 When starting mining and reloading the config to include the new subnet, you can sometimes get errors in the agent logs saying that the checkpoint manager couldn't be spawned successfully because the on-chain ID of the validator couldn't be change. This is because the subnet hasn't been fully initialized yet. You can `./ipc-agent/bin/ipc-agent config reload` to re-spawn the checkpoint manager and fix the error. - +With this, we have everything in place, and our subnet should start automatically validating new blocks. You can find additional documentation on how to run the infrastructure in the [Fendermint docs](https://github.com/consensus-shipyard/fendermint/blob/main/docs/ipc.md). -## Step 11: Deploy IPC Gateway [optional] +## Step 7: Configure your subnet in the IPC CLI -If you'd like to interact with your subnet using Metamask or other tooling, you should deploy a `lotus-gateway` instance for tokenless RPC access. - ->💡 The instructions below assume you do not have a local `lotus` set-up. If you do, you may want to create a separate directory for `lotus-gw` and pass it as an argument to the application. - -* Install Go [Linux] ([details](https://go.dev/doc/install)) +* Edit the `ipc-cli` configuration `config.toml` ```bash -curl -fsSL https://golang.org/dl/go1.19.7.linux-amd64.tar.gz | sudo tar -xz -C /usr/local -echo 'export PATH=$PATH:/usr/local/go/bin' >> ~/.bashrc && source ~/.bashrc +nano ~/.ipc/config.toml ``` -* Download and compile eudico (might take a while) -```bash -git clone https://github.com/consensus-shipyard/lotus.git -(cd lotus && make spacenet && make lotus-gateway) -``` - -* Create the config directory and populate the configuration -```bash -mkdir -p ~/.lotus/datastore -echo '/ip4/127.0.0.1/tcp/1251/http' > ~/.lotus/api -echo '' > ~/.lotus/token -``` +* Append the new subnet to the configuration +```toml +[[subnets]] +id = "/r314159" -* Obtain your Chain ID -```bash -./ipc-agent/bin/ipc-agent subnet rpc --subnet +[subnets.config] +gateway_addr = "0xff00000000000000000000000000000000000064" +network_type = "fevm" +provider_http = "http://127.0.0.1:" +registry_addr = "0xff00000000000000000000000000000000000065" ``` -* [**In a new session**] Start your lotus-gateway +With this you should be able to start interacting with your local subnet directly through your `ipc-cli`. You can try to fetch the balances of your wallets through: ```bash -./lotus/lotus-gateway run --api-max-lookback=1600000h --api-wait-lookback-limit 2000 +./bin/ipc-cli wallet balances -w evm --subnet= ``` ->💡 You may now use your chain ID and `http://:2346/rpc/v1` as your RPC endpoint in EVM tooling. +## Step 8: Interact with your the ETH RPC +For information about how to connect your Ethereum tooling with your subnet refer to the [following docs](./contracts.md). -## Step 11: What now? -* Proceed to the [usage](usage.md) guide to learn how you can test your new subnet. -* If something went wrong, please have a look at the [README](https://github.com/consensus-shipyard/ipc-agent). If it doesn't help, please join us in #ipc-help. In either case, let us know your experience! -* Please note that to repeat this guide or spawn a new subnet, you may need to change the parameters or reset your system. +## Step 9: What now? +> WIP: Docs in progress + + + diff --git a/docs/subnet.md b/docs/subnet.md index f2a29cec..aa5cd240 100644 --- a/docs/subnet.md +++ b/docs/subnet.md @@ -2,7 +2,7 @@ >💡 For background and setup information, make sure to start with the [README](/README.md). -To spawn a new subnet, our IPC agent should be connected to the parent subnet (or rootnet) from which we plan to deploy a new subnet. Please refer to the [README](/README.md) for information on how to run or connect to a rootnet. This instructions will assume the deployment of a subnet from `/r31415926`, but the steps are equivalent for any other parent subnet. +To spawn a new subnet, we should configure our `ipc-cli` to point to the parent subnet (or rootnet) from which we plan to deploy a new subnet. Please refer to the [README](/README.md) for information on how to run or connect to a rootnet. This instructions will assume the deployment of a subnet from `/r314159` (i.e. Calibration), but the steps are equivalent for any other parent subnet. We provide instructions for running both a [simple single-validator subnet](#running-a-simple-subnet-with-a-single-validator) and a more useful [multi-validator subnet](#running-a-subnet-with-several-validators). The two sets mostly overlap. diff --git a/ipc/cli/src/commands/crossmsg/fund.rs b/ipc/cli/src/commands/crossmsg/fund.rs index a2df865d..392b499a 100644 --- a/ipc/cli/src/commands/crossmsg/fund.rs +++ b/ipc/cli/src/commands/crossmsg/fund.rs @@ -26,7 +26,7 @@ impl CommandLineHandler for Fund { let mut provider = get_ipc_provider(global)?; let subnet = SubnetID::from_str(&arguments.subnet)?; let from = match &arguments.from { - Some(address) => Some(Address::from_str(address)?), + Some(address) => Some(require_fil_addr_from_str(address)?), None => None, }; let to = match &arguments.to { diff --git a/ipc/cli/src/commands/crossmsg/release.rs b/ipc/cli/src/commands/crossmsg/release.rs index 7d2a1a5d..ae509070 100644 --- a/ipc/cli/src/commands/crossmsg/release.rs +++ b/ipc/cli/src/commands/crossmsg/release.rs @@ -4,7 +4,6 @@ use async_trait::async_trait; use clap::Args; -use fvm_shared::address::Address; use ipc_sdk::subnet_id::SubnetID; use std::{fmt::Debug, str::FromStr}; @@ -26,7 +25,7 @@ impl CommandLineHandler for Release { let mut provider = get_ipc_provider(global)?; let subnet = SubnetID::from_str(&arguments.subnet)?; let from = match &arguments.from { - Some(address) => Some(Address::from_str(address)?), + Some(address) => Some(require_fil_addr_from_str(address)?), None => None, }; let to = match &arguments.to { diff --git a/ipc/cli/src/commands/subnet/bootstrap.rs b/ipc/cli/src/commands/subnet/bootstrap.rs new file mode 100644 index 00000000..9c15f9fe --- /dev/null +++ b/ipc/cli/src/commands/subnet/bootstrap.rs @@ -0,0 +1,76 @@ +// Copyright 2022-2023 Protocol Labs +// SPDX-License-Identifier: MIT +//! Subnet bootstrap-related commands + +use async_trait::async_trait; +use clap::Args; +use ipc_sdk::subnet_id::SubnetID; +use std::{fmt::Debug, str::FromStr}; + +use crate::{get_ipc_provider, require_fil_addr_from_str, CommandLineHandler, GlobalArguments}; + +/// The command to add a bootstrap subnet +pub struct AddBootstrap; + +#[async_trait] +impl CommandLineHandler for AddBootstrap { + type Arguments = AddBootstrapArgs; + + async fn handle(global: &GlobalArguments, arguments: &Self::Arguments) -> anyhow::Result<()> { + log::debug!("add subnet bootstrap with args: {:?}", arguments); + + let mut provider = get_ipc_provider(global)?; + let subnet = SubnetID::from_str(&arguments.subnet)?; + let from = match &arguments.from { + Some(address) => Some(require_fil_addr_from_str(address)?), + None => None, + }; + + provider + .add_bootstrap(&subnet, from, arguments.endpoint.clone()) + .await + } +} + +#[derive(Debug, Args)] +#[command(name = "add-bootstrap", about = "Advertise bootstrap in the subnet")] +pub struct AddBootstrapArgs { + #[arg( + long, + short, + help = "The address of the validator adding the bootstrap" + )] + pub from: Option, + #[arg(long, short, help = "The subnet to add the bootstrap to")] + pub subnet: String, + #[arg(long, short, help = "The bootstrap node's network endpoint")] + pub endpoint: String, +} + +/// The command to list bootstrap nodes in a subnet +pub struct ListBootstraps; + +#[async_trait] +impl CommandLineHandler for ListBootstraps { + type Arguments = ListBootstrapsArgs; + + async fn handle(global: &GlobalArguments, arguments: &Self::Arguments) -> anyhow::Result<()> { + log::debug!("add subnet bootstrap with args: {:?}", arguments); + + let provider = get_ipc_provider(global)?; + let subnet = SubnetID::from_str(&arguments.subnet)?; + + for s in provider.list_bootstrap_nodes(&subnet).await? { + print!("{s},"); + } + println!(); + Ok(()) + } +} + +#[derive(Debug, Args)] +#[command(name = "list-bootstraps", about = "List bootstraps in the subnet")] +pub struct ListBootstrapsArgs { + #[arg(long, short, help = "The subnet to list bootstraps from")] + pub subnet: String, +} diff --git a/ipc/cli/src/commands/subnet/create.rs b/ipc/cli/src/commands/subnet/create.rs index b31227e4..b522c21f 100644 --- a/ipc/cli/src/commands/subnet/create.rs +++ b/ipc/cli/src/commands/subnet/create.rs @@ -4,14 +4,13 @@ use async_trait::async_trait; use clap::Args; -use fvm_shared::address::Address; use fvm_shared::clock::ChainEpoch; use ipc_sdk::subnet_id::SubnetID; use std::fmt::Debug; use std::str::FromStr; use crate::commands::get_ipc_provider; -use crate::{f64_to_token_amount, CommandLineHandler, GlobalArguments}; +use crate::{f64_to_token_amount, require_fil_addr_from_str, CommandLineHandler, GlobalArguments}; const DEFAULT_ACTIVE_VALIDATORS: u16 = 100; @@ -25,8 +24,9 @@ impl CreateSubnet { ) -> anyhow::Result { let mut provider = get_ipc_provider(global)?; let parent = SubnetID::from_str(&arguments.parent)?; + let from = match &arguments.from { - Some(address) => Some(Address::from_str(address)?), + Some(address) => Some(require_fil_addr_from_str(address)?), None => None, }; diff --git a/ipc/cli/src/commands/subnet/join.rs b/ipc/cli/src/commands/subnet/join.rs index c1aaca2a..8062b5c3 100644 --- a/ipc/cli/src/commands/subnet/join.rs +++ b/ipc/cli/src/commands/subnet/join.rs @@ -4,11 +4,13 @@ use async_trait::async_trait; use clap::Args; -use fvm_shared::address::Address; use ipc_sdk::subnet_id::SubnetID; use std::{fmt::Debug, str::FromStr}; -use crate::{f64_to_token_amount, get_ipc_provider, CommandLineHandler, GlobalArguments}; +use crate::{ + f64_to_token_amount, get_ipc_provider, require_fil_addr_from_str, CommandLineHandler, + GlobalArguments, +}; /// The command to join a subnet pub struct JoinSubnet; @@ -23,7 +25,7 @@ impl CommandLineHandler for JoinSubnet { let mut provider = get_ipc_provider(global)?; let subnet = SubnetID::from_str(&arguments.subnet)?; let from = match &arguments.from { - Some(address) => Some(Address::from_str(address)?), + Some(address) => Some(require_fil_addr_from_str(address)?), None => None, }; let public_key = hex::decode(&arguments.public_key)?; @@ -68,7 +70,7 @@ impl CommandLineHandler for StakeSubnet { let mut provider = get_ipc_provider(global)?; let subnet = SubnetID::from_str(&arguments.subnet)?; let from = match &arguments.from { - Some(address) => Some(Address::from_str(address)?), + Some(address) => Some(require_fil_addr_from_str(address)?), None => None, }; provider diff --git a/ipc/cli/src/commands/subnet/kill.rs b/ipc/cli/src/commands/subnet/kill.rs index 42afe0ad..17fe42b0 100644 --- a/ipc/cli/src/commands/subnet/kill.rs +++ b/ipc/cli/src/commands/subnet/kill.rs @@ -4,11 +4,10 @@ use async_trait::async_trait; use clap::Args; -use fvm_shared::address::Address; use ipc_sdk::subnet_id::SubnetID; use std::{fmt::Debug, str::FromStr}; -use crate::{get_ipc_provider, CommandLineHandler, GlobalArguments}; +use crate::{get_ipc_provider, require_fil_addr_from_str, CommandLineHandler, GlobalArguments}; /// The command to kill an existing subnet. pub struct KillSubnet; @@ -23,7 +22,7 @@ impl CommandLineHandler for KillSubnet { let mut provider = get_ipc_provider(global)?; let subnet = SubnetID::from_str(&arguments.subnet)?; let from = match &arguments.from { - Some(address) => Some(Address::from_str(address)?), + Some(address) => Some(require_fil_addr_from_str(address)?), None => None, }; diff --git a/ipc/cli/src/commands/subnet/leave.rs b/ipc/cli/src/commands/subnet/leave.rs index 0e4af7d2..5fad89a4 100644 --- a/ipc/cli/src/commands/subnet/leave.rs +++ b/ipc/cli/src/commands/subnet/leave.rs @@ -4,11 +4,10 @@ use async_trait::async_trait; use clap::Args; -use fvm_shared::address::Address; use ipc_sdk::subnet_id::SubnetID; use std::{fmt::Debug, str::FromStr}; -use crate::{get_ipc_provider, CommandLineHandler, GlobalArguments}; +use crate::{get_ipc_provider, require_fil_addr_from_str, CommandLineHandler, GlobalArguments}; /// The command to leave a new subnet. pub struct LeaveSubnet; @@ -23,7 +22,7 @@ impl CommandLineHandler for LeaveSubnet { let mut provider = get_ipc_provider(global)?; let subnet = SubnetID::from_str(&arguments.subnet)?; let from = match &arguments.from { - Some(address) => Some(Address::from_str(address)?), + Some(address) => Some(require_fil_addr_from_str(address)?), None => None, }; provider.leave_subnet(subnet, from).await @@ -52,7 +51,7 @@ impl CommandLineHandler for Claim { let mut provider = get_ipc_provider(global)?; let subnet = SubnetID::from_str(&arguments.subnet)?; let from = match &arguments.from { - Some(address) => Some(Address::from_str(address)?), + Some(address) => Some(require_fil_addr_from_str(address)?), None => None, }; if !&arguments.rewards { diff --git a/ipc/cli/src/commands/subnet/mod.rs b/ipc/cli/src/commands/subnet/mod.rs index 3d197a2e..2fba3195 100644 --- a/ipc/cli/src/commands/subnet/mod.rs +++ b/ipc/cli/src/commands/subnet/mod.rs @@ -11,9 +11,12 @@ use crate::commands::subnet::send_value::{SendValue, SendValueArgs}; use crate::{CommandLineHandler, GlobalArguments}; use clap::{Args, Subcommand}; +use self::bootstrap::{AddBootstrap, AddBootstrapArgs, ListBootstraps, ListBootstrapsArgs}; use self::join::{StakeSubnet, StakeSubnetArgs}; use self::leave::{Claim, ClaimArgs}; +use self::rpc::{ChainIdSubnet, ChainIdSubnetArgs}; +pub mod bootstrap; pub mod create; pub mod join; pub mod kill; @@ -40,11 +43,14 @@ impl SubnetCommandsArgs { Commands::List(args) => ListSubnets::handle(global, args).await, Commands::Join(args) => JoinSubnet::handle(global, args).await, Commands::Rpc(args) => RPCSubnet::handle(global, args).await, + Commands::ChainId(args) => ChainIdSubnet::handle(global, args).await, Commands::Leave(args) => LeaveSubnet::handle(global, args).await, Commands::Kill(args) => KillSubnet::handle(global, args).await, Commands::SendValue(args) => SendValue::handle(global, args).await, Commands::Stake(args) => StakeSubnet::handle(global, args).await, Commands::Claim(args) => Claim::handle(global, args).await, + Commands::AddBootstrap(args) => AddBootstrap::handle(global, args).await, + Commands::ListBootstraps(args) => ListBootstraps::handle(global, args).await, } } } @@ -55,9 +61,12 @@ pub(crate) enum Commands { List(ListSubnetsArgs), Join(JoinSubnetArgs), Rpc(RPCSubnetArgs), + ChainId(ChainIdSubnetArgs), Leave(LeaveSubnetArgs), Kill(KillSubnetArgs), SendValue(SendValueArgs), Stake(StakeSubnetArgs), Claim(ClaimArgs), + AddBootstrap(AddBootstrapArgs), + ListBootstraps(ListBootstrapsArgs), } diff --git a/ipc/cli/src/commands/subnet/rpc.rs b/ipc/cli/src/commands/subnet/rpc.rs index b6bbc25f..01f1111c 100644 --- a/ipc/cli/src/commands/subnet/rpc.rs +++ b/ipc/cli/src/commands/subnet/rpc.rs @@ -36,8 +36,35 @@ impl CommandLineHandler for RPCSubnet { #[derive(Debug, Args)] #[command(name = "rpc", about = "RPC endpoint for a subnet")] pub struct RPCSubnetArgs { - #[arg(long, short, help = "The JSON RPC server url for ipc agent")] - pub ipc_agent_url: Option, - #[arg(long, short, help = "The subnet to get the RPC from")] + #[arg(long, short, help = "The subnet to get the ChainId from")] + pub subnet: String, +} + +/// The command to get the chain ID for a subnet +pub struct ChainIdSubnet; + +#[async_trait] +impl CommandLineHandler for ChainIdSubnet { + type Arguments = ChainIdSubnetArgs; + + async fn handle(global: &GlobalArguments, arguments: &Self::Arguments) -> anyhow::Result<()> { + log::debug!("get chain-id for subnet with args: {:?}", arguments); + + let provider = get_ipc_provider(global)?; + let subnet = SubnetID::from_str(&arguments.subnet)?; + let conn = match provider.connection(&subnet) { + None => return Err(anyhow::anyhow!("target subnet not found")), + Some(conn) => conn, + }; + + println!("{:}", conn.manager().get_chain_id().await?); + Ok(()) + } +} + +#[derive(Debug, Args)] +#[command(name = "chain-id", about = "Chain ID endpoint for a subnet")] +pub struct ChainIdSubnetArgs { + #[arg(long, short, help = "The subnet to get the Chain ID from")] pub subnet: String, } diff --git a/ipc/cli/src/commands/subnet/send_value.rs b/ipc/cli/src/commands/subnet/send_value.rs index fa0869b1..bc0859c4 100644 --- a/ipc/cli/src/commands/subnet/send_value.rs +++ b/ipc/cli/src/commands/subnet/send_value.rs @@ -4,7 +4,6 @@ use async_trait::async_trait; use clap::Args; -use fvm_shared::address::Address; use ipc_sdk::subnet_id::SubnetID; use std::{fmt::Debug, str::FromStr}; @@ -25,7 +24,7 @@ impl CommandLineHandler for SendValue { let mut provider = get_ipc_provider(global)?; let subnet = SubnetID::from_str(&arguments.subnet)?; let from = match &arguments.from { - Some(address) => Some(Address::from_str(address)?), + Some(address) => Some(require_fil_addr_from_str(address)?), None => None, }; diff --git a/ipc/cli/src/commands/wallet/export.rs b/ipc/cli/src/commands/wallet/export.rs index 66a1f50c..d6bf20ca 100644 --- a/ipc/cli/src/commands/wallet/export.rs +++ b/ipc/cli/src/commands/wallet/export.rs @@ -27,6 +27,10 @@ impl WalletExport { .get(&address.into())? .ok_or_else(|| anyhow!("key does not exists"))?; + if arguments.fendermint { + return Ok(BASE64_STANDARD.encode(key_info.private_key())); + } + let info = PersistentKeyInfo::new( format!("{:?}", address), hex::encode(key_info.private_key()), @@ -39,6 +43,9 @@ impl WalletExport { let addr = Address::from_str(&arguments.address)?; let key_info = wallet.write().unwrap().export(&addr)?; + if arguments.fendermint { + return Ok(BASE64_STANDARD.encode(key_info.private_key())); + } Ok(serde_json::to_string(&LotusJsonKeyType { r#type: WalletKeyType::try_from(*key_info.key_type())?.to_string(), private_key: BASE64_STANDARD.encode(key_info.private_key()), @@ -71,8 +78,7 @@ impl CommandLineHandler for WalletExport { ); } None => { - println!("exported new wallet with address {:?}", arguments.address); - println!("Key: {:?}", v); + println!("{}", v); } } @@ -93,6 +99,12 @@ pub(crate) struct WalletExportArgs { pub output: Option, #[arg(long, short, help = "The type of the wallet, i.e. fvm, evm")] pub wallet_type: String, + #[arg( + long, + short, + help = "Only returns the secret key in base64 as Fendermint expects" + )] + pub fendermint: bool, } pub(crate) struct WalletPublicKey; diff --git a/ipc/identity/src/fvm/keystore.rs b/ipc/identity/src/fvm/keystore.rs index 254e2dea..f919f5dc 100644 --- a/ipc/identity/src/fvm/keystore.rs +++ b/ipc/identity/src/fvm/keystore.rs @@ -16,7 +16,7 @@ use argon2::{ }; use base64::{prelude::BASE64_STANDARD, Engine}; use fvm_shared::crypto::signature::SignatureType; -use log::{error, warn}; +use log::{debug, error}; use rand::{rngs::OsRng, RngCore}; use serde::{Deserialize, Serialize}; use thiserror::Error; @@ -221,7 +221,7 @@ impl KeyStore { } Err(e) => { if e.kind() == ErrorKind::NotFound { - warn!( + debug!( "Keystore does not exist, initializing new keystore at: {:?}", file_path ); @@ -255,7 +255,7 @@ impl KeyStore { if read_bytes == 0 { // New encrypted keystore if file exists but is zero bytes (i.e., touch) - warn!( + debug!( "Keystore does not exist, initializing new keystore at {:?}", file_path ); @@ -309,7 +309,7 @@ impl KeyStore { } } Err(_) => { - warn!("Encrypted keystore does not exist, initializing new keystore"); + debug!("Encrypted keystore does not exist, initializing new keystore"); let (salt, encryption_key) = EncryptedKeyStore::derive_key(&passphrase, None).map_err(|error| { diff --git a/ipc/provider/src/lib.rs b/ipc/provider/src/lib.rs index b4fd777f..7496c6c4 100644 --- a/ipc/provider/src/lib.rs +++ b/ipc/provider/src/lib.rs @@ -624,6 +624,38 @@ impl IpcProvider { conn.manager().chain_head_height().await } + + /// Advertises the endpoint of a bootstrap node for the subnet. + pub async fn add_bootstrap( + &mut self, + subnet: &SubnetID, + from: Option
, + endpoint: String, + ) -> anyhow::Result<()> { + let parent = subnet.parent().ok_or_else(|| anyhow!("no parent found"))?; + let conn = match self.connection(&parent) { + None => return Err(anyhow!("target parent subnet not found")), + Some(conn) => conn, + }; + + let subnet_config = conn.subnet(); + let sender = self.check_sender(subnet_config, from)?; + + conn.manager() + .add_bootstrap(subnet, &sender, endpoint) + .await + } + + /// Lists the bootstrap nodes of a subnet + pub async fn list_bootstrap_nodes(&self, subnet: &SubnetID) -> anyhow::Result> { + let parent = subnet.parent().ok_or_else(|| anyhow!("no parent found"))?; + let conn = match self.connection(&parent) { + None => return Err(anyhow!("target parent subnet not found")), + Some(conn) => conn, + }; + + conn.manager().list_bootstrap_nodes(subnet).await + } } /// Lotus JSON keytype format @@ -747,7 +779,7 @@ pub fn new_evm_keystore_from_path( pub fn new_fvm_keystore_from_path(repo_str: &str) -> anyhow::Result { let repo = Path::new(&repo_str); let repo = expand_tilde(repo); - let keystore_config = KeyStoreConfig::Persistent(repo.join(ipc_identity::KEYSTORE_NAME)); + let keystore_config = KeyStoreConfig::Persistent(repo); // TODO: we currently only support persistent keystore in the default repo directory. KeyStore::new(keystore_config).map_err(|e| anyhow!("Failed to create keystore: {}", e)) } diff --git a/ipc/provider/src/manager/evm/manager.rs b/ipc/provider/src/manager/evm/manager.rs index 86bfaca4..a120425c 100644 --- a/ipc/provider/src/manager/evm/manager.rs +++ b/ipc/provider/src/manager/evm/manager.rs @@ -12,6 +12,8 @@ use ipc_actors_abis::{ }; use ipc_sdk::evm::{fil_to_eth_amount, payload_to_evm_address, subnet_id_to_evm_addresses}; use ipc_sdk::validator::from_contract_validators; +use std::net::{IpAddr, SocketAddr}; + use ipc_sdk::{eth_to_fil_amount, ethers_address_to_fil_address}; use crate::config::subnet::SubnetConfig; @@ -630,6 +632,40 @@ impl SubnetManager for EthSubnetManager { validators: from_contract_validators(contract.genesis_validators().call().await?)?, }) } + + async fn add_bootstrap( + &self, + subnet: &SubnetID, + from: &Address, + endpoint: String, + ) -> Result<()> { + let address = contract_address_from_subnet(subnet)?; + + if is_valid_bootstrap_addr(&endpoint).is_none() { + return Err(anyhow!("wrong format for bootstrap endpoint")); + } + + let signer = Arc::new(self.get_signer(from)?); + let contract = + subnet_actor_manager_facet::SubnetActorManagerFacet::new(address, signer.clone()); + + call_with_premium_estimation(signer, contract.add_bootstrap_node(endpoint)) + .await? + .send() + .await? + .await?; + + Ok(()) + } + + async fn list_bootstrap_nodes(&self, subnet: &SubnetID) -> Result> { + let address = contract_address_from_subnet(subnet)?; + let contract = subnet_actor_getter_facet::SubnetActorGetterFacet::new( + address, + Arc::new(self.ipc_contract_info.provider.clone()), + ); + Ok(contract.get_bootstrap_nodes().call().await?) + } } #[async_trait] @@ -1085,6 +1121,21 @@ fn block_number_from_receipt( } } +fn is_valid_bootstrap_addr(input: &str) -> Option<(String, IpAddr, u16)> { + let parts: Vec<&str> = input.split('@').collect(); + + if parts.len() == 2 { + let pubkey = parts[0].to_string(); + let addr_str = parts[1]; + + if let Ok(addr) = addr_str.parse::() { + return Some((pubkey, addr.ip(), addr.port())); + } + } + + None +} + /// Convert the ipc SubnetID type to an evm address. It extracts the last address from the Subnet id /// children and turns it into evm address. pub(crate) fn contract_address_from_subnet(subnet: &SubnetID) -> Result { diff --git a/ipc/provider/src/manager/subnet.rs b/ipc/provider/src/manager/subnet.rs index 40ccf857..b5f6c6ae 100644 --- a/ipc/provider/src/manager/subnet.rs +++ b/ipc/provider/src/manager/subnet.rs @@ -116,6 +116,17 @@ pub trait SubnetManager: Send + Sync + TopDownCheckpointQuery + BottomUpCheckpoi /// Gets the genesis information required to bootstrap a child subnet async fn get_genesis_info(&self, subnet: &SubnetID) -> Result; + + /// Advertises the endpoint of a bootstrap node for the subnet. + async fn add_bootstrap( + &self, + subnet: &SubnetID, + from: &Address, + endpoint: String, + ) -> Result<()>; + + /// Lists the bootstrap nodes of a subnet + async fn list_bootstrap_nodes(&self, subnet: &SubnetID) -> Result>; } #[derive(Debug)] diff --git a/scripts/install_infra.sh b/scripts/install_infra.sh index fe32e4e3..15799585 100755 --- a/scripts/install_infra.sh +++ b/scripts/install_infra.sh @@ -9,10 +9,21 @@ PWD=$(pwd) infra_path="$PWD/bin/ipc-infra" git_repo_url="https://github.com/consensus-shipyard/fendermint.git" +if ! command -v cargo-make &> /dev/null +then + echo "[*] 'cargo make' not found. Installing..." + cargo install cargo-make +else + echo "[*] 'cargo make' is already installed." +fi + build_infra() { echo "[*] Building fendermint..." make build docker-build cd $PWD + + echo "[*] Updating infra scripts..." + cp -r $infra_path/fendermint/infra/* $infra_path } # Function to display help message @@ -83,4 +94,3 @@ if [ -n "$(git status --porcelain)" ]; then else echo "[*] No changes detected in the repository. Doing nothing!" fi -