Skip to content

Commit

Permalink
Merge pull request #724 from autonomys/incorporate_vexrs_cli_tips_back
Browse files Browse the repository at this point in the history
Incorporate vexrs cli tips suggestions
  • Loading branch information
EmilFattakhov authored Oct 30, 2024
2 parents 94be058 + 7b6c7f1 commit 6b5d24a
Show file tree
Hide file tree
Showing 3 changed files with 95 additions and 130 deletions.
178 changes: 94 additions & 84 deletions docs/farming-&-staking/farming/advanced-cli/cli-tips.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,8 @@ keywords:
- Docker
---

import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';

## Additional Tips

Expand All @@ -28,26 +30,100 @@ Explore the Autonomys Network in depth with our variety of telemetry tools and b

- **[Polkadot.js Block Explorer](https://polkadot.js.org/apps/?rpc=wss%3A%2F%2Frpc.taurus.subspace.foundation%2Fws#/explorer)**: For users familiar with the Polkadot ecosystem, this explorer offers a seamless experience for exploring the Autonomys Network using the Polkadot.js interface.

### Switching to a new snapshot from older/different versions of Autonomys

:::info
Unless specifically mentioned by the Development team you should **NOT** have to wipe your configuration on new releases.
:::

### Snap Sync
In general you should be able to download the latest release, and re-start the Node & Farmer with the same commands as you started to prior version with no errors.

Snap Sync works by jumping to near the end of the chain, allowing you to sync to the current block more quickly. Once the initial jump is complete, it behaves like a full sync. In fact, if you haven't synced for more than a few days, it might be quicker to remove the node database and let Snap Sync quickly sync to the most current block.
There are some cases where version updates will cause issue with your Node & Farmer and you may have to wipe your node, typically when errors occur. If you have any issues you can always check our [Forums](https://forum.autonomys.xyz) and hop in our [Discord](https://autonomys.xyz/discord) Server to ask for help.

There are cases where you might not want to use Snap Sync. To perform a full sync you can either omit the sync parameter, or use `--sync full`. This is useful if you want to run an RPC node for public use, become an operator or indexing. In that case, you would need to run an archival node, not just a pruned node. An archival node requires a full sync.
### Wipe

If you were running a node previously, and want to switch to a new network, please perform these steps and then follow the guide again:
```bash
# Replace `FARMER_FILE_NAME` with the name of the farmer file you downloaded from releases
./FARMER_FILE_NAME wipe PATH_TO_FARM
# Replace `NODE_FILE_NAME` with the name of the node file you downloaded from releases
./NODE_FILE_NAME wipe NODE_DATA_PATH
```

Now follow the installation guide from the beginning.

### Utilizing Multiple Disks

To maximize storage capabilities, you can engage multiple disks directly. This is often more efficient than relying on RAID configurations:

Example:
```shell-session
./FARMER_FILE_NAME farm --reward-address WALLET_ADDRESS \
path=/media/ssd1,size=100GiB \
path=/media/ssd2,size=10T \
path=/media/ssd3,size=10T
```
### Optimizing DSN Syncing

:::note

The DSN can be a nuanced topic, to better understand our Decentralized Storage Network, please refer to [this guide](https://academy.autonomys.xyz/subspace-protocol/network-architecture/distributed-storage-network) from our Academy.

:::

:::warning Parameters to Use with Caution
While it might be tempting to adjust certain parameters that seem related to DSN Syncing, it's advised to be cautious. Some options will not enhance syncing and are best left at their default values:

```
--out-peers
--in-peers
--dsn-target-connections
--dsn-pending-in-connections
--dsn-in-connections
```
:::

#### Recommended Parameters

The default parameters are set with the capabilities of common consumer modem/routers in mind. Adjusting certain parameters could enhance DSN sync performance by increasing parallelism. However, if you decide to increase them significantly, ensure that your modem/router is performant enough to handle the increased traffic.
**Node:**
```
--dsn-out-connections
--dsn-pending-out-connections
```

**Farmer:**
Increasing the values of the farmer parameters could increase the plotting speed.
```
--out-connections
--pending-out-connections
```


### Help

Both the node and the farmer have a variety of flags and parameters. To see a full list, append the `--help` flag to either the node or the farmer command.


## Some Additional Tips for Node, Farmer and Docker

<Tabs groupId="component">

<TabItem value="farmer" label="Farmer">

### Plotting concurrency (CPU only)

During plotting, there are both parallel and sequential parts of the table generation. By generating several tables simultaneously, we can overlap the sequential parts with parallel parts, thus improving CPU utilization. While generating tables for all records requires significant RAM, producing tables for only a few records at a time offers an optimal balance between CPU and RAM usage.
We added the `--cpu-record-encoding-concurrency` option to override the default behavior, which allocates one record for every two cores but does not exceed eight in parallel. According to our internal testing with P-cores, E-cores, and combinations of P+E cores, this setting appears to achieve peak performance.
If you prefer to use the previous behavior, or if your RAM usage becomes too high, you can set `--cpu-record-encoding-concurrency` to `1`. You may also experiment with setting it to `2`, `3`, etc., which may yield better results depending on your specific CPU/RAM configuration.


### Create Missing Farms

A farmer has the option to specify whether the system should automatically create missing farms upon startup. If you provide a path to a plot that isn't found, the system will generate a new one. However, this may not be desirable if a drive fails to mount properly.
By default, this option is set to `true`, but you can override it by adding `--create false`. Setting this flag to `false` after establishing your plots can prevent unintentional file writes to the wrong drive.


### Record Chunks Reading Mode

Upon startup, each farm will [benchmark](#benchmarking-your-farmer) the performance of every plot to identify the most efficient proving method, yielding either `ConcurrentChunks` or `WholeSector` results. If you already know the optimal method for your setup, you can specify it as an argument for each farm to save time benchmarking.
Expand Down Expand Up @@ -75,7 +151,7 @@ There is a second command available, which helps you determine how much time you
```

Each farmer has ~4 seconds to audit and prove, so depending on how fast auditing is, the remaining time will be used for proving.
Proving performance doesnt depend on the plot size.
Proving performance doesn't depend on the plot size.

If an environment does not have enough time for the proving step, a message such as this is logged by the farmer:

Expand All @@ -92,6 +168,7 @@ In certain situations, especially when the farmer terminates unexpectedly or enc

Ensure to replace /path/to/your/plot with your actual plot path. Use this command cautiously as it modifies the plot state to recover from errors.


### Specify Exact CPU Cores for Plotting/Replotting (CPU only)
This option will override any custom logic that the **subspace farmer** might otherwise use.
You can specify the plotting CPU cores by adding `--cpu-plotting-cores`, followed by the desired cores parameters. Cores should be listed as comma-separated values, with whitespace used to separate different thread pools or encoding instances.
Expand All @@ -101,28 +178,19 @@ Similarly, you can customize the replotting CPU cores using `--cpu-replotting-co

Note that setting `--cpu-plotting-cores` requires `--cpu-replotting-cores` to be configured with the same number of CPU core groups. If the `--cpu-replotting-cores` setting is omitted, the node will default to using the same thread pool as for plotting.

</TabItem>

### Switching to a new snapshot from older/different versions of Autonomys

:::info
Unless specifically mentioned by the Development team you should **NOT** have to wipe your configuration on new releases.
:::
<TabItem value="node" label="Node">

In general you should be able to download the latest release, and re-start the Node & Farmer with the same commands as you started to prior version with no errors.
### Snap Sync

There are some cases where version updates will cause issue with your Node & Farmer and you may have to wipe your node, typically when errors occur. If you have any issues you can always check our [Forums](https://forum.autonomys.xyz) and hop in our [Discord](https://autonomys.xyz/discord) Server to ask for help.
Snap Sync works by jumping to near the end of the chain, allowing you to sync to the current block more quickly. Once the initial jump is complete, it behaves like a full sync. In fact, if you haven't synced for more than a few days, it might be quicker to remove the node database and let Snap Sync quickly sync to the most current block.

### Wipe
There are cases where you might not want to use Snap Sync. To perform a full sync you can either omit the sync parameter, or use `--sync full`. This is useful if you want to run an RPC node for public use, become an operator or indexing. In that case, you would need to run an archival node, not just a pruned node. An archival node requires a full sync.

If you were running a node previously, and want to switch to a new network, please perform these steps and then follow the guide again:
```bash
# Replace `FARMER_FILE_NAME` with the name of the farmer file you downloaded from releases
./FARMER_FILE_NAME wipe PATH_TO_FARM
# Replace `NODE_FILE_NAME` with the name of the node file you downloaded from releases
./NODE_FILE_NAME wipe NODE_DATA_PATH
```
</TabItem>

Now follow the installation guide from the beginning.
<TabItem value="docker" label="Docker">

### Docker Wipe

Expand All @@ -135,89 +203,31 @@ If you are having issues with running a node or farmer for Autonomys, feel free
- [Forum](https://forum.autonomys.xyz)
- [Discord](https://autonomys.xyz/discord)

### Help
</TabItem>
</Tabs>

There are extra commands and parameters you can use on farmer or node, use the `--help` after any other command to display additional options.
---

```bash
# Replace `FARMER_FILE_NAME` with the name of the node file you downloaded from releases
./FARMER_FILE_NAME farm --help
```

---
### Timekeepers
## Timekeepers

Gemini-3g introduces Proof-of-Time and a new, optional role has been added to the node. Timekeepers help run PoT to ensure the security of the network. Timekeeping requires a high-performance core CPU but can be undertaken by any node runner. You can enable timekeeping with the following commands.
- `--timekeeper`: To enable timekeeping on the node
- `--timekeeper-cpu-cores`: To specify which cores the Timekeeper will run on.

Read more about [Timekeepers](/farming/timekeeper)

### Node & Farmer Commands Guide

Both the node and the farmer have a variety of flags and parameters. To see a full list, append the `--help` flag to either the node or the farmer command.
---

### Common Command Examples

:::note Farming Changes
Please note that as of `Gemini-3g` farming no longer occurs while plotting. This is to ensure the plotting process occurs in the most efficient manner. You can change this by adding the `--farm-during-initial-plotting` flag to the farmer.
:::

For both the node and the farmer, here are some frequently used commands:

- Display farm information: `./FARMER_FILE_NAME info PATH_TO_FARM`
- Scrub the farm for errors: `./FARMER_FILE_NAME scrub PATH_TO_FARM`
- Erase all farmer-related data: `./FARMER_FILE_NAME wipe PATH_TO_FARM`
- Erase all node-related data: `./NODE_FILE_NAME wipe NODE_DATA_PATH`

### Utilizing Multiple Disks

To maximize storage capabilities, you can engage multiple disks directly. This is often more efficient than relying on RAID configurations:

Example:
```shell-session
./FARMER_FILE_NAME farm --reward-address WALLET_ADDRESS \
path=/media/ssd1,size=100GiB \
path=/media/ssd2,size=10T \
path=/media/ssd3,size=10T
```

### Optimizing DSN Syncing

:::note

The DSN can be a nuanced topic, to better understand our Decentralized Storage Network, please refer to [this guide](https://academy.autonomys.xyz/subspace-protocol/network-architecture/distributed-storage-network) from our Academy.

:::

:::warning Parameters to Use with Caution
While it might be tempting to adjust certain parameters that seem related to DSN Syncing, it's advised to be cautious. Some options will not enhance syncing and are best left at their default values:

```
--out-peers
--in-peers
--dsn-target-connections
--dsn-pending-in-connections
--dsn-in-connections
```
:::

#### Recommended Parameters

The default parameters are set with the capabilities of common consumer modem/routers in mind. Adjusting certain parameters could enhance DSN sync performance by increasing parallelism. However, if you decide to increase them significantly, ensure that your modem/router is performant enough to handle the increased traffic.
**Node:**
```
--dsn-out-connections
--dsn-pending-out-connections
```

**Farmer:**
Increasing the values of the farmer parameters could increase the plotting speed.
```
--out-connections
--pending-out-connections
```

---

### NUMA Support

Expand Down

This file was deleted.

3 changes: 1 addition & 2 deletions docusaurus.config.js
Original file line number Diff line number Diff line change
Expand Up @@ -70,8 +70,7 @@ const config = {
{ to: '/farming/space-acres/translate', from: ['/docs/farming-&-staking/farming/space-acres/translate_space_acres'] },
{ to: '/farming/advanced-cli/install', from: ['/docs/farming-&amp;-staking/farming/advanced-cli/cli-install', '/docs/protocol/cli/', '/docs/protocol/substrate-cli/', '/docs/Farming%20&%20Staking/Farming/Advanced-Cli/cli-install', '/docs/category/advanced-cli-recommended', '/docs/category/advanced-cli', '/docs/farming-&-staking/farming/advanced-cli/cli-install'] },
{ to: '/farming/advanced-cli/cluster', from: ['/docs/farming-&-staking/farming/advanced-cli/cli-farming-cluster'] },
{ to: '/farming/advanced-cli/tips', from: ['/docs/farming-&-staking/farming/advanced-cli/cli-tips'] },
{ to: '/farming/advanced-cli/troubleshooting', from: ['/docs/farming-&-staking/farming/advanced-cli/cli-troubleshooting'] },
{ to: '/farming/advanced-cli/tips', from: ['/docs/farming-&-staking/farming/advanced-cli/cli-tips', '/docs/farming-&-staking/farming/advanced-cli/cli-troubleshooting', '/farming/advanced-cli/troubleshooting'] },
{ to: '/farming/common-problems', from: ['/docs/farming-&-staking/farming/common_problems'] },
{ to: '/farming/guides/gpu-plotter', from: ['/docs/farming-&-staking/farming/additional-guides/gpu-plotter'] },
{ to: '/farming/guides/port-config', from: ['/docs/farming-&-staking/farming/additional-guides/port-config'] },
Expand Down

0 comments on commit 6b5d24a

Please sign in to comment.