Gemini 3h guide updates #401

EmilFattakhov · 2024-01-16T22:22:25Z

This PR reflects the changes introduced in:

This will require some dogfooding and testing before mering, especially on the operators side (key generation, starting an operator node, starting an operator+farmer node).

For now, requesting a review for any obvious missed flags or inconsistencies. Thanks!

Edit:

Also reflected the PR 2452

netlify · 2024-01-16T22:22:30Z

✅ Deploy Preview for subspace-docs-staging ready!

Name	Link
🔨 Latest commit	`674dd34`
🔍 Latest deploy log	https://app.netlify.com/sites/subspace-docs-staging/deploys/65ba5e7f6af49b000846a015
😎 Deploy Preview	https://deploy-preview-401--subspace-docs-staging.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

nazar-pc

Looked through everything briefly and left a few comments

nazar-pc · 2024-01-20T14:23:30Z

docs/farming-&-staking/farming/advanced-cli/cli-prerequisites.mdx

@@ -80,3 +80,15 @@ You can use any file system with default settings, we advise against their use f
 In order to disable **CoW** on **BTRFS**, run the command `chattr +C /your_farmer_directory/`.

 In case of **ZFS**, you can try adjusting the **recordsize** by running `zfs set recordsize=128K tank/dataset`, or creating a separate **dataset** for the farmer with adjusted **recordsize** parameters.
+
+
+### Folder structure


How important is this information to users? They can see the structure when they run software anyway.

Agree. Unless we add some explanation of what the folders are for this is pretty much self-documenting.

It's important in the context of keypair location for operators, but I agree that they look a bit out-of-context here. Removed.

It is not important in the context of keypair location because farmers will not need to know or care about keypair location anymore. There is no --keypair-path anywhere now.

nazar-pc · 2024-01-20T14:25:52Z

docs/farming-&-staking/staking/operators/register-operator.mdx

+--bootstrap-nodes /ip4/3.87.28.170/tcp/40333/p2p/12D3KooWGHtULvhdKMZtzigSK1438uWXPj9rBQHVzTaKMWv1WRXp \`
+--bootstrap-nodes /ip4/140.82.45.36/tcp/40333/p2p/12D3KooWJxB2PVzyNqZywoCWAMLGhKr2LCbFMXbAPzUPQZon7yt1 \`
+--bootstrap-nodes /ip4/167.179.116.135/tcp/40333/p2p/12D3KooWHsRae3ip2uQAUPtpMcStHGGw777SJT3nu9maFmtsGry7 \`


We really need to replace this mess of IP addresses with readable domain names for 3h. Also note that these can be in the chain spec and we should add them for 3h such that for domains created by us specifying bootstrap nodes will not be required, but have "advanced" section about adding these for custom domains (bootstrap nodes for which we will probably not run at all).

Who can help us with that? Is it @DaMandal0rian?

Yes, he should create it on infrastructure level and someone needs to update chain spec afterwards.

@jim-counter these will be in the format:

"--bootstrap-nodes", "/dns/bootstrap-0.nova.gemini-3h.subspace.network/tcp/30334/p2p/ID"
"--bootstrap-nodes", "/dns/bootstrap-1.nova.gemini-3h.subspace.network/tcp/30334/p2p/ID"
"--bootstrap-nodes", "/dns/bootstrap-2.nova.gemini-3h.subspace.network/tcp/30334/p2p/ID"

Thank you very much Muhammad!

nazar-pc · 2024-01-20T14:27:13Z

docs/farming-&-staking/staking/operators/register-operator.mdx

      "--base-path", "/var/subspace",
-      "--port", "30333",
+      "--listen-on", "30333",


Here and in multiple other places syntax is wrong, it is not just port anymore

Yes, the --port has been replaced by --listen-on. Perhaps some github quirk, just double checked that all --port commands were indeed replaced.

I was talking about value, not the key name. The value is wrong.

nazar-pc · 2024-01-20T14:27:41Z

docs/farming-&-staking/staking/operators/tips-operator.mdx

      "--dsn-listen-on", "/ip4/0.0.0.0/udp/30433/quic-v1",
      "--dsn-listen-on", "/ip4/0.0.0.0/tcp/30433",
      "--rpc-cors", "all",
      "--rpc-methods", "unsafe",
-      "--rpc-external",
+      "--rpc-listen-on",


Here is and in many other places syntax is wrong, this is not a boolean option

Sorry, what we should replace --rpc-external with in this case? I thought it could be used with or without specifying the port, as it has a default value unless specified explicitly.

Definitely not. --rpc-listen-on replaces a bunch of options, it is not a simple renaming. --help explains the syntax of each option.

--rpc-listen-on is not a port, it is IP:port. If you want to listen on all IPs then you specify 0.0.0.0 as IP explicitly. As to port - you specify the port you want, probably 9944, so the value is 0.0.0.0:9944 here. In future we'll make RPC optional (Substrate currently always starts it unconditionally) and will not run at all unless this option is specified.

jfrank-summit · 2024-01-22T22:00:41Z

docs/farming-&-staking/farming/advanced-cli/platforms/_linux.mdx

-    --chain gemini-3g \
+  ./subspace-node-ubuntu-x86_64-skylake-gemini-3g-2024-jan-08 run \
+    --base-path path_to_node \
+    --chain gemini-3h \
    --blocks-pruning 256 \
    --state-pruning archive-canonical \
    --no-private-ipv4 \


I believe this option has been removed or changed as it no longer works as a CLI parameter.

cc @nazar-pc

jfrank-summit · 2024-01-23T19:41:29Z

docs/farming-&-staking/staking/operators/tips-operator.mdx

@@ -18,7 +18,7 @@ An operator needs a key pair to participate in bundle production.
 You can create a key using the following command:

 ```bash
-target/production/subspace-node key generate --scheme sr25519
+target/production/subspace-node run key generate --scheme sr25519


@nazar-pc should this work? When I try to ./subspace-node run key ... I get an error: unexpected argument 'key' found.

No, run x doesn't make sense, that command is for running exclusively. All key-related commands are gone except importing, but now I see that we need to bring generation back. I'll start a Slack thread about this to figure out what the interface should look like.

EmilFattakhov · 2024-01-23T20:07:53Z

I will take this evening to test the commands with the 3h build and push the update either later today or tomorrow. Appreciate your feedback everyone!

jfrank-summit · 2024-01-23T21:35:00Z

docs/develop/nova/local_development.md

@@ -33,7 +33,7 @@ After downloading both files that suit your system, start a node using your pref
 You can do this with the following command:

 ```bash
-./your_subspace_node_path --dev --alice --rpc-port 9444 -- --domain-id 3 --dev --rpc-port 8545
+./your_subspace_node_path run --dev --rpc-listen-on 9444 -- --domain-id 3 --dev --rpc-listen-on 8545


The format for --rpc-listen-on is 127.0.0.1:9944, not just the port.

jim-counter

Suggested some changes. I have not made them in version-latest but I expect they need to be there too.

docs/farming-&-staking/farming/advanced-cli/cli-tips.mdx

jim-counter · 2024-01-26T17:13:34Z

docs/farming-&-staking/farming/advanced-cli/cli-tips.mdx


 If you were running a node previously, and want to switch to a new network, please perform these steps and then follow the guide again:
 ```bash
 # Replace `FARMER_FILE_NAME` with the name of the farmer file you downloaded from releases
 ./FARMER_FILE_NAME wipe PATH_TO_FARM
 # Replace `NODE_FILE_NAME` with the name of the node file you downloaded from releases
-./NODE_FILE_NAME purge-chain --chain gemini-3g
+./NODE_FILE_NAME wipe PATH_TO_NODE


Is PATH_TO_NODE the value we use for --base-path?

Yes, it is the same value, just positional argument in this case rather than named option

It's named inconsistently in different places. And PATH_TO_NODE_DATA or even NODE_DATA_PATH feels like a better explanation.

Just checked and we use the latter elsewhere. I think we should standardise on that.

jim-counter · 2024-01-26T17:15:21Z

docs/farming-&-staking/farming/advanced-cli/cli-tips.mdx

+Previously plotting/replotting thread pools were created for each farm separately even though only a configured number of them can be used at a time (by default just one).
+With the introduction of NUMA support, the farmer application has a thread pool manager that will create a necessary number of thread pools that will be allocated to currently plotting/replotting farms. 
+When a thread pool is created, it is assigned to a set of CPU cores and will only be able to use those cores. Pinning doesn’t pin threads to cores 1:1, instead, OS is free to move threads between cores, but only within CPU cores allocated for thread pool. This will ensure plotting for a particular sector only happens on a particular CPU/NUMA node.


Suggested change

When a thread pool is created, it is assigned to a set of CPU cores and will only be able to use those cores. Pinning doesn’t pin threads to cores 1:1, instead, OS is free to move threads between cores, but only within CPU cores allocated for thread pool. This will ensure plotting for a particular sector only happens on a particular CPU/NUMA node.

When a thread pool is created, it is assigned to a set of CPU cores and will only be able to use those cores. Pinning doesn’t pin threads to cores 1:1, instead, the OS is free to move threads between cores, but only within CPU cores allocated for the thread pool. This will ensure plotting for a particular sector only happens on a particular CPU/NUMA node.

jim-counter · 2024-01-26T17:15:55Z

docs/farming-&-staking/farming/advanced-cli/cli-tips.mdx

+#### Enabling NUMA on Windows/Linux machines
+
+On Linux and Windows, the farmer will detect NUMA systems and create a number of thread pools that correspond to a number of NUMA nodes. This means the default behavior will change for large CPUs and consume more memory as a result, but that can be changed to the previous behavior with CLI options if desired.


Suggested change

On Linux and Windows, the farmer will detect NUMA systems and create a number of thread pools that correspond to a number of NUMA nodes. This means the default behavior will change for large CPUs and consume more memory as a result, but that can be changed to the previous behavior with CLI options if desired.

On Linux and Windows, the farmer will detect NUMA systems and create a number of thread pools that correspond to the number of NUMA nodes. This means the default behavior will change for large CPUs and consume more memory as a result, but that can be changed to the previous behavior with CLI options if desired.

jim-counter · 2024-01-26T17:16:38Z

docs/farming-&-staking/farming/advanced-cli/cli-tips.mdx

+To use NUMA, you need to enable it via the BIOS of your motherboard for the farmer to know it exists. This option is present in motherboards for **Threadripper/Epyc processors** but might exist in others too. If you don’t enable it, both OS and farmer will think you have a single UMA processor and will not be able to apply optimizations.
+
+To read more about NUMA support and benefits it provides for large CPUs read [this forum post](https://forum.subspace.network/t/numa-support-is-coming/2299?u=nazar-pc).


Suggested change

To read more about NUMA support and benefits it provides for large CPUs read [this forum post](https://forum.subspace.network/t/numa-support-is-coming/2299?u=nazar-pc).

To read more about NUMA support and the benefits it provides for large CPUs read [this forum post](https://forum.subspace.network/t/numa-support-is-coming/2299?u=nazar-pc).

jim-counter · 2024-01-26T17:24:38Z

docs/farming-&-staking/staking/operators/register-operator.mdx


 **You have successfully generated an operator key, congratulations!**

+The keys were generated in the subfolder of **PATH_TO_NODE** under `/domains/domainID/keystore`. If you don't see the generated keypair in the subfolder, something went wrong. 


Should we include a warning about how important it is that users record the seed phrase output to the CLI?

Probably, though CLI will already print a warning on its own. Also you can always rotate the key, it is just important to not leak it or else someone can get operator slashed.

Added a short note below.

docs/farming-&-staking/staking/operators/tips-operator.mdx

jim-counter · 2024-01-26T17:26:41Z

docs/farming-&-staking/staking/operators/tips-operator.mdx


 #### Insert key to Keystore:
-The key generated above needs to be added to the Keystore so that the operator node can use it to participate in bundle production.
+This might be useful if you decicded to switch domains and already have **the secret phrase**. Read more about switching domain in the next section.


Suggested change

This might be useful if you decicded to switch domains and already have **the secret phrase**. Read more about switching domain in the next section.

This might be useful if you decided to switch domains or already have **the secret phrase**. Read more about switching domain in the next section.

I'd say "or already have" because keys can be generated externally.

I'd say "or already have" because keys can be generated externally.

Updated.

docs/farming-&-staking/staking/operators/tips-operator.mdx

docs/farming-&-staking/staking/staking.md

docs/farming-&-staking/farming/advanced-cli/cli-tips.mdx

nazar-pc · 2024-01-26T19:24:59Z

docs/farming-&-staking/farming/advanced-cli/cli-tips.mdx


 If you were running a node previously, and want to switch to a new network, please perform these steps and then follow the guide again:
 ```bash
 # Replace `FARMER_FILE_NAME` with the name of the farmer file you downloaded from releases
 ./FARMER_FILE_NAME wipe PATH_TO_FARM
 # Replace `NODE_FILE_NAME` with the name of the node file you downloaded from releases
-./NODE_FILE_NAME purge-chain --chain gemini-3g
+./NODE_FILE_NAME wipe PATH_TO_NODE


Yes, it is the same value, just positional argument in this case rather than named option

nazar-pc · 2024-01-26T19:26:51Z

docs/farming-&-staking/staking/operators/register-operator.mdx


 **You have successfully generated an operator key, congratulations!**

+The keys were generated in the subfolder of **PATH_TO_NODE** under `/domains/domainID/keystore`. If you don't see the generated keypair in the subfolder, something went wrong. 


Probably, though CLI will already print a warning on its own. Also you can always rotate the key, it is just important to not leak it or else someone can get operator slashed.

nazar-pc · 2024-01-26T19:28:02Z

docs/farming-&-staking/staking/operators/tips-operator.mdx


 #### Insert key to Keystore:
-The key generated above needs to be added to the Keystore so that the operator node can use it to participate in bundle production.
+This might be useful if you decicded to switch domains and already have **the secret phrase**. Read more about switching domain in the next section.


I'd say "or already have" because keys can be generated externally.

docs/farming-&-staking/staking/operators/tips-operator.mdx

EmilFattakhov · 2024-01-29T21:25:47Z

Everything should be updated now with all comments resolved. Thanks again for the careful reviews, everyone!

jim-counter

A few small items and there's an existing unresolved conversation.

docs/farming-&-staking/staking/intro.md

docs/farming-&-staking/staking/operators/tips-operator.mdx

versioned_docs/version-latest/farming-&-staking/farming/advanced-cli/cli-tips.mdx

EmilFattakhov · 2024-01-29T22:40:07Z

Thanks @jim-counter, hopefully a last ping from me 😅

jim-counter

LGTM!

EmilFattakhov added 4 commits January 16, 2024 13:30

Reflect PR 2408

27bd9ee

Reflect PR 2412

36d3b94

Add a block on NUMA support

0a21692

Reflect PR 2405

93e2bd3

EmilFattakhov added documentation Improvements or additions to documentation enhancement New feature or request do not merge and removed enhancement New feature or request labels Jan 16, 2024

EmilFattakhov added 6 commits January 16, 2024 17:24

Update local_development section to reflect PR 2405

4774711

Reflect PR 2414

2e55384

Reflect directory changes PR 2377

516e519

Ensure guides enforse setting --base-path and --chain flag

3612e2c

Update operators guide to reflect node / domain CLI changes

ccb2086

Improve message on NUMA support

027217d

EmilFattakhov marked this pull request as ready for review January 18, 2024 21:36

Reflect Subnomicon PR 16

2ac349a

EmilFattakhov changed the title ~~WIP Gemini 3h guide updates~~ Gemini 3h guide updates Jan 18, 2024

EmilFattakhov requested review from ImmaZoni and jim-counter January 19, 2024 20:00

nazar-pc reviewed Jan 20, 2024

View reviewed changes

Remove subspace folder structure description

4b7059b

jfrank-summit reviewed Jan 22, 2024

View reviewed changes

jfrank-summit reviewed Jan 23, 2024

View reviewed changes

EmilFattakhov added 2 commits January 25, 2024 10:21

Updates after feedback and testing

56028ea

Reflect PR 2452

d4647be

jim-counter requested changes Jan 26, 2024

View reviewed changes

nazar-pc reviewed Jan 26, 2024

View reviewed changes

EmilFattakhov added 2 commits January 29, 2024 13:50

Make further adjustments after the review

2a3d055

Merge up to date main

e18b827

EmilFattakhov requested a review from jim-counter January 29, 2024 21:26

jim-counter requested changes Jan 29, 2024

View reviewed changes

EmilFattakhov added 2 commits January 29, 2024 17:19

Resolve merge conflict, improve grammar and wording

94c6fb4

Add a few small clarifications

4e2020d

EmilFattakhov requested a review from jim-counter January 29, 2024 22:40

jim-counter approved these changes Jan 30, 2024

View reviewed changes

EmilFattakhov added 6 commits January 30, 2024 14:43

Adjust chain from 3g to 3h

90daf3c

Update stake withdraw instructions

1a5b412

Merge up-to-date master

356125a

adjust the chain to 3h

7cc0b64

Update Nova bootstrap nodes IPs

2f76c38

Adjust chain for docker commands

674dd34

EmilFattakhov merged commit 5e4ed11 into main Jan 31, 2024
5 checks passed

EmilFattakhov deleted the 3h_guide_updates branch January 31, 2024 15:02

	When a thread pool is created, it is assigned to a set of CPU cores and will only be able to use those cores. Pinning doesn’t pin threads to cores 1:1, instead, OS is free to move threads between cores, but only within CPU cores allocated for thread pool. This will ensure plotting for a particular sector only happens on a particular CPU/NUMA node.
	When a thread pool is created, it is assigned to a set of CPU cores and will only be able to use those cores. Pinning doesn’t pin threads to cores 1:1, instead, the OS is free to move threads between cores, but only within CPU cores allocated for the thread pool. This will ensure plotting for a particular sector only happens on a particular CPU/NUMA node.

		#### Enabling NUMA on Windows/Linux machines

		On Linux and Windows, the farmer will detect NUMA systems and create a number of thread pools that correspond to a number of NUMA nodes. This means the default behavior will change for large CPUs and consume more memory as a result, but that can be changed to the previous behavior with CLI options if desired.

		To use NUMA, you need to enable it via the BIOS of your motherboard for the farmer to know it exists. This option is present in motherboards for Threadripper/Epyc processors but might exist in others too. If you don’t enable it, both OS and farmer will think you have a single UMA processor and will not be able to apply optimizations.

		To read more about NUMA support and benefits it provides for large CPUs read [this forum post](https://forum.subspace.network/t/numa-support-is-coming/2299?u=nazar-pc).

	To read more about NUMA support and benefits it provides for large CPUs read [this forum post](https://forum.subspace.network/t/numa-support-is-coming/2299?u=nazar-pc).
	To read more about NUMA support and the benefits it provides for large CPUs read [this forum post](https://forum.subspace.network/t/numa-support-is-coming/2299?u=nazar-pc).


		You have successfully generated an operator key, congratulations!

		The keys were generated in the subfolder of PATH_TO_NODE under `/domains/domainID/keystore`. If you don't see the generated keypair in the subfolder, something went wrong.

	This might be useful if you decicded to switch domains and already have the secret phrase. Read more about switching domain in the next section.
	This might be useful if you decided to switch domains or already have the secret phrase. Read more about switching domain in the next section.

Gemini 3h guide updates #401

Gemini 3h guide updates #401

Conversation

EmilFattakhov commented Jan 16, 2024 • edited Loading

netlify bot commented Jan 16, 2024 • edited Loading

✅ Deploy Preview for subspace-docs-staging ready!

nazar-pc left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

DaMandal0rian Jan 23, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nazar-pc Jan 23, 2024 • edited Loading

Choose a reason for hiding this comment

EmilFattakhov commented Jan 23, 2024 • edited Loading

Choose a reason for hiding this comment

jim-counter left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jim-counter Jan 26, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

EmilFattakhov commented Jan 29, 2024

jim-counter left a comment

Choose a reason for hiding this comment

EmilFattakhov commented Jan 29, 2024

jim-counter left a comment

Choose a reason for hiding this comment

EmilFattakhov commented Jan 16, 2024 •

edited

Loading

netlify bot commented Jan 16, 2024 •

edited

Loading

DaMandal0rian Jan 23, 2024 •

edited

Loading

nazar-pc Jan 23, 2024 •

edited

Loading

EmilFattakhov commented Jan 23, 2024 •

edited

Loading

jim-counter left a comment •

edited

Loading

jim-counter Jan 26, 2024 •

edited

Loading