Merge of #5494

.github/workflows/test-suite.yml

@@ -383,6 +383,8 @@ jobs:
       run: make audit-CI
     - name: Run cargo vendor to make sure dependencies can be vendored for packaging, reproducibility and archival purpose
       run:  CARGO_HOME=$(readlink -f $HOME) make vendor
+    - name: Markdown-linter
+      run: make mdlint
   check-msrv:
     name: check-msrv
     runs-on: ubuntu-latest

Makefile

@@ -214,6 +214,10 @@ cli:
 cli-local:
 	make && ./scripts/cli.sh
 
+# Check for markdown files
+mdlint:
+	./scripts/mdlint.sh
+
 # Runs the entire test suite, downloading test vectors if required.
 test-full: cargo-fmt test-release test-debug test-ef test-exec-engine
 

book/.markdownlint.yml

@@ -0,0 +1,28 @@
+# MD010: https://github.com/DavidAnson/markdownlint/blob/main/doc/Rules.md#md010---hard-tabs
+MD010:
+  # Set code blocks to false so that code blocks will be ignored, default is true
+  code_blocks: false
+
+#MD013 line length: https://github.com/DavidAnson/markdownlint/blob/main/doc/md013.md
+# Set to false as this will also interfere with help_x.md files, and it is not necessary to comply with the line length of 80
+MD013: false
+
+# MD028: set to false to allow blank line between blockquote: https://github.com/DavidAnson/markdownlint/blob/main/doc/md028.md
+# This is because the blockquotes are shown separatedly (a deisred outcome) when having a blank line in between
+MD028: false
+
+# MD024: set siblings_only to true so that same headings with different parent headings are allowed
+# https://github.com/DavidAnson/markdownlint/blob/main/doc/md024.md
+MD024:
+  siblings_only: true
+
+# MD033 in-line html: https://github.com/DavidAnson/markdownlint/blob/main/doc/md033.md
+# In-line html is fine in the markdown files, so this is set to false
+MD033: false
+
+# MD036 set to false to preserve the emphasis on deprecation notice on key-management.md (a heading is not necessary)
+MD036: false
+
+# MD040 code blocks should have a language specified: https://github.com/DavidAnson/markdownlint/blob/main/doc/md040.md
+# Set to false as the help_x.md files are code blocks without a language specified, which is fine and does not need to change
+MD040: false

book/src/SUMMARY.md

@@ -2,66 +2,66 @@
 
 * [Introduction](./intro.md)
 * [Installation](./installation.md)
-    * [Pre-Built Binaries](./installation-binaries.md)
-    * [Docker](./docker.md)
-    * [Build from Source](./installation-source.md)
-    * [Raspberry Pi 4](./pi.md)
-    * [Cross-Compiling](./cross-compiling.md)
-    * [Homebrew](./homebrew.md)
-    * [Update Priorities](./installation-priorities.md)
+  * [Pre-Built Binaries](./installation-binaries.md)
+  * [Docker](./docker.md)
+  * [Build from Source](./installation-source.md)
+  * [Raspberry Pi 4](./pi.md)
+  * [Cross-Compiling](./cross-compiling.md)
+  * [Homebrew](./homebrew.md)
+  * [Update Priorities](./installation-priorities.md)
 * [Run a Node](./run_a_node.md)
 * [Become a Validator](./mainnet-validator.md)
 * [Validator Management](./validator-management.md)
-    * [The `validator-manager` Command](./validator-manager.md)
-        * [Creating validators](./validator-manager-create.md)
-        * [Moving validators](./validator-manager-move.md)
-    * [Slashing Protection](./slashing-protection.md)
-    * [Voluntary Exits](./voluntary-exit.md)
-    * [Partial Withdrawals](./partial-withdrawal.md)
-    * [Validator Monitoring](./validator-monitoring.md)
-    * [Doppelganger Protection](./validator-doppelganger.md)
-    * [Suggested Fee Recipient](./suggested-fee-recipient.md)
-    * [Validator Graffiti](./graffiti.md)
+  * [The `validator-manager` Command](./validator-manager.md)
+    * [Creating validators](./validator-manager-create.md)
+    * [Moving validators](./validator-manager-move.md)
+  * [Slashing Protection](./slashing-protection.md)
+  * [Voluntary Exits](./voluntary-exit.md)
+  * [Partial Withdrawals](./partial-withdrawal.md)
+  * [Validator Monitoring](./validator-monitoring.md)
+  * [Doppelganger Protection](./validator-doppelganger.md)
+  * [Suggested Fee Recipient](./suggested-fee-recipient.md)
+  * [Validator Graffiti](./graffiti.md)
 * [APIs](./api.md)
-    * [Beacon Node API](./api-bn.md)
-        * [Lighthouse API](./api-lighthouse.md)
-        * [Validator Inclusion APIs](./validator-inclusion.md)
-    * [Validator Client API](./api-vc.md)
-        * [Endpoints](./api-vc-endpoints.md)
-        * [Authorization Header](./api-vc-auth-header.md)
-        * [Signature Header](./api-vc-sig-header.md)
-    * [Prometheus Metrics](./advanced_metrics.md)
+  * [Beacon Node API](./api-bn.md)
+    * [Lighthouse API](./api-lighthouse.md)
+    * [Validator Inclusion APIs](./validator-inclusion.md)
+  * [Validator Client API](./api-vc.md)
+    * [Endpoints](./api-vc-endpoints.md)
+    * [Authorization Header](./api-vc-auth-header.md)
+    * [Signature Header](./api-vc-sig-header.md)
+  * [Prometheus Metrics](./advanced_metrics.md)
 * [Lighthouse UI (Siren)](./lighthouse-ui.md)
-	* [Installation](./ui-installation.md)
-    * [Authentication](./ui-authentication.md)
-	* [Configuration](./ui-configuration.md)
-	* [Usage](./ui-usage.md)
-	* [FAQs](./ui-faqs.md)
+  * [Installation](./ui-installation.md)
+  * [Authentication](./ui-authentication.md)
+  * [Configuration](./ui-configuration.md)
+  * [Usage](./ui-usage.md)
+  * [FAQs](./ui-faqs.md)
 * [Advanced Usage](./advanced.md)
-    * [Checkpoint Sync](./checkpoint-sync.md)
-    * [Custom Data Directories](./advanced-datadir.md)
-    * [Proposer Only Beacon Nodes](./advanced-proposer-only.md)
-    * [Remote Signing with Web3Signer](./validator-web3signer.md)
-    * [Database Configuration](./advanced_database.md)
-    * [Database Migrations](./database-migrations.md)
-    * [Key Management (Deprecated)](./key-management.md)
-    * [Key Recovery](./key-recovery.md)
-    * [Advanced Networking](./advanced_networking.md)
-    * [Running a Slasher](./slasher.md)
-    * [Redundancy](./redundancy.md)
-    * [Release Candidates](./advanced-release-candidates.md)
-    * [MEV](./builders.md)
-    * [Merge Migration](./merge-migration.md)
-    * [Late Block Re-orgs](./late-block-re-orgs.md)
-    * [Blobs](./advanced-blobs.md)
+  * [Checkpoint Sync](./checkpoint-sync.md)
+  * [Custom Data Directories](./advanced-datadir.md)
+  * [Proposer Only Beacon Nodes](./advanced-proposer-only.md)
+  * [Remote Signing with Web3Signer](./validator-web3signer.md)
+  * [Database Configuration](./advanced_database.md)
+  * [Database Migrations](./database-migrations.md)
+  * [Key Management (Deprecated)](./key-management.md)
+  * [Key Recovery](./key-recovery.md)
+  * [Advanced Networking](./advanced_networking.md)
+  * [Running a Slasher](./slasher.md)
+  * [Redundancy](./redundancy.md)
+  * [Release Candidates](./advanced-release-candidates.md)
+  * [MEV](./builders.md)
+  * [Merge Migration](./merge-migration.md)
+  * [Late Block Re-orgs](./late-block-re-orgs.md)
+  * [Blobs](./advanced-blobs.md)
 * [Built-In Documentation](./help_general.md)
-    * [Beacon Node](./help_bn.md)
-    * [Validator Client](./help_vc.md)
-    * [Validator Manager](./help_vm.md)
-        * [Create](./help_vm_create.md)
-        * [Import](./help_vm_import.md)
-        * [Move](./help_vm_move.md)  
+  * [Beacon Node](./help_bn.md)
+  * [Validator Client](./help_vc.md)
+  * [Validator Manager](./help_vm.md)
+    * [Create](./help_vm_create.md)
+    * [Import](./help_vm_import.md)
+    * [Move](./help_vm_move.md)  
 * [Contributing](./contributing.md)
-    * [Development Environment](./setup.md)
+  * [Development Environment](./setup.md)
 * [FAQs](./faq.md)
-* [Protocol Developers](./developers.md)
+* [Protocol Developers](./developers.md)

book/src/advanced-blobs.md

@@ -1,42 +1,41 @@
 # Blobs
 
-In the Deneb network upgrade, one of the changes is the implementation of EIP-4844, also known as [Proto-danksharding](https://blog.ethereum.org/2024/02/27/dencun-mainnet-announcement). Alongside with this, a new term named `blob` (binary large object) is introduced. Blobs are "side-cars" carrying transaction data in a block. They are mainly used by Ethereum layer 2 operators. As far as stakers are concerned, the main difference with the introduction of blobs is the increased storage requirement. 
+In the Deneb network upgrade, one of the changes is the implementation of EIP-4844, also known as [Proto-danksharding](https://blog.ethereum.org/2024/02/27/dencun-mainnet-announcement). Alongside with this, a new term named `blob` (binary large object) is introduced. Blobs are "side-cars" carrying transaction data in a block. They are mainly used by Ethereum layer 2 operators. As far as stakers are concerned, the main difference with the introduction of blobs is the increased storage requirement.
 
-### FAQ
+## FAQ
 
 1. What is the storage requirement for blobs?
 
    We expect an additional increase of ~50 GB of storage requirement for blobs (on top of what is required by the consensus and execution clients database). The calculation is as below:
 
    One blob is 128 KB in size. Each block can carry a maximum of 6 blobs. Blobs will be kept for 4096 epochs and pruned afterwards. This means that the maximum increase in storage requirement will be:
 
-   ```
+   ```text
    2**17 bytes / blob * 6 blobs / block * 32 blocks / epoch * 4096 epochs = 96 GB
    ```
 
    However, the blob base fee targets 3 blobs per block and it works similarly to how EIP-1559 operates in the Ethereum gas fee. Therefore, practically it is very likely to average to 3 blobs per blocks, which translates to a storage requirement of 48 GB.
 
-
 1. Do I have to add any flags for blobs?
 
-   No, you can use the default values for blob-related flags, which means you do not need add or remove any flags. 
+   No, you can use the default values for blob-related flags, which means you do not need add or remove any flags.
 
 1. What if I want to keep all blobs?
 
    Use the flag `--prune-blobs false` in the beacon node. The storage requirement will be:
 
-   ```
+   ```text
    2**17 bytes * 3 blobs / block * 7200 blocks / day * 30 days = 79GB / month or 948GB / year
    ```
-   
+
    To keep blobs for a custom period, you may use the flag `--blob-prune-margin-epochs <EPOCHS>` which keeps blobs for 4096+EPOCHS specified in the flag.
 
 1. How to see the info of the blobs database?
 
-   We can call the API: 
+   We can call the API:
 
    ```bash
    curl "http://localhost:5052/lighthouse/database/info" | jq
    ```
 
-   Refer to [Lighthouse API](./api-lighthouse.md#lighthousedatabaseinfo) for an example response. 
+   Refer to [Lighthouse API](./api-lighthouse.md#lighthousedatabaseinfo) for an example response.

book/src/advanced-datadir.md

@@ -1,4 +1,4 @@
-## Custom Data Directories
+# Custom Data Directories
 
 Users can override the default Lighthouse data directories (e.g., `~/.lighthouse/mainnet`) using the `--datadir` flag. The custom data directory mirrors the structure of any network specific default directory (e.g. `~/.lighthouse/mainnet`).
 
@@ -11,10 +11,11 @@ lighthouse --network mainnet --datadir /var/lib/my-custom-dir account validator
 lighthouse --network mainnet --datadir /var/lib/my-custom-dir bn --staking
 lighthouse --network mainnet --datadir /var/lib/my-custom-dir vc
 ```
+
 The first step creates a `validators` directory under `/var/lib/my-custom-dir` which contains the imported keys and [`validator_definitions.yml`](./validator-management.md).
 After that, we simply run the beacon chain and validator client with the custom dir path.
 
-### Relative Paths
+## Relative Paths
 
 [#2682]: https://github.com/sigp/lighthouse/pull/2682
 [#2846]: https://github.com/sigp/lighthouse/pull/2846
@@ -40,7 +41,7 @@ be applied. On start-up, if a split directory scenario is detected (i.e. `~/here
 Lighthouse will continue to operate with split directories. In such a scenario, the following
 harmless log will show:
 
-```
+```text
 WARN Legacy datadir location    location: "/home/user/datadir/beacon", msg: this occurs when using relative paths for a datadir location
 ```
 

book/src/advanced-proposer-only.md

@@ -2,7 +2,7 @@
 
 Lighthouse allows for more exotic setups that can minimize attack vectors by
 adding redundant beacon nodes and dividing the roles of attesting and block
-production between them. 
+production between them.
 
 The purpose of this is to minimize attack vectors
 where malicious users obtain the network identities (IP addresses) of beacon
@@ -24,7 +24,7 @@ harder to identify as a potential node to attack and will also consume less
 resources.
 
 Specifically, this flag reduces the default peer count (to a safe minimal
-number as maintaining peers on attestation subnets do not need to be considered), 
+number as maintaining peers on attestation subnets do not need to be considered),
 prevents the node from subscribing to any attestation-subnets or
 sync-committees which is a primary way for attackers to de-anonymize
 validators.
@@ -34,7 +34,6 @@ validators.
 > normal beacon node, the validator may fail to handle its duties correctly and
 > result in a loss of income.
 
-
 ## The Validator Client
 
 The validator client can be given a list of HTTP API endpoints representing
@@ -53,7 +52,6 @@ these nodes for added security).
 > producing a more profitable block. Any block builders should therefore be
 > attached to the `--beacon-nodes` and not necessarily the `--proposer-nodes`.
 
-
 ## Setup Overview
 
 The intended set-up to take advantage of this mechanism is to run one (or more)

book/src/advanced-release-candidates.md

@@ -20,7 +20,7 @@ you're looking for stable Lighthouse**.
 From time to time, Lighthouse may use the terms "release candidate" and "pre release"
 interchangeably. A pre release is identical to a release candidate.
 
-### Examples
+## Examples
 
 [`v1.4.0-rc.0`] has `rc` in the version string and is therefore a release candidate. This release is
 *not* stable and is *not* intended for critical tasks on mainnet (e.g., staking).
@@ -36,9 +36,8 @@ Users may wish to try a release candidate for the following reasons:
 - To help detect bugs and regressions before they reach production.
 - To provide feedback on annoyances before they make it into a release and become harder to change or revert.
 
-There can also be a scenario that a bug has been found and requires an urgent fix. An example of incidence is [v4.0.2-rc.0](https://github.com/sigp/lighthouse/releases/tag/v4.0.2-rc.0) which contains a hot-fix to address high CPU usage experienced after the [Capella](https://ethereum.org/en/history/#capella) upgrade on 12<sup>th</sup> April 2023.  In this scenario, we will announce the release candidate on [Github](https://github.com/sigp/lighthouse/releases) and also on [Discord](https://discord.gg/cyAszAh) to recommend users to update to the release candidate version. 
+There can also be a scenario that a bug has been found and requires an urgent fix. An example of incidence is [v4.0.2-rc.0](https://github.com/sigp/lighthouse/releases/tag/v4.0.2-rc.0) which contains a hot-fix to address high CPU usage experienced after the [Capella](https://ethereum.org/en/history/#capella) upgrade on 12<sup>th</sup> April 2023.  In this scenario, we will announce the release candidate on [Github](https://github.com/sigp/lighthouse/releases) and also on [Discord](https://discord.gg/cyAszAh) to recommend users to update to the release candidate version.
 
 ## When *not* to use a release candidate
 
 Other than the above scenarios, it is generally not recommended to use release candidates for any critical tasks on mainnet (e.g., staking). To test new release candidate features, try one of the testnets (e.g., Holesky).
-

book/src/advanced.md

@@ -15,7 +15,7 @@ tips about how things work under the hood.
 * [Key Management](./key-management.md): explore how to generate wallet with Lighthouse.
 * [Key Recovery](./key-recovery.md): explore how to recover wallet and validator with Lighthouse.
 * [Advanced Networking](./advanced_networking.md): open your ports to have a diverse and healthy set of peers.
-* [Running a Slasher](./slasher.md): contribute to the health of the network by running a slasher. 
+* [Running a Slasher](./slasher.md): contribute to the health of the network by running a slasher.
 * [Redundancy](./redundancy.md): want to have more than one beacon node as backup? This is for you.
 * [Release Candidates](./advanced-release-candidates.md): latest release of Lighthouse to get feedback from users.
 * [Maximal Extractable Value](./builders.md): use external builders for a potential higher rewards during block proposals

book/src/advanced_database.md

@@ -29,7 +29,7 @@ some example values.
 | Enthusiast (prev. default) | 2048 | hundreds of GB    | 10.2 s                |
 | Validator only (default)   | 8192 | tens of GB        | 41 s                  |
 
-*Last update: Dec 2023. 
+*Last update: Dec 2023.
 
 As we can see, it's a high-stakes trade-off! The relationships to disk usage and historical state
 load time are both linear – doubling SPRP halves disk usage and doubles load time. The minimum SPRP
@@ -40,9 +40,9 @@ The default value is 8192 for databases synced from scratch using Lighthouse v2.
 
 The values shown in the table are approximate, calculated using a simple heuristic: each
 `BeaconState` consumes around 145MB of disk space, and each block replayed takes around 5ms.  The
-**Yearly Disk Usage** column shows the approximate size of the freezer DB _alone_ (hot DB not included), calculated proportionally using the total freezer database disk usage. 
+**Yearly Disk Usage** column shows the approximate size of the freezer DB _alone_ (hot DB not included), calculated proportionally using the total freezer database disk usage.
 The **Load Historical State** time is the worst-case load time for a state in the last slot
-before a restore point. 
+before a restore point.
 
 To run a full archival node with fast access to beacon states and a SPRP of 32, the disk usage will be more than 10 TB per year, which is impractical for many users. As such, users may consider running the [tree-states](https://github.com/sigp/lighthouse/releases/tag/v5.0.111-exp) release, which only uses less than 200 GB for a full archival node. The caveat is that it is currently experimental and in alpha release (as of Dec 2023), thus not recommended for running mainnet validators. Nevertheless, it is suitable to be used for analysis purposes, and if you encounter any issues in tree-states, we do appreciate any feedback. We plan to have a stable release of tree-states in 1H 2024.  
 

book/src/advanced_metrics.md

@@ -30,7 +30,6 @@ curl localhost:5054/metrics
 
 ## Validator Client Metrics
 
-
 By default, these metrics are disabled but can be enabled with the `--metrics`
 flag. Use the `--metrics-address`, `--metrics-port` and
 `--metrics-allow-origin` flags to customize the metrics server.
@@ -78,7 +77,7 @@ You can adjust the frequency at which Lighthouse sends metrics to the remote ser
 `--monitoring-endpoint-period` flag. It takes an integer value in seconds, defaulting to 60
 seconds.
 
-```
+```bash
 lighthouse bn --monitoring-endpoint-period 60 --monitoring-endpoint "https://url"
 ```