From 4edfef6fed7bd3a4eea08bba0b7fc1f7baf97971 Mon Sep 17 00:00:00 2001 From: Urvi Date: Wed, 5 Jul 2023 10:54:50 -0700 Subject: [PATCH 1/4] Partner Experience: update platform server prerequisites (#143) Update docs/run-platform-server/prerequisites.mdx Co-authored-by: shawn Update docs/run-platform-server/prerequisites.mdx Co-authored-by: shawn Update docs/run-platform-server/prerequisites.mdx Co-authored-by: shawn Update docs/run-platform-server/prerequisites.mdx Co-authored-by: shawn Update docs/run-platform-server/prerequisites.mdx Co-authored-by: shawn Update docs/run-platform-server/prerequisites.mdx Co-authored-by: shawn Addressing review comments --- docs/run-platform-server/prerequisites.mdx | 63 ++++++++++------------ 1 file changed, 27 insertions(+), 36 deletions(-) diff --git a/docs/run-platform-server/prerequisites.mdx b/docs/run-platform-server/prerequisites.mdx index 38f7b45d4..3124732cb 100644 --- a/docs/run-platform-server/prerequisites.mdx +++ b/docs/run-platform-server/prerequisites.mdx @@ -3,56 +3,47 @@ title: Prerequisites sidebar_position: 10 --- -Horizon only has one true dependency: a PostgreSQL server that it uses to store data that has been processed and ingested from Stellar Core. **Horizon requires PostgreSQL version >= 9.5**. +The Horizon service is responsible for synchronizing with the Stellar network and processing ledger data. We recommend reading the [configuring](./configuring.mdx) section as a primer on scope of Horizon's services before diving into compute pre-requisites. -As far as system requirements go, there are a few main things to keep in mind. Starting from version 2.0, Horizon must be run as a standalone service. A full Horizon build consists of three functions: +The Horizon service can be [installed](./installing.mdx) on bare metal or a virtual machine. It is natively supported on both Linux and Windows operating systems. -1. **ingesting data** _from_ the decentralized Stellar network, -1. **submitting transactions** _to_ the network, and -1. **serving** API requests +**Single Instance Deployment model** -The first two happen through _Captive Core_, a pared down, non-validating version of Stellar Core packaged directly into Horizon. +At a minimum, following the [Single Instance Deployment model](./configuring.mdx#single-instance-deployment) will require two compute instances: -With these three functions in mind, you can also run Horizon in two different ways: **real-time ingestion** and **historical catch-up**: +- one compute instance for Horizon +- one compute instance for PostgreSQL server. -- _Real-time ingestion_ is an “online” process: it involves keeping in sync with the live Stellar network state and digesting ledger data into a holistic view of the network. If you just want function (3) from above, you still need to do this. +To run Horizon, we recommend a single compute profile which is applicable for single or multiple instances, the minimum hardware specifications are: -- _Historical catch-up_ is an “offline” process: it lets you look into the past and catch up your Horizon instance to a given retention period (e.g. 30 days of history). Because it’s typically done offline and a one-time process, you can dedicate more compute power and configure parallel workers to catch up faster. +**_Horizon compute instance:_** -### Historical Catch-up +- CPU: x cores +- RAM: x GB +- Storage: SSD with a capacity of x GB -In this scenario, the hardware specifications are more demanding than what is necessary for the day-to-day operation of real-time ingestion, but catch-up only needs to occur once. +**_PostgreSQL database server compute instance:_** -However, the requirements will vary depending on your chosen retention period and desired catch-up speed. Note that **most operators will not need full history**, and as the network continues to grow, tracking full history will become increasingly prohibitive. As of late 2021, DB storage to support historical retention is growing at a rate of 0.8 TB / month. It is highly recommended to configure retention of only the history needed to support your functionality. +- CPU: x cores +- RAM: x GB +- Storage: SSD with a capacity of x TB (NVMe or Direct Attached Storage) capable of handling > xk IOPS (I/O operations per second). -#### Requirements +These specifications consider a 30-day retention window for data storage. For a longer retention window, the system requirements will change accordingly. Consult the [ingestion](./ingestion.mdx) section for details on data ingestion and history retention, and managing storage requirements. -Minimally, your disk storage type **must** be an SSD (e.g. NVMe, Direct Attached Storage) and your I/O **must** handle >15k iops (I/O operations per second). The following table breaks down hardware specifications for ingestion at different retention levels and performance tiers. +**Multiple Instance Deployment** -Note that each component can be scaled independently and for redundancy, in the manner of traditional _n_-tier systems which is covered later in [Scaling](./scaling.mdx). Ingestion can be sped up via configuring more Captive Core parallel workers (requiring more compute and RAM). +To ensure high availability, redundancy, and achieve high throughput, please refer to the [scaling](./scaling.mdx) strategy. It provides detailed prerequisites and guidelines for determining the [number of Horizon instances](./configuring.mdx#multiple-instance-deployment) to deploy. -| Component | | Retention Period | -| :-- | :-- | :-- | --- | -| | **30 days** | **90 days** | **Full History** | -| **Parallel worker count**
(est. ingestion time) | 6 workers (1 day) | 10 workers (1 day) | 20+ workers (2 days) | -| **Horizon** | **CPU**: 10 cores (min: 6)
**RAM**: 64 GB (min: 32)
| **CPU**: 16 (min: 8)
**RAM**: 128 GB (64)
| **CPU**: 16 (10)
**RAM**: 512 GB (256) | -| **Database** | **CPU**: 16 cores (min: 8)
**RAM**: 64 GB (min: 32GB)
**Storage**: 2 TB
**IOPS**: 20K (min: 15K) | **CPU**: 16 (12)
**RAM**: 128 GB (64)
**Storage**: 4 TB
**IOPS**: 20K (15K) | **CPU**: 64 (32)
**RAM**: 512 GB (256)
**Storage**: 10 TB
**IOPS**: 20k (15k) | -| **Storage**
(all same) | | **SSD** (NVMe, Direct Attached Storage preferred) | | -| **AWS**
(reference) | **Captive Core**: `m5.2xlarge`
**Database**: `r5.2xlarge` | **Captive Core**: `m5.4xlarge`
**DB**: `r5.4xlarge` | **Captive Core**: `c5.2xlarge` (x2)
**DB**: `r5.16xlarge` (ro)
`r5.8xlarge` (rw) | +**Network Access** -### Real-Time Ingestion +- The Horizon compute instance must be able to connect to the PostgreSQL database instance. The default port for PostgreSQL is 5432. +- A stable and fast network connection with outbound path to internet is recommended for any Horizon compute instance running the ingestion role. This is to ensure it has efficient link to remote hosts in the [quorum set](https://developers.stellar.org/docs/run-core-node/configuring#choosing-your-quorum-set) for the chosen Stellar network. Network transaction data is emitted from these hosts and received by the Horizon instance during ingestion through its local captive core sub-process.\ + It is recommended to configure Horizon using the NETWORK config parameter from the [configuring](./configuring.mdx) section. This configuration automatically generates the list of hosts for the quorum set. You can obtain this generated list of hosts anytime by using the command: -In this scenario, the goal is just to stay in sync with the Stellar network for day-to-day operations. + ```bash + stellar-horizon --show-network-quorum [testnet|pubnet] + ``` -There are two extremes to this spectrum: running a **single private instance** of Horizon for a specific application all the way up to a serious **enterprise public instance** of Horizon. In the former case, you’d run all three functions on a single machine and have low request volume; in the latter case, you’d have high-availability, redundancy, high request volume, full history, etc. +**Additional Notes** -#### Requirements - -The following table breaks down requirements along this spectrum; if you fall somewhere in between, interpolate the requirements accordingly. - -| Category | Private Instance | Enterprise Public Instance | -| :-- | :-- | :-- | -| **Compute** | Both **API Service** + **Captive Core**:
**CPU**: 4
**RAM**: 32 GB | **API Service**
**CPU**: 4
**RAM**: 8 GB
N instances, load balanced

**Captive Core**
**CPU**: 8
**RAM**: 256 GB
2 instances for redundancy | -| **Database** | **CPU**: 4
**RAM**: 32 GB
**IOPS**: 7k (min: 2.5k) | **CPU**: 32 - 64
**RAM**: 256 - 512 GB
**IOPS**: 20k (min: 15k)
2 HA instances: 1RO, 1RW | -| **Storage** (SSD) | depends on retention period | 10 TB | -| **AWS** (reference) | **API Service + Captive Core**
`m5.2xlarge`

**Database**
`r5.2xlarge` (ro)
`r5.xlarge` (rw) | **API Service**
`c5.xlarge` (_n_)

**Captive Core**
`c5.2xlarge` (x2)

**Database** `r5.16xlarge` (ro)
`r5.8xlarge` (rw) | +_Hardware requirements may increase if sharing resources or using custom configs, and as the Stellar network grows._ From f8f7f7302f7eadc6a698d50bdeaa3f0154b2e15f Mon Sep 17 00:00:00 2001 From: Urvi Date: Wed, 9 Aug 2023 14:32:34 -0700 Subject: [PATCH 2/4] Updated the specs and other minor rewording --- docs/run-platform-server/prerequisites.mdx | 42 +++++++++++----------- 1 file changed, 20 insertions(+), 22 deletions(-) diff --git a/docs/run-platform-server/prerequisites.mdx b/docs/run-platform-server/prerequisites.mdx index 3124732cb..4e87ab4a7 100644 --- a/docs/run-platform-server/prerequisites.mdx +++ b/docs/run-platform-server/prerequisites.mdx @@ -3,47 +3,45 @@ title: Prerequisites sidebar_position: 10 --- -The Horizon service is responsible for synchronizing with the Stellar network and processing ledger data. We recommend reading the [configuring](./configuring.mdx) section as a primer on scope of Horizon's services before diving into compute pre-requisites. +The Horizon service is responsible for synchronizing with the Stellar network and processing ledger data. To understand the scope of Horizon's services, please read the [configuring](./configuring.mdx) section before you move on to the prerequisites for computation. The Horizon service can be [installed](./installing.mdx) on bare metal or a virtual machine. It is natively supported on both Linux and Windows operating systems. **Single Instance Deployment model** -At a minimum, following the [Single Instance Deployment model](./configuring.mdx#single-instance-deployment) will require two compute instances: +For a basic setup using the [Single Instance Deployment model](./configuring.mdx#single-instance-deployment), you will need two distinct compute instances: -- one compute instance for Horizon -- one compute instance for PostgreSQL server. +- One for hosting the Horizon service. +- Another for hosting the PostgreSQL server. -To run Horizon, we recommend a single compute profile which is applicable for single or multiple instances, the minimum hardware specifications are: +The minimum hardware specifications to effectively run Horizon are as follows: -**_Horizon compute instance:_** +**_Horizon Compute Instance:_** -- CPU: x cores -- RAM: x GB -- Storage: SSD with a capacity of x GB +- CPU: 4 cores +- RAM: 32 GB +- Storage: SSD with a capacity of 100 GB -**_PostgreSQL database server compute instance:_** +**_PostgreSQL Database Server Compute Instance:_** -- CPU: x cores -- RAM: x GB -- Storage: SSD with a capacity of x TB (NVMe or Direct Attached Storage) capable of handling > xk IOPS (I/O operations per second). +- CPU: 4 cores +- RAM: 32 GB +- Storage: SSD with a capacity of 2 TB (NVMe or Direct Attached Storage) capable of handling at least 7K IOPS (I/O operations per second). -These specifications consider a 30-day retention window for data storage. For a longer retention window, the system requirements will change accordingly. Consult the [ingestion](./ingestion.mdx) section for details on data ingestion and history retention, and managing storage requirements. +Please note that a minimum of PostgreSQL version 12 is required. + +These specifications assume a 30-day retention window for data storage. For a longer retention window, the system requirements will be higher. For more information about data ingestion, history retention, and managing storage, check the [ingestion](./ingestion.mdx) section. **Multiple Instance Deployment** -To ensure high availability, redundancy, and achieve high throughput, please refer to the [scaling](./scaling.mdx) strategy. It provides detailed prerequisites and guidelines for determining the [number of Horizon instances](./configuring.mdx#multiple-instance-deployment) to deploy. +To achieve high availability, redundancy, and high throughput, explore the [scaling](./scaling.mdx) strategy. It provides detailed prerequisites and guidelines to determine the appropriate [number of Horizon instances](./configuring.mdx#multiple-instance-deployment) to deploy. **Network Access** -- The Horizon compute instance must be able to connect to the PostgreSQL database instance. The default port for PostgreSQL is 5432. -- A stable and fast network connection with outbound path to internet is recommended for any Horizon compute instance running the ingestion role. This is to ensure it has efficient link to remote hosts in the [quorum set](https://developers.stellar.org/docs/run-core-node/configuring#choosing-your-quorum-set) for the chosen Stellar network. Network transaction data is emitted from these hosts and received by the Horizon instance during ingestion through its local captive core sub-process.\ - It is recommended to configure Horizon using the NETWORK config parameter from the [configuring](./configuring.mdx) section. This configuration automatically generates the list of hosts for the quorum set. You can obtain this generated list of hosts anytime by using the command: +- Ensure that the Horizon instance can establish a connection with the PostgreSQL database instance. The default port for PostgreSQL is 5432. - ```bash - stellar-horizon --show-network-quorum [testnet|pubnet] - ``` +- A stable and fast network connection with the Internet is required for any Horizon instance running the ingestion role. This is to ensure it has efficient connectivity to remote hosts in the [quorum set](https://developers.stellar.org/docs/run-core-node/configuring#choosing-your-quorum-set) for the chosen Stellar network. During ingestion, the Horizon instance communicates with these hosts, receiving network transaction data through its local captive core sub-process. -**Additional Notes** +**Note** _Hardware requirements may increase if sharing resources or using custom configs, and as the Stellar network grows._ From fbfaa7bf33d0dcc2dd1c2247ad17d6d433923117 Mon Sep 17 00:00:00 2001 From: urvisavla Date: Thu, 10 Aug 2023 15:08:57 -0700 Subject: [PATCH 3/4] Update docs/run-platform-server/prerequisites.mdx Co-authored-by: shawn --- docs/run-platform-server/prerequisites.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/run-platform-server/prerequisites.mdx b/docs/run-platform-server/prerequisites.mdx index 4e87ab4a7..0105d54f5 100644 --- a/docs/run-platform-server/prerequisites.mdx +++ b/docs/run-platform-server/prerequisites.mdx @@ -40,7 +40,7 @@ To achieve high availability, redundancy, and high throughput, explore the [scal - Ensure that the Horizon instance can establish a connection with the PostgreSQL database instance. The default port for PostgreSQL is 5432. -- A stable and fast network connection with the Internet is required for any Horizon instance running the ingestion role. This is to ensure it has efficient connectivity to remote hosts in the [quorum set](https://developers.stellar.org/docs/run-core-node/configuring#choosing-your-quorum-set) for the chosen Stellar network. During ingestion, the Horizon instance communicates with these hosts, receiving network transaction data through its local captive core sub-process. +- A stable and fast network connection with the Internet is required for any Horizon instance running the ingestion role. This is to ensure it has efficient outbound connectivity to remote hosts in the [quorum set](https://developers.stellar.org/docs/run-core-node/configuring#choosing-your-quorum-set) and [archive urls](https://developers.stellar.org/docs/run-core-node/configuring#history) for the chosen Stellar network. During ingestion, the Horizon instance communicates with these hosts, receiving network transaction data through its local captive core sub-process. **Note** From a9697b377c1e07c0480ab9207a6321a136e79d3e Mon Sep 17 00:00:00 2001 From: urvisavla Date: Thu, 10 Aug 2023 15:19:38 -0700 Subject: [PATCH 4/4] Update docs/run-platform-server/prerequisites.mdx Co-authored-by: shawn --- docs/run-platform-server/prerequisites.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/run-platform-server/prerequisites.mdx b/docs/run-platform-server/prerequisites.mdx index 0105d54f5..53248f7c9 100644 --- a/docs/run-platform-server/prerequisites.mdx +++ b/docs/run-platform-server/prerequisites.mdx @@ -9,7 +9,7 @@ The Horizon service can be [installed](./installing.mdx) on bare metal or a virt **Single Instance Deployment model** -For a basic setup using the [Single Instance Deployment model](./configuring.mdx#single-instance-deployment), you will need two distinct compute instances: +For a basic setup using the [Single Instance Deployment model](./configuring.mdx#single-instance-deployment), you will need a sum of two distinct compute profiles: - One for hosting the Horizon service. - Another for hosting the PostgreSQL server.