Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Partner Experience: update platform server prerequisites (#143) #170

Merged
merged 4 commits into from
Aug 14, 2023
Merged
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 24 additions & 35 deletions docs/run-platform-server/prerequisites.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -3,56 +3,45 @@ title: Prerequisites
sidebar_position: 10
---

Horizon only has one true dependency: a PostgreSQL server that it uses to store data that has been processed and ingested from Stellar Core. **Horizon requires PostgreSQL version >= 9.5**.
The Horizon service is responsible for synchronizing with the Stellar network and processing ledger data. To understand the scope of Horizon's services, please read the [configuring](./configuring.mdx) section before you move on to the prerequisites for computation.

As far as system requirements go, there are a few main things to keep in mind. Starting from version 2.0, Horizon must be run as a standalone service. A full Horizon build consists of three functions:
The Horizon service can be [installed](./installing.mdx) on bare metal or a virtual machine. It is natively supported on both Linux and Windows operating systems.

1. **ingesting data** _from_ the decentralized Stellar network,
1. **submitting transactions** _to_ the network, and
1. **serving** API requests
**Single Instance Deployment model**

The first two happen through _Captive Core_, a pared down, non-validating version of Stellar Core packaged directly into Horizon.
For a basic setup using the [Single Instance Deployment model](./configuring.mdx#single-instance-deployment), you will need two distinct compute instances:
urvisavla marked this conversation as resolved.
Show resolved Hide resolved

With these three functions in mind, you can also run Horizon in two different ways: **real-time ingestion** and **historical catch-up**:
- One for hosting the Horizon service.
- Another for hosting the PostgreSQL server.

- _Real-time ingestion_ is an “online” process: it involves keeping in sync with the live Stellar network state and digesting ledger data into a holistic view of the network. If you just want function (3) from above, you still need to do this.
The minimum hardware specifications to effectively run Horizon are as follows:

- _Historical catch-up_ is an “offline” process: it lets you look into the past and catch up your Horizon instance to a given retention period (e.g. 30 days of history). Because it’s typically done offline and a one-time process, you can dedicate more compute power and configure parallel workers to catch up faster.
**_Horizon Compute Instance:_**

### Historical Catch-up
- CPU: 4 cores
- RAM: 32 GB
- Storage: SSD with a capacity of 100 GB

In this scenario, the hardware specifications are more demanding than what is necessary for the day-to-day operation of real-time ingestion, but catch-up only needs to occur once.
**_PostgreSQL Database Server Compute Instance:_**

However, the requirements will vary depending on your chosen retention period and desired catch-up speed. Note that **most operators will not need full history**, and as the network continues to grow, tracking full history will become increasingly prohibitive. As of late 2021, DB storage to support historical retention is growing at a rate of 0.8 TB / month. It is highly recommended to configure retention of only the history needed to support your functionality.
- CPU: 4 cores
- RAM: 32 GB
- Storage: SSD with a capacity of 2 TB (NVMe or Direct Attached Storage) capable of handling at least 7K IOPS (I/O operations per second).

#### Requirements
Please note that a minimum of PostgreSQL version 12 is required.

Minimally, your disk storage type **must** be an SSD (e.g. NVMe, Direct Attached Storage) and your I/O **must** handle >15k iops (I/O operations per second). The following table breaks down hardware specifications for ingestion at different retention levels and performance tiers.
These specifications assume a 30-day retention window for data storage. For a longer retention window, the system requirements will be higher. For more information about data ingestion, history retention, and managing storage, check the [ingestion](./ingestion.mdx) section.

Note that each component can be scaled independently and for redundancy, in the manner of traditional _n_-tier systems which is covered later in [Scaling](./scaling.mdx). Ingestion can be sped up via configuring more Captive Core parallel workers (requiring more compute and RAM).
**Multiple Instance Deployment**

| Component | | Retention Period |
| :-- | :-- | :-- | --- |
| | **30 days** | **90 days** | **Full History** |
| **Parallel worker count**<br/>(est. ingestion time) | 6 workers (1 day) | 10 workers (1 day) | 20+ workers (2 days) |
| **Horizon** | **CPU**: 10 cores (min: 6) <br/>**RAM**: 64 GB (min: 32)<br/> | **CPU**: 16 (min: 8)<br/> **RAM**: 128 GB (64)<br/> | **CPU**: 16 (10)<br/> **RAM**: 512 GB (256) |
| **Database** | **CPU**: 16 cores (min: 8)<br/>**RAM**: 64 GB (min: 32GB)<br/>**Storage**: 2 TB<br/>**IOPS**: 20K (min: 15K) | **CPU**: 16 (12)<br/>**RAM**: 128 GB (64)<br/>**Storage**: 4 TB<br/>**IOPS**: 20K (15K) | **CPU**: 64 (32)<br/>**RAM**: 512 GB (256)<br/>**Storage**: 10 TB<br/>**IOPS**: 20k (15k) |
| **Storage**<br/>(all same) | | **SSD** (NVMe, Direct Attached Storage preferred) | |
| **AWS**<br/>(reference) | **Captive Core**: `m5.2xlarge`<br/>**Database**: `r5.2xlarge` | **Captive Core**: `m5.4xlarge`<br/>**DB**: `r5.4xlarge` | **Captive Core**: `c5.2xlarge` (x2)<br/>**DB**: `r5.16xlarge` (ro)<br/>`r5.8xlarge` (rw) |
To achieve high availability, redundancy, and high throughput, explore the [scaling](./scaling.mdx) strategy. It provides detailed prerequisites and guidelines to determine the appropriate [number of Horizon instances](./configuring.mdx#multiple-instance-deployment) to deploy.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm, do we need to update scaling.mdx, it hasn't been touched as of the rest of partner experience tickets, maybe it falls under the purview of prerequisites?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I'll take a look at the scaling doc.


### Real-Time Ingestion
**Network Access**

In this scenario, the goal is just to stay in sync with the Stellar network for day-to-day operations.
- Ensure that the Horizon instance can establish a connection with the PostgreSQL database instance. The default port for PostgreSQL is 5432.

There are two extremes to this spectrum: running a **single private instance** of Horizon for a specific application all the way up to a serious **enterprise public instance** of Horizon. In the former case, you’d run all three functions on a single machine and have low request volume; in the latter case, you’d have high-availability, redundancy, high request volume, full history, etc.
- A stable and fast network connection with the Internet is required for any Horizon instance running the ingestion role. This is to ensure it has efficient connectivity to remote hosts in the [quorum set](https://developers.stellar.org/docs/run-core-node/configuring#choosing-your-quorum-set) for the chosen Stellar network. During ingestion, the Horizon instance communicates with these hosts, receiving network transaction data through its local captive core sub-process.
urvisavla marked this conversation as resolved.
Show resolved Hide resolved

#### Requirements
**Note**

The following table breaks down requirements along this spectrum; if you fall somewhere in between, interpolate the requirements accordingly.

| Category | Private Instance | Enterprise Public Instance |
| :-- | :-- | :-- |
| **Compute** | Both **API Service** + **Captive Core**:<br/>**CPU**: 4<br/>**RAM**: 32 GB | **API Service**<br/>**CPU**: 4<br/>**RAM**: 8 GB<br/>N instances, load balanced <br/><br/> **Captive Core**<br/>**CPU**: 8<br/>**RAM**: 256 GB<br/>2 instances for redundancy |
| **Database** | **CPU**: 4<br/> **RAM**: 32 GB<br/> **IOPS**: 7k (min: 2.5k) | **CPU**: 32 - 64<br/> **RAM**: 256 - 512 GB<br/> **IOPS**: 20k (min: 15k)<br/> 2 HA instances: 1RO, 1RW |
| **Storage** (SSD) | depends on retention period | 10 TB |
| **AWS** (reference) | **API Service + Captive Core**<br/>`m5.2xlarge`<br/><br/>**Database**<br/>`r5.2xlarge` (ro)<br/>`r5.xlarge` (rw) | **API Service**<br/>`c5.xlarge` (_n_) <br/><br/>**Captive Core**<br/>`c5.2xlarge` (x2)<br/><br/>**Database** `r5.16xlarge` (ro)<br/>`r5.8xlarge` (rw) |
_Hardware requirements may increase if sharing resources or using custom configs, and as the Stellar network grows._