Skip to content

Commit

Permalink
Sync docs from Discourse (canonical#606)
Browse files Browse the repository at this point in the history
Co-authored-by: GitHub Actions <41898282+github-actions[bot]@users.noreply.github.com>
  • Loading branch information
github-actions[bot] authored Oct 15, 2024
1 parent 75f7f01 commit 3c7c783
Show file tree
Hide file tree
Showing 35 changed files with 1,516 additions and 404 deletions.
17 changes: 17 additions & 0 deletions docs/explanation/e-connection-pooling.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# Connection pooling

Connection pooling is a strategy to reduce the amount of active connections and the costs of reopening connections. It requires maintaining a set of persistently opened connections, called a pool, that can be reused by clients.

Since PostgreSQL spawns separate processes for client connections, it can be beneficial in some use cases to maintain a client side connection pool rather than increase the database connection limits and resource consumption.

A way to achieve this with Charmed PostgreSQL is by integrating with the [PgBouncer charm](https://charmhub.io/pgbouncer?channel=1/stable).

## Increasing maximum allowed connections

If using PgBouncer is not enough to handle the connections load of your application, you can increase the amount of connections that PostgreSQL can open via the [`experimental_max_connections` config parameter](https://charmhub.io/postgresql/configurations#experimental_max_connections).

[note type="caution"]
**Disclaimer:** Each connection opened by PostgreSQL spawns a new process, which is resource-intensive. Use this option as a last resort.

[Contact us](/t/11863) for more guidance about your use-case.
[/note]
8 changes: 5 additions & 3 deletions docs/how-to/h-async-integrate.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,11 @@
This guide will show you how to integrate a client application with a cross-regional async setup using an example PostgreSQL deployment with two servers: one in Rome and one in Lisbon.

## Prerequisites
* `juju v.3.4.2+`
* Juju `v.3.4.2+`
* Make sure your machine(s) fulfill the [system requirements](/t/11743)
* See [supported target/source model relationships](/t/15412#substrate-dependencies).
* A cross-regional async replication setup
* Refer to the page [How to set up clusters](/t/13991)
* See [How to set up clusters](/t/13991)

## Summary
* [Configure database endpoints](#configure-database-endpoints)
Expand Down Expand Up @@ -53,7 +55,7 @@ juju switch app
juju deploy postgresql-test-app
juju deploy pgbouncer --channel 1/stable

juju integrate postgresql-test-app:first-database pgbouncer
juju integrate postgresql-test-app:database pgbouncer
juju integrate pgbouncer db1database
```

Expand Down
6 changes: 4 additions & 2 deletions docs/how-to/h-async-remove-recover.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,11 @@
This guide will cover how to manage clusters using an example PostgreSQL deployment with two servers: one in Rome and one in Lisbon.

## Prerequisites
* `juju v.3.4.2+`
* Juju `v.3.4.2+`
* Make sure your machine(s) fulfill the [system requirements](/t/11743)
* See [supported target/source model relationships](/t/15412#substrate-dependencies).
* A cross-regional async replication setup
* Refer to the page [How to set up clusters](/t/13991)
* See [How to set up clusters](/t/13991)

## Summary
* [Switchover](#switchover)
Expand Down
21 changes: 13 additions & 8 deletions docs/how-to/h-async-set-up.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,11 @@ Cross-regional (or multi-server) asynchronous replication focuses on disaster re

This guide will show you the basics of initiating a cross-regional async setup using an example PostgreSQL deployment with two servers: one in Rome and one in Lisbon.

## Prerequisites
* Juju `v.3.4.2+`
* Make sure your machine(s) fulfill the [system requirements](/t/11743)
* See [supported target/source model relationships](/t/15412#substrate-dependencies).

## Summary
* [Deploy](#deploy)
* [Offer](#offer)
Expand All @@ -22,10 +27,10 @@ juju add-model rome
juju add-model lisbon

juju switch rome # active model must correspond to cluster
juju deploy postgresql db1 --channel=14/edge/async-replication --config profile=testing --base [email protected]
juju deploy postgresql db1

juju switch lisbon
juju deploy postgresql db2 --channel=14/edge/async-replication --config profile=testing --base [email protected]
juju deploy postgresql db2
```

## Offer
Expand All @@ -34,30 +39,30 @@ juju deploy postgresql db2 --channel=14/edge/async-replication --config profile=

```shell
juju switch rome
juju offer db1:async-primary async-primary
juju offer db1:replication-offer replication-offer
```

## Consume

Consume asynchronous replication on planned `Standby` cluster (Lisbon):
```shell
juju switch lisbon
juju consume rome.async-primary
juju integrate async-primary db2:async-replica
juju consume rome.replication-offer
juju integrate replication-offer db2:replication
```

## Promote or switchover a cluster

To define the primary cluster, use the `promote-cluster` action.
To define the primary cluster, use the `create-replication` action.

```shell
juju run -m rome db1/leader promote-cluster
juju run -m rome db1/leader create-replication
```

To switchover and use `lisbon` as the primary instead, run

```shell
juju run -m lisbon db2/leader promote-cluster force-promotion=true
juju run -m lisbon db2/leader promote-to-primary
```

## Scale a cluster
Expand Down
30 changes: 30 additions & 0 deletions docs/how-to/h-async.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
# Cross-regional async replication

Cross-regional (or multi-server) asynchronous replication focuses on disaster recovery by distributing data across different servers.

## Prerequisites
* Juju `v.3.4.2+`
* Make sure your machine(s) fulfill the [system requirements](/t/11743)

### Substrate dependencies

The following table shows the source and target controller/model combinations that are currently supported:

| | AWS | GCP | Azure |
|---|---|:---:|:---:|
| AWS | | | |
| GCP | | ![ check ] | ![ check ] |
| Azure | | ![ check ] | ![ check ] |

## How-to guides

* [How to set up clusters for cross-regional async replication](/t/13991)
* [How to integrate with a client application](/t/13992)
* [How to remove or recover a cluster](/t/13994)
* [Switchover](/t/13994#switchover)
* [Detach](/t/13994#detach-a-cluster)
* [Recover](/t/13994#recover-a-cluster)

<!-- BADGES -->
[check]: https://img.shields.io/badge/%E2%9C%93-brightgreen
[cross]: https://img.shields.io/badge/x-white
144 changes: 144 additions & 0 deletions docs/how-to/h-deploy-airgapped.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,144 @@
# Deploy in an offline or air-gapped environment

An air-gapped environment refers to a system that does not have access to the public internet.
This guide goes through the special configuration steps for installing Charmed PostgreSQL VM in an air-gapped environment.

## Requirements

Canonical does not prescribe how you should set up your specific air-gapped environment. However, it is assumed that it meets the following conditions:

* A VM/hardware resources available for Juju.
* DNS is configured to the local nameservers.
* [Juju is configured](https://documentation.ubuntu.com/snap-store-proxy/en/airgap-charmhub/#configure-juju) to use local air-gapped services.
* The [`store-admin`](https://snapcraft.io/store-admin) tool is installed and configured.
* [Air-gapped CharmHub](https://documentation.ubuntu.com/snap-store-proxy/en/airgap-charmhub/) is installed and running.
* [Air-gapped Snap Store Proxy](https://documentation.ubuntu.com/snap-store-proxy/) is installed and running.
* Local APT and LXD Images caches are reachable.

## Air-gapped setup summary

[1\. Export snaps and charms](#1-export-snaps-and-charms) <br>
[2\. Transfer binary blobs](#2-transfer-binary-blobs) <br>
[3\. Import snaps and charms](3-import-snaps-and-charms) <br>
[4\. Deploy PostgreSQL](#4-deploy-postgresql)

## Air-gapped day-to-day example

### 1. Export snaps and charms
Exporting VM SNAPs and Charms and are currently independent processes. The `store-admin` tool is designed to simplify the process.

Future improvements are planned to the `store-admin` tool so that it could potentially export all necessary SNAP resource(s) from the official SnapStore with Charms simultaneously. Other planned improvements include supporting the export of specific charm and resource by revisions ([PF-5369](https://warthogs.atlassian.net/browse/PF-5369), [PF-5185](https://warthogs.atlassian.net/browse/PF-5185)).

#### Charms
The necessary charm(s) can be exported as bundle or independently (charm-by-charm). See the Snap Proxy documentation:
* [Offline Charmhub configuration > Export charm bundle](https://documentation.ubuntu.com/snap-store-proxy/en/airgap-charmhub/#export-charm-bundles)
* [Offline Charmhub configuration > Export charms](https://documentation.ubuntu.com/snap-store-proxy/en/airgap-charmhub/#export-charms)

The bundle export example:

<details>
<summary><code>store-admin export bundle postgresql-bundle --channel=14/edge --series=jammy --arch=amd64</code></summary>

```
Downloading postgresql-bundle revision 140 (14/edge)
[####################################] 100%
Downloading data-integrator revision 71 (edge)
[####################################] 100%
Downloading grafana-agent revision 286 (edge)
[####################################] 100%
Downloading landscape-client revision 69 (edge)
[####################################] 100%
Downloading pgbouncer revision 473 (1/edge)
[####################################] 100%
Downloading postgresql revision 487 (14/edge)
[####################################] 100%
Downloading postgresql-test-app revision 256 (edge)
[####################################] 100%
Downloading s3-integrator revision 59 (edge)
[####################################] 100%
Downloading self-signed-certificates revision 200 (edge)
[####################################] 100%
Downloading sysbench revision 78 (edge)
[####################################] 100%
Downloading ubuntu-advantage revision 113 (edge)
[####################################] 100%
Successfully exported charm bundle postgresql-bundle: /home/ubuntu/snap/store-admin/common/export/postgresql-bundle-20241008T083251.tar.gz
```
</details>

#### SNAPs
Usually charms require SNAPs (and some manually pin them). For the manual SNAP exports, follow the official Snap Store Proxy documentation: [Offline Charmhub configuration > Export SNAP](https://documentation.ubuntu.com/snap-store-proxy/en/airgap-charmhub/#export-snap-resources). Data team is shipping the mapping [snap.yaml](https://github.com/canonical/postgresql-bundle/blob/main/releases/latest/) to the published [bundle.yaml](https://github.com/canonical/postgresql-bundle/blob/main/releases/latest/):

> **Warning**: always use snap.yaml and bundle.yaml from the same Git commit (to match each other)!
<details>
<summary><code>store-admin export snaps --from-yaml snaps.yaml</code></summary>

```shell
Downloading grafana-agent revision 51 (latest/stable amd64)
[####################################] 100%
Downloading grafana-agent revision 82 (latest/stable amd64)
[####################################] 100%
Downloading charmed-pgbouncer revision 16 (1/edge amd64)
[####################################] 100%
Downloading charmed-postgresql revision 133 (14/edge amd64)
[####################################] 100%
Downloading canonical-livepatch revision 282 (latest/stable amd64)
[####################################] 100%
Successfully exported snaps:
grafana-agent: /home/ubuntu/snap/store-admin/common/export/grafana-agent-20241008T082122.tar.gz
charmed-pgbouncer: /home/ubuntu/snap/store-admin/common/export/charmed-pgbouncer-20241008T082122.tar.gz
charmed-postgresql: /home/ubuntu/snap/store-admin/common/export/charmed-postgresql-20241008T082122.tar.gz
canonical-livepatch: /home/ubuntu/snap/store-admin/common/export/canonical-livepatch-20241008T082122.tar.gz
```
</details>
### 2. Transfer the binary blobs
Transfer the binary blobs using the way of your choice into the air-gapped environment.
```shell
cp /home/ubuntu/snap/store-admin/common/export/*.tar.gz /media/usb/

...
cp /media/usb/*.tar.gz /var/snap/snap-store-proxy/common/charms-to-push/
```
> **Note**: always check [checksum](https://en.wikipedia.org/wiki/Checksum) for the transferred blobs!
### 3. Import snaps and charms
Import the [snap](https://documentation.ubuntu.com/snap-store-proxy/en/airgap/#importing-pushing-snaps) and [charm](https://documentation.ubuntu.com/snap-store-proxy/en/airgap-charmhub/#import-packages) blobs into local air-gapped CharmHub:
> **Note**: when importing machine charms that depend on a snap for functionality, you must first manually import the required snap.
```shell
sudo snap-store-proxy push-snap /var/snap/snap-store-proxy/common/snaps-to-push/charmed-postgresql-20241008T082122.tar.gz
sudo snap-store-proxy push-charm-bundle /var/snap/snap-store-proxy/common/charms-to-push/postgresql-bundle-20241003T104903.tar.gz
```
> **Note**: when [re-importing](https://documentation.ubuntu.com/snap-store-proxy/en/airgap-charmhub/#import-packages) charms or importing other revisions, make sure to provide the `--push-channel-map`.
### 4. Deploy PostgreSQL
Deploy and operate Juju charms normally:
```shell
juju deploy postgresql
```
[note]
**Note**: All the charms revisions and snap revisions deployed in the air-gapped environment must match the official CharmHub and SnapStore revisions.
Use [the official release notes](/t/11875) as a reference.
[/note]
## Additional resources
* https://docs.ubuntu.com/snap-store-proxy/en/airgap
* https://documentation.ubuntu.com/snap-store-proxy/
* https://documentation.ubuntu.com/snap-store-proxy/en/airgap-charmhub/
* https://ubuntu.com/kubernetes/docs/install-offline
* [Charmed Kubeflow > Install in an airgapped environment](https://charmed-kubeflow.io/docs/install-in-airgapped-environment)
* [Wikipedia > Air gap (networking)](https://en.wikipedia.org/wiki/Air_gap_(networking))
[note]
If you expect having several concurrent connections frequently, it is highly recommended to deploy [PgBouncer](https://charmhub.io/pgbouncer?channel=1/stable) alongside PostgreSQL. For more information, read our explanation about [Connection pooling](/t/15777).
[/note]
Loading

0 comments on commit 3c7c783

Please sign in to comment.