From 3c7c783d61d4bee4ce64c190a9f7d4a78048e4e7 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" <41898282+github-actions[bot]@users.noreply.github.com> Date: Tue, 15 Oct 2024 23:00:00 +0200 Subject: [PATCH] Sync docs from Discourse (#606) Co-authored-by: GitHub Actions <41898282+github-actions[bot]@users.noreply.github.com> --- docs/explanation/e-connection-pooling.md | 17 ++ docs/how-to/h-async-integrate.md | 8 +- docs/how-to/h-async-remove-recover.md | 6 +- docs/how-to/h-async-set-up.md | 21 +- docs/how-to/h-async.md | 30 ++ docs/how-to/h-deploy-airgapped.md | 144 +++++++++ docs/how-to/h-deploy-ec2.md | 264 +++++++++++++++++ docs/how-to/h-deploy-gce.md | 274 ++++++++++++++++++ docs/how-to/h-deploy-lxd.md | 32 +- docs/how-to/h-deploy-maas.md | 8 +- docs/how-to/h-deploy-terraform.md | 3 + ...ur-charm.md => h-development-integrate.md} | 0 docs/how-to/h-enable-alert-rules.md | 21 +- docs/how-to/h-enable-tls.md | 2 +- docs/how-to/h-enable-tracing.md | 88 +++--- docs/how-to/h-external-access.md | 13 + docs/how-to/h-integrate.md | 96 ++++++ docs/how-to/h-manage-applications.md | 58 ---- docs/how-to/h-restore-backup.md | 56 +++- docs/how-to/h-rollback-major.md | 9 - docs/how-to/h-rollback-minor.md | 31 +- docs/how-to/h-upgrade-intro.md | 17 -- docs/how-to/h-upgrade-major.md | 9 - docs/how-to/h-upgrade-minor.md | 36 +-- docs/overview.md | 73 +++-- docs/reference/r-all-releases.md | 32 -- docs/reference/r-releases.md | 101 +++++++ docs/reference/r-revision-467-468.md | 157 ++++++++++ .../r-troubleshooting.md} | 0 docs/tutorial/t-access.md | 162 +++++++++++ docs/tutorial/t-clean-up.md | 4 +- docs/tutorial/t-deploy.md | 110 +------ docs/tutorial/t-enable-tls.md | 3 +- docs/tutorial/t-integrate.md | 12 +- docs/tutorial/t-overview.md | 23 +- 35 files changed, 1516 insertions(+), 404 deletions(-) create mode 100644 docs/explanation/e-connection-pooling.md create mode 100644 docs/how-to/h-async.md create mode 100644 docs/how-to/h-deploy-airgapped.md create mode 100644 docs/how-to/h-deploy-ec2.md create mode 100644 docs/how-to/h-deploy-gce.md rename docs/how-to/{h-integrate-db-with-your-charm.md => h-development-integrate.md} (100%) create mode 100644 docs/how-to/h-external-access.md create mode 100644 docs/how-to/h-integrate.md delete mode 100644 docs/how-to/h-manage-applications.md delete mode 100644 docs/how-to/h-rollback-major.md delete mode 100644 docs/how-to/h-upgrade-intro.md delete mode 100644 docs/how-to/h-upgrade-major.md delete mode 100644 docs/reference/r-all-releases.md create mode 100644 docs/reference/r-releases.md create mode 100644 docs/reference/r-revision-467-468.md rename docs/{how-to/h-troubleshooting.md => reference/r-troubleshooting.md} (100%) create mode 100644 docs/tutorial/t-access.md diff --git a/docs/explanation/e-connection-pooling.md b/docs/explanation/e-connection-pooling.md new file mode 100644 index 0000000000..a8d1a99300 --- /dev/null +++ b/docs/explanation/e-connection-pooling.md @@ -0,0 +1,17 @@ +# Connection pooling + +Connection pooling is a strategy to reduce the amount of active connections and the costs of reopening connections. It requires maintaining a set of persistently opened connections, called a pool, that can be reused by clients. + +Since PostgreSQL spawns separate processes for client connections, it can be beneficial in some use cases to maintain a client side connection pool rather than increase the database connection limits and resource consumption. + +A way to achieve this with Charmed PostgreSQL is by integrating with the [PgBouncer charm](https://charmhub.io/pgbouncer?channel=1/stable). + +## Increasing maximum allowed connections + +If using PgBouncer is not enough to handle the connections load of your application, you can increase the amount of connections that PostgreSQL can open via the [`experimental_max_connections` config parameter](https://charmhub.io/postgresql/configurations#experimental_max_connections). + +[note type="caution"] +**Disclaimer:** Each connection opened by PostgreSQL spawns a new process, which is resource-intensive. Use this option as a last resort. + +[Contact us](/t/11863) for more guidance about your use-case. +[/note] \ No newline at end of file diff --git a/docs/how-to/h-async-integrate.md b/docs/how-to/h-async-integrate.md index 327e8872b8..99d6a347e5 100644 --- a/docs/how-to/h-async-integrate.md +++ b/docs/how-to/h-async-integrate.md @@ -3,9 +3,11 @@ This guide will show you how to integrate a client application with a cross-regional async setup using an example PostgreSQL deployment with two servers: one in Rome and one in Lisbon. ## Prerequisites -* `juju v.3.4.2+` +* Juju `v.3.4.2+` +* Make sure your machine(s) fulfill the [system requirements](/t/11743) +* See [supported target/source model relationships](/t/15412#substrate-dependencies). * A cross-regional async replication setup - * Refer to the page [How to set up clusters](/t/13991) + * See [How to set up clusters](/t/13991) ## Summary * [Configure database endpoints](#configure-database-endpoints) @@ -53,7 +55,7 @@ juju switch app juju deploy postgresql-test-app juju deploy pgbouncer --channel 1/stable -juju integrate postgresql-test-app:first-database pgbouncer +juju integrate postgresql-test-app:database pgbouncer juju integrate pgbouncer db1database ``` diff --git a/docs/how-to/h-async-remove-recover.md b/docs/how-to/h-async-remove-recover.md index 2766485d89..06621e33c4 100644 --- a/docs/how-to/h-async-remove-recover.md +++ b/docs/how-to/h-async-remove-recover.md @@ -3,9 +3,11 @@ This guide will cover how to manage clusters using an example PostgreSQL deployment with two servers: one in Rome and one in Lisbon. ## Prerequisites -* `juju v.3.4.2+` +* Juju `v.3.4.2+` +* Make sure your machine(s) fulfill the [system requirements](/t/11743) +* See [supported target/source model relationships](/t/15412#substrate-dependencies). * A cross-regional async replication setup - * Refer to the page [How to set up clusters](/t/13991) + * See [How to set up clusters](/t/13991) ## Summary * [Switchover](#switchover) diff --git a/docs/how-to/h-async-set-up.md b/docs/how-to/h-async-set-up.md index f5c50262fa..4c8e73b71a 100644 --- a/docs/how-to/h-async-set-up.md +++ b/docs/how-to/h-async-set-up.md @@ -4,6 +4,11 @@ Cross-regional (or multi-server) asynchronous replication focuses on disaster re This guide will show you the basics of initiating a cross-regional async setup using an example PostgreSQL deployment with two servers: one in Rome and one in Lisbon. +## Prerequisites +* Juju `v.3.4.2+` +* Make sure your machine(s) fulfill the [system requirements](/t/11743) +* See [supported target/source model relationships](/t/15412#substrate-dependencies). + ## Summary * [Deploy](#deploy) * [Offer](#offer) @@ -22,10 +27,10 @@ juju add-model rome juju add-model lisbon juju switch rome # active model must correspond to cluster -juju deploy postgresql db1 --channel=14/edge/async-replication --config profile=testing --base ubuntu@22.04 +juju deploy postgresql db1 juju switch lisbon -juju deploy postgresql db2 --channel=14/edge/async-replication --config profile=testing --base ubuntu@22.04 +juju deploy postgresql db2 ``` ## Offer @@ -34,7 +39,7 @@ juju deploy postgresql db2 --channel=14/edge/async-replication --config profile= ```shell juju switch rome -juju offer db1:async-primary async-primary +juju offer db1:replication-offer replication-offer ``` ## Consume @@ -42,22 +47,22 @@ juju offer db1:async-primary async-primary Consume asynchronous replication on planned `Standby` cluster (Lisbon): ```shell juju switch lisbon -juju consume rome.async-primary -juju integrate async-primary db2:async-replica +juju consume rome.replication-offer +juju integrate replication-offer db2:replication ``` ## Promote or switchover a cluster -To define the primary cluster, use the `promote-cluster` action. +To define the primary cluster, use the `create-replication` action. ```shell -juju run -m rome db1/leader promote-cluster +juju run -m rome db1/leader create-replication ``` To switchover and use `lisbon` as the primary instead, run ```shell -juju run -m lisbon db2/leader promote-cluster force-promotion=true +juju run -m lisbon db2/leader promote-to-primary ``` ## Scale a cluster diff --git a/docs/how-to/h-async.md b/docs/how-to/h-async.md new file mode 100644 index 0000000000..71da3b71da --- /dev/null +++ b/docs/how-to/h-async.md @@ -0,0 +1,30 @@ +# Cross-regional async replication + +Cross-regional (or multi-server) asynchronous replication focuses on disaster recovery by distributing data across different servers. + +## Prerequisites +* Juju `v.3.4.2+` +* Make sure your machine(s) fulfill the [system requirements](/t/11743) + +### Substrate dependencies + +The following table shows the source and target controller/model combinations that are currently supported: + +| | AWS | GCP | Azure | +|---|---|:---:|:---:| +| AWS | | | | +| GCP | | ![ check ] | ![ check ] | +| Azure | | ![ check ] | ![ check ] | + +## How-to guides + +* [How to set up clusters for cross-regional async replication](/t/13991) +* [How to integrate with a client application](/t/13992) +* [How to remove or recover a cluster](/t/13994) + * [Switchover](/t/13994#switchover) + * [Detach](/t/13994#detach-a-cluster) + * [Recover](/t/13994#recover-a-cluster) + + +[check]: https://img.shields.io/badge/%E2%9C%93-brightgreen +[cross]: https://img.shields.io/badge/x-white \ No newline at end of file diff --git a/docs/how-to/h-deploy-airgapped.md b/docs/how-to/h-deploy-airgapped.md new file mode 100644 index 0000000000..510cf68b0b --- /dev/null +++ b/docs/how-to/h-deploy-airgapped.md @@ -0,0 +1,144 @@ +# Deploy in an offline or air-gapped environment + +An air-gapped environment refers to a system that does not have access to the public internet. +This guide goes through the special configuration steps for installing Charmed PostgreSQL VM in an air-gapped environment. + +## Requirements + +Canonical does not prescribe how you should set up your specific air-gapped environment. However, it is assumed that it meets the following conditions: + +* A VM/hardware resources available for Juju. +* DNS is configured to the local nameservers. +* [Juju is configured](https://documentation.ubuntu.com/snap-store-proxy/en/airgap-charmhub/#configure-juju) to use local air-gapped services. +* The [`store-admin`](https://snapcraft.io/store-admin) tool is installed and configured. +* [Air-gapped CharmHub](https://documentation.ubuntu.com/snap-store-proxy/en/airgap-charmhub/) is installed and running. +* [Air-gapped Snap Store Proxy](https://documentation.ubuntu.com/snap-store-proxy/) is installed and running. +* Local APT and LXD Images caches are reachable. + +## Air-gapped setup summary + +[1\. Export snaps and charms](#1-export-snaps-and-charms)
+[2\. Transfer binary blobs](#2-transfer-binary-blobs)
+[3\. Import snaps and charms](3-import-snaps-and-charms)
+[4\. Deploy PostgreSQL](#4-deploy-postgresql) + +## Air-gapped day-to-day example + +### 1. Export snaps and charms +Exporting VM SNAPs and Charms and are currently independent processes. The `store-admin` tool is designed to simplify the process. + +Future improvements are planned to the `store-admin` tool so that it could potentially export all necessary SNAP resource(s) from the official SnapStore with Charms simultaneously. Other planned improvements include supporting the export of specific charm and resource by revisions ([PF-5369](https://warthogs.atlassian.net/browse/PF-5369), [PF-5185](https://warthogs.atlassian.net/browse/PF-5185)). + +#### Charms + The necessary charm(s) can be exported as bundle or independently (charm-by-charm). See the Snap Proxy documentation: +* [Offline Charmhub configuration > Export charm bundle](https://documentation.ubuntu.com/snap-store-proxy/en/airgap-charmhub/#export-charm-bundles) +* [Offline Charmhub configuration > Export charms](https://documentation.ubuntu.com/snap-store-proxy/en/airgap-charmhub/#export-charms) + +The bundle export example: + +
+store-admin export bundle postgresql-bundle --channel=14/edge --series=jammy --arch=amd64 + +``` +Downloading postgresql-bundle revision 140 (14/edge) + [####################################] 100% +Downloading data-integrator revision 71 (edge) + [####################################] 100% +Downloading grafana-agent revision 286 (edge) + [####################################] 100% +Downloading landscape-client revision 69 (edge) + [####################################] 100% +Downloading pgbouncer revision 473 (1/edge) + [####################################] 100% +Downloading postgresql revision 487 (14/edge) + [####################################] 100% +Downloading postgresql-test-app revision 256 (edge) + [####################################] 100% +Downloading s3-integrator revision 59 (edge) + [####################################] 100% +Downloading self-signed-certificates revision 200 (edge) + [####################################] 100% +Downloading sysbench revision 78 (edge) + [####################################] 100% +Downloading ubuntu-advantage revision 113 (edge) + [####################################] 100% +Successfully exported charm bundle postgresql-bundle: /home/ubuntu/snap/store-admin/common/export/postgresql-bundle-20241008T083251.tar.gz + +``` +
+ +#### SNAPs +Usually charms require SNAPs (and some manually pin them). For the manual SNAP exports, follow the official Snap Store Proxy documentation: [Offline Charmhub configuration > Export SNAP](https://documentation.ubuntu.com/snap-store-proxy/en/airgap-charmhub/#export-snap-resources). Data team is shipping the mapping [snap.yaml](https://github.com/canonical/postgresql-bundle/blob/main/releases/latest/) to the published [bundle.yaml](https://github.com/canonical/postgresql-bundle/blob/main/releases/latest/): + +> **Warning**: always use snap.yaml and bundle.yaml from the same Git commit (to match each other)! + +
+store-admin export snaps --from-yaml snaps.yaml + +```shell +Downloading grafana-agent revision 51 (latest/stable amd64) + [####################################] 100% +Downloading grafana-agent revision 82 (latest/stable amd64) + [####################################] 100% +Downloading charmed-pgbouncer revision 16 (1/edge amd64) + [####################################] 100% +Downloading charmed-postgresql revision 133 (14/edge amd64) + [####################################] 100% +Downloading canonical-livepatch revision 282 (latest/stable amd64) + [####################################] 100% +Successfully exported snaps: +grafana-agent: /home/ubuntu/snap/store-admin/common/export/grafana-agent-20241008T082122.tar.gz +charmed-pgbouncer: /home/ubuntu/snap/store-admin/common/export/charmed-pgbouncer-20241008T082122.tar.gz +charmed-postgresql: /home/ubuntu/snap/store-admin/common/export/charmed-postgresql-20241008T082122.tar.gz +canonical-livepatch: /home/ubuntu/snap/store-admin/common/export/canonical-livepatch-20241008T082122.tar.gz +``` +
+ +### 2. Transfer the binary blobs + +Transfer the binary blobs using the way of your choice into the air-gapped environment. + +```shell +cp /home/ubuntu/snap/store-admin/common/export/*.tar.gz /media/usb/ + +... +cp /media/usb/*.tar.gz /var/snap/snap-store-proxy/common/charms-to-push/ +``` +> **Note**: always check [checksum](https://en.wikipedia.org/wiki/Checksum) for the transferred blobs! + +### 3. Import snaps and charms + + Import the [snap](https://documentation.ubuntu.com/snap-store-proxy/en/airgap/#importing-pushing-snaps) and [charm](https://documentation.ubuntu.com/snap-store-proxy/en/airgap-charmhub/#import-packages) blobs into local air-gapped CharmHub: + +> **Note**: when importing machine charms that depend on a snap for functionality, you must first manually import the required snap. +```shell +sudo snap-store-proxy push-snap /var/snap/snap-store-proxy/common/snaps-to-push/charmed-postgresql-20241008T082122.tar.gz + +sudo snap-store-proxy push-charm-bundle /var/snap/snap-store-proxy/common/charms-to-push/postgresql-bundle-20241003T104903.tar.gz +``` +> **Note**: when [re-importing](https://documentation.ubuntu.com/snap-store-proxy/en/airgap-charmhub/#import-packages) charms or importing other revisions, make sure to provide the `--push-channel-map`. + +### 4. Deploy PostgreSQL + + Deploy and operate Juju charms normally: +```shell +juju deploy postgresql +``` +[note] +**Note**: All the charms revisions and snap revisions deployed in the air-gapped environment must match the official CharmHub and SnapStore revisions. + +Use [the official release notes](/t/11875) as a reference. +[/note] + +## Additional resources + +* https://docs.ubuntu.com/snap-store-proxy/en/airgap +* https://documentation.ubuntu.com/snap-store-proxy/ +* https://documentation.ubuntu.com/snap-store-proxy/en/airgap-charmhub/ +* https://ubuntu.com/kubernetes/docs/install-offline +* [Charmed Kubeflow > Install in an airgapped environment](https://charmed-kubeflow.io/docs/install-in-airgapped-environment) +* [Wikipedia > Air gap (networking)](https://en.wikipedia.org/wiki/Air_gap_(networking)) + +[note] +If you expect having several concurrent connections frequently, it is highly recommended to deploy [PgBouncer](https://charmhub.io/pgbouncer?channel=1/stable) alongside PostgreSQL. For more information, read our explanation about [Connection pooling](/t/15777). +[/note] \ No newline at end of file diff --git a/docs/how-to/h-deploy-ec2.md b/docs/how-to/h-deploy-ec2.md new file mode 100644 index 0000000000..5eb0b9d975 --- /dev/null +++ b/docs/how-to/h-deploy-ec2.md @@ -0,0 +1,264 @@ +# How to deploy on AWS EC2 + +[Amazon Web Services](https://aws.amazon.com/) is a popular subsidiary of Amazon that provides on-demand cloud computing platforms on a metered pay-as-you-go basis. Access the AWS web console at [console.aws.amazon.com](https://console.aws.amazon.com/). + +## Summary +* [Install AWS and Juju tooling](#install-aws-and-juju-tooling) + * [Authenticate](#authenticate) +* [Bootstrap Juju controller on AWS EC2](#bootstrap-juju-controller-on-aws-ec2) +* [Deploy charms](#deploy-charms) +* [Expose database (optional)](#expose-database-optional) +* [Clean up](#clean-up) + +--- + +## Install AWS and Juju tooling + +Install Juju via snap: +```shell +sudo snap install juju +``` + +Follow the installation guides for: +* [AWs CLI](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html) - the Amazon Web Services CLI + +To check they are all correctly installed, you can run the commands demonstrated below with sample outputs: + +```console +~$ juju version +3.5.4-genericlinux-amd64 + +~$ aws --version +aws-cli/2.13.25 Python/3.11.5 Linux/6.2.0-33-generic exe/x86_64.ubuntu.23 prompt/off +``` +### Authenticate +[Create an IAM account](https://docs.aws.amazon.com/eks/latest/userguide/getting-started-console.html) (or use legacy access keys) to operate AWS EC2: +```shell +mkdir -p ~/.aws && cat <<- EOF > ~/.aws/credentials.yaml +credentials: + aws: + NAME_OF_YOUR_CREDENTIAL: + auth-type: access-key + access-key: SECRET_ACCESS_KEY_ID + secret-key: SECRET_ACCESS_KEY_VALUE +EOF +``` + + + +## Bootstrap Juju controller on AWS EC2 + +Add AWS credentials to Juju: +```shell +juju add-credential aws -f ~/.aws/credentials.yaml +``` +Bootstrap Juju controller ([check all supported configuration options](https://juju.is/docs/juju/amazon-ec2)): +```shell +juju bootstrap aws +``` +[details="Output example"] +```shell +> juju bootstrap aws +Creating Juju controller "aws-us-east-1" on aws/us-east-1 +Looking for packaged Juju agent version 3.5.4 for amd64 +Located Juju agent version 3.5.4-ubuntu-amd64 at https://juju-dist-aws.s3.amazonaws.com/agents/agent/3.5.4/juju-3.5.4-linux-amd64.tgz +Launching controller instance(s) on aws/us-east-1... + - i-0f4615983d113166d (arch=amd64 mem=8G cores=2) +Installing Juju agent on bootstrap instance +Waiting for address +Attempting to connect to 54.226.221.6:22 +Attempting to connect to 172.31.20.34:22 +Connected to 54.226.221.6 +Running machine configuration script... +Bootstrap agent now started +Contacting Juju controller at 54.226.221.6 to verify accessibility... + +Bootstrap complete, controller "aws-us-east-1" is now available +Controller machines are in the "controller" model + +Now you can run + juju add-model +to create a new model to deploy workloads. +``` +[/details] + +You can check the [AWS EC2 instance availability](https://us-east-1.console.aws.amazon.com/ec2/home?region=us-east-1#Instances:instanceState=running) (ensure the right AWS region chosen!): +![image|690x118](upload://putAO5NyHdaeWE6jXI8X1hZHTYv.png) + +Create a new Juju model: +```shell +juju add-model welcome +``` +> (Optional) Increase the debug level if you are troubleshooting charms: +> ```shell +> juju model-config logging-config='=INFO;unit=DEBUG' +> ``` + +## Deploy charms + +The following command deploys PostgreSQL and [Data-Integrator](https://charmhub.io/data-integrator) (the charm to request a test DB): + +```shell +juju deploy postgresql +juju deploy data-integrator --config database-name=test123 +juju relate postgresql data-integrator +``` +Check the status: +```shell +> juju status --relations +Model Controller Cloud/Region Version SLA Timestamp +welcome aws-us-east-1 aws/us-east-1 3.5.4 unsupported 13:33:05+02:00 + +App Version Status Scale Charm Channel Rev Exposed Message +data-integrator active 1 data-integrator latest/stable 41 no +postgresql 14.12 active 1 postgresql 14/stable 468 no + +Unit Workload Agent Machine Public address Ports Message +data-integrator/0* active idle 1 50.19.16.57 +postgresql/0* active idle 0 54.224.41.249 5432/tcp Primary + +Machine State Address Inst id Base AZ Message +0 started 54.224.41.249 i-0f374435695ffc54c ubuntu@22.04 us-east-1b running +1 started 50.19.16.57 i-061e0d10d36c8cffe ubuntu@22.04 us-east-1a running + +Integration provider Requirer Interface Type Message +data-integrator:data-integrator-peers data-integrator:data-integrator-peers data-integrator-peers peer +postgresql:database data-integrator:postgresql postgresql_client regular +postgresql:database-peers postgresql:database-peers postgresql_peers peer +postgresql:restart postgresql:restart rolling_op peer +postgresql:upgrade postgresql:upgrade upgrade peer +``` + +Once deployed, request the credentials for your newly bootstrapped PostgreSQL database. + +For Juju 2.9 use: +```shell +juju run-action --wait data-integrator/leader get-credentials +``` +and for newer Juju 3+ use: +```shell +juju run data-integrator/leader get-credentials +``` + +The output example: +```shell +postgresql: + data: '{"database": "test123", "external-node-connectivity": "true", "requested-secrets": + "[\"username\", \"password\", \"tls\", \"tls-ca\", \"uris\"]"}' + database: test123 + endpoints: 172.31.40.178:5432 + password: rmnk0AO4jICAixCL + uris: postgresql://relation-4:rmnk0AO4jICAixCL@172.31.40.178:5432/test123 + username: relation-4 + version: "14.12" +``` + +At this point, you can access your DB inside AWS using the internal IP address. All further Juju applications will use the database through the internal network: +```shell +> psql postgresql://relation-4:rmnk0AO4jICAixCL@172.31.40.178:5432/test123 +psql (15.6 (Ubuntu 15.6-0ubuntu0.23.10.1), server 14.12 (Ubuntu 14.12-0ubuntu0.22.04.1)) +Type "help" for help. + +test123=> +``` + +From here you can [use/scale/backup/restore/refresh](/t/9707) your newly deployed Charmed PostgreSQL. + +## Expose database (optional) + +If necessary to access DB from outside of AWS (warning: [opening ports to public is risky](https://www.beyondtrust.com/blog/entry/what-is-an-open-port-what-are-the-security-implications)) open the AWS firewall using the simple [juju expose](https://juju.is/docs/juju/juju-expose) functionality: +```shell +juju expose postgresql +``` + +Once exposed, you can connect your database using the same credentials as above (Important: this time use the EC2 Public IP assigned to the PostgreSQL instance): +```shell +> juju status postgresql +... +Unit Workload Agent Machine Public address Ports Message +postgresql/0* active idle 0 54.224.41.249 5432/tcp Primary +... + +> psql postgresql://relation-4:rmnk0AO4jICAixCL@54.224.41.249:5432/test123 +psql (15.6 (Ubuntu 15.6-0ubuntu0.23.10.1), server 14.12 (Ubuntu 14.12-0ubuntu0.22.04.1)) +Type "help" for help. + +test123=> +``` +To close the public access run: +```shell +juju unexpose postgresql +``` +## Clean up + +[note type="caution"] +Always clean AWS resources that are no longer necessary - they could be costly! +[/note] + +To destroy the Juju controller and remove AWS instance (warning: all your data will be permanently removed): +```shell +> juju controllers +Controller Model User Access Cloud/Region Models Nodes HA Version +aws-us-east-1* - admin superuser aws/us-east-1 1 1 none 3.5.4 + +> juju destroy-controller aws-us-east-1 --destroy-all-models --destroy-storage --force +``` + +Next, check and manually delete all unnecessary AWS EC2 instances, to show the list of all your EC2 instances run the following command (make sure the correct region used!): +```shell +aws ec2 describe-instances --region us-east-1 --query "Reservations[].Instances[*].{InstanceType: InstanceType, InstanceId: InstanceId, State: State.Name}" --output table +``` +[details="Output example"] +```shell +------------------------------------------------------- +| DescribeInstances | ++---------------------+----------------+--------------+ +| InstanceId | InstanceType | State | ++---------------------+----------------+--------------+ +| i-0f374435695ffc54c| m7i.large | terminated | +| i-0e1e8279f6b2a08e0| m7i.large | terminated | +| i-061e0d10d36c8cffe| m7i.large | terminated | +| i-0f4615983d113166d| m7i.large | terminated | ++---------------------+----------------+--------------+ +``` +[/details] + +List your Juju credentials: +```shell +> juju credentials +... +Client Credentials: +Cloud Credentials +aws NAME_OF_YOUR_CREDENTIAL +... +``` +Remove AWS EC2 CLI credentials from Juju: +```shell +> juju remove-credential aws NAME_OF_YOUR_CREDENTIAL +``` + +Finally, remove AWS CLI user credentials (to avoid forgetting and leaking): +```shell +rm -f ~/.aws/credentials.yaml +``` + +[note] +If you expect having several concurrent connections frequently, it is highly recommended to deploy [PgBouncer](https://charmhub.io/pgbouncer?channel=1/stable) alongside PostgreSQL. For more information, read our explanation about [Connection pooling](/t/15777). +[/note] \ No newline at end of file diff --git a/docs/how-to/h-deploy-gce.md b/docs/how-to/h-deploy-gce.md new file mode 100644 index 0000000000..5c03bab242 --- /dev/null +++ b/docs/how-to/h-deploy-gce.md @@ -0,0 +1,274 @@ +# How to deploy on GCE + +[Google Compute Engine](https://cloud.google.com/products/compute) is a popular subsidiary of Google that provides on-demand cloud computing platforms on a metered pay-as-you-go basis. Access the GCloud web console at [console.cloud.google.com](https://console.cloud.google.com/compute/instances). + +## Summary +* [Install GCloud and Juju tooling](#install-gcloud-and-juju-tooling) + * [Authenticate](#authenticate) +* [Bootstrap Juju controller on GCE](#bootstrap-juju-controller-on-gce) +* [Deploy charms](#deploy-charms) +* [Expose database (optional)](#expose-database-optional) +* [Clean up](#clean-up) + +--- + +## Install GCloud and Juju tooling + +Install Juju via snap: +```shell +sudo snap install juju +sudo snap install google-cloud-cli --classic +``` + +Check the official the [Google Cloud (GCloud) CLI](https://cloud.google.com/sdk/docs/install) documentation about other installation options. + +To check they are all correctly installed, you can run the commands demonstrated below with sample outputs: + +```console +~$ juju version +3.5.4-genericlinux-amd64 + +~$ gcloud --version +Google Cloud SDK 474.0.0 +... +``` +### Authenticate +Login to GCloud: +```shell +gcloud auth login +``` + +[Create an service IAM account](https://cloud.google.com/iam/docs/service-accounts-create) for Juju to operate GCE: +```shell +> gcloud iam service-accounts create juju-gce-account --display-name="Juju GCE service account" +Created service account [juju-gce-account]. + +> gcloud iam service-accounts list +DISPLAY NAME EMAIL DISABLED +... +Juju GCE service account juju-gce-account@canonical-data-123456.iam.gserviceaccount.com False +... + +> gcloud iam service-accounts keys create sa-private-key.json --iam-account=juju-gce-account@canonical-data-123456.iam.gserviceaccount.com +created key [aaaaaaa....aaaaaaa] of type [json] as [sa-private-key.json] for [juju-gce-account@canonical-data-123456.iam.gserviceaccount.com] + +> gcloud projects add-iam-policy-binding canonical-data-123456 --role=roles/compute.admin \ +--member serviceAccount:juju-gce-account@canonical-data-123456.iam.gserviceaccount.com +``` + +## Bootstrap Juju controller on GCE + +> **Note**: move newly exported GCloud jsonfile into SNAP accessible folder due to the known Juju [issue](https://bugs.launchpad.net/juju/+bug/2007575). +```shell +sudo mv sa-private-key.json /var/snap/juju/common/sa-private-key.json +sudo chmod a+r /var/snap/juju/common/sa-private-key.json +``` + +Add GCE credentials to Juju: + +```shell +> juju add-credential google +... +Enter credential name: juju-gce-account +... + +Auth Types + jsonfile + oauth2 + +Select auth type [jsonfile]: jsonfile + +Enter path to the .json file containing a service account key for your project +Path: /var/snap/juju/common/sa-private-key.json + +Credential "juju-gce-account" added locally for cloud "google". +``` + +Bootstrap Juju controller ([check all supported configuration options](https://juju.is/docs/juju/google-gce)): +```shell +juju bootstrap google gce +``` +[details="Output example"] +```shell +> juju bootstrap google gce +Creating Juju controller "gce" on google/us-east1 +Looking for packaged Juju agent version 3.5.4 for amd64 +Located Juju agent version 3.5.4-ubuntu-amd64 at https://streams.canonical.com/juju/tools/agent/3.5.4/juju-3.5.4-linux-amd64.tgz +Launching controller instance(s) on google/us-east1... + - juju-33f662-0 (arch=amd64 mem=3.6G cores=4) +Installing Juju agent on bootstrap instance +Waiting for address +Attempting to connect to 35.231.246.157:22 +Attempting to connect to 10.142.0.17:22 +Connected to 35.231.246.157 +Running machine configuration script... +Bootstrap agent now started +Contacting Juju controller at 35.231.246.157 to verify accessibility... + +Bootstrap complete, controller "gce" is now available +Controller machines are in the "controller" model + +Now you can run + juju add-model +to create a new model to deploy workloads. +``` +[/details] + +You can check the [GCE instance availability](https://console.cloud.google.com/compute/instances) (ensure the right GCloud project chosen!): + +![image|690x172](upload://4t1wS2BwCYgBfrdR1MdOBbUALJ1.png) + +Create a new Juju model: +```shell +juju add-model welcome +``` +> (Optional) Increase the debug level if you are troubleshooting charms: +> ```shell +> juju model-config logging-config='=INFO;unit=DEBUG' +> ``` + +## Deploy charms + +The following command deploys PostgreSQL and [Data-Integrator](https://charmhub.io/data-integrator) (the charm to request a test DB): + +```shell +juju deploy postgresql +juju deploy data-integrator --config database-name=test123 +juju relate postgresql data-integrator +``` +Check the status: +```shell +> juju status --relations +Model Controller Cloud/Region Version SLA Timestamp +welcome gce google/us-east1 3.5.4 unsupported 23:24:05+02:00 + +App Version Status Scale Charm Channel Rev Exposed Message +data-integrator active 1 data-integrator latest/stable 41 no +postgresql 14.12 active 1 postgresql 14/stable 468 no + +Unit Workload Agent Machine Public address Ports Message +data-integrator/0* active idle 1 34.73.238.173 +postgresql/0* active idle 0 35.237.64.81 5432/tcp Primary + +Machine State Address Inst id Base AZ Message +0 started 35.237.64.81 juju-e2b96f-0 ubuntu@22.04 us-east1-b RUNNING +1 started 34.73.238.173 juju-e2b96f-1 ubuntu@22.04 us-east1-d RUNNING + +Integration provider Requirer Interface Type Message +data-integrator:data-integrator-peers data-integrator:data-integrator-peers data-integrator-peers peer +postgresql:database data-integrator:postgresql postgresql_client regular +postgresql:database-peers postgresql:database-peers postgresql_peers peer +postgresql:restart postgresql:restart rolling_op peer +postgresql:upgrade postgresql:upgrade upgrade peer +``` + +Once deployed, request the credentials for your newly bootstrapped PostgreSQL database. + +For Juju 2.9 use: +```shell +juju run-action --wait data-integrator/leader get-credentials +``` +and for newer Juju 3+ use: +```shell +juju run data-integrator/leader get-credentials +``` + +The output example: +```shell +postgresql: + data: '{"database": "test123", "external-node-connectivity": "true", "requested-secrets": + "[\"username\", \"password\", \"tls\", \"tls-ca\", \"uris\"]"}' + database: test123 + endpoints: 10.142.0.18:5432 + password: SkQ7KpYsyC6PmVVK + uris: postgresql://relation-4:SkQ7KpYsyC6PmVVK@10.142.0.18:5432/test123 + username: relation-4 + version: "14.12" +``` + +At this point, you can access your DB inside GCloud using the internal IP address. All further Juju applications will use the database through the internal network: +```shell +> psql postgresql://relation-4:SkQ7KpYsyC6PmVVK@10.142.0.18:5432/test123 +psql (14.12 (Ubuntu 14.12-0ubuntu0.22.04.1)) +Type "help" for help. + +test123=> +``` + +From here you can [use/scale/backup/restore/refresh](/t/9707) your newly deployed Charmed PostgreSQL. + +## Expose database (optional) + +If necessary to access DB from outside of GCloud (warning: [opening ports to public is risky](https://www.beyondtrust.com/blog/entry/what-is-an-open-port-what-are-the-security-implications)) open the GCloud firewall using the simple [juju expose](https://juju.is/docs/juju/juju-expose) functionality: +```shell +juju expose postgresql +``` + +Once exposed, you can connect your database using the same credentials as above (Important: this time use the GCE Public IP assigned to the PostgreSQL instance): +```shell +> juju status postgresql +... +Unit Workload Agent Machine Public address Ports Message +postgresql/0* active idle 0 35.237.64.81 5432/tcp Primary +... + +> psql postgresql://relation-4:SkQ7KpYsyC6PmVVK@35.237.64.81:5432/test123 +psql (15.6 (Ubuntu 15.6-0ubuntu0.23.10.1), server 14.12 (Ubuntu 14.12-0ubuntu0.22.04.1)) +Type "help" for help. + +test123=> +``` +To close the public access run: +```shell +juju unexpose postgresql +``` +## Clean up + +[note type="caution"] +Always clean GCE resources that are no longer necessary - they could be costly! +[/note] + +To destroy the Juju controller and remove GCE instance (warning: all your data will be permanently removed): +```shell +> juju controllers +Controller Model User Access Cloud/Region Models Nodes HA Version +gce* welcome admin superuser google/us-east1 2 1 none 3.5.4 + +> juju destroy-controller gce --destroy-all-models --destroy-storage --force +``` + +Next, check and manually delete all unnecessary GCloud resources, to show the list of all your GCE instances run the following command (make sure the correct region used!): +```shell +gcloud compute instances list +``` +[details="Output example"] +```shell +NAME ZONE MACHINE_TYPE PREEMPTIBLE INTERNAL_IP EXTERNAL_IP STATUS +juju-33f662-0 us-east1-b n1-highcpu-4 10.142.0.17 35.231.246.157 RUNNING +juju-e2b96f-0 us-east1-b n2d-highcpu-2 10.142.0.18 35.237.64.81 STOPPING +juju-e2b96f-1 us-east1-d n2d-highcpu-2 10.142.0.19 34.73.238.173 STOPPING +``` +[/details] + +List your Juju credentials: +```shell +> juju credentials +... +Client Credentials: +Cloud Credentials +google juju-gce-account +... +``` +Remove GCloud credentials from Juju: +```shell +> juju remove-credential google juju-gce-account +``` + +Finally, remove GCloud jsonfile user credentials (to avoid forgetting and leaking): +```shell +rm -f /var/snap/juju/common/sa-private-key.json +``` + +[note] +If you expect having several concurrent connections frequently, it is highly recommended to deploy [PgBouncer](https://charmhub.io/pgbouncer?channel=1/stable) alongside PostgreSQL. For more information, read our explanation about [Connection pooling](/t/15777). +[/note] \ No newline at end of file diff --git a/docs/how-to/h-deploy-lxd.md b/docs/how-to/h-deploy-lxd.md index 286ca7fc3b..02a1b4323c 100644 --- a/docs/how-to/h-deploy-lxd.md +++ b/docs/how-to/h-deploy-lxd.md @@ -1,18 +1,30 @@ # How to deploy on LXD -For a detailed walkthrough of deploying the charm on LXD, refer to the [Charmed PostgreSQL Tutorial](/t/9707). +This guide assumes you have a running Juju and LXD environment. -For a short summary of the commands on Ubuntu 22.04 LTS, see below: +For a detailed walkthrough of setting up an environment and deploying the charm on LXD, refer to the [Tutorial](/t/9707). + +## Prerequisites +* Canonical LXD 5.12+ +* Fulfil the general [system requirements](/t/11743) + +--- + +[Bootstrap](https://juju.is/docs/juju/juju-bootstrap) a juju controller and create a [model](https://juju.is/docs/juju/juju-add-model) if you haven't already: ```shell -sudo snap install multipass -multipass launch --cpus 4 --memory 8G --disk 30G --name my-vm charm-dev # tune CPU/RAM/HDD accordingly to your needs -multipass shell my-vm +juju bootstrap localhost +juju add-model +``` -juju add-model postgresql -juju deploy postgresql --channel 14/stable +Deploy PostgreSQL: +```shell +juju deploy postgresql ``` +> See the [`juju deploy` documentation](https://juju.is/docs/juju/juju-deploy) for all available options at deploy time. +> +> See the [Configurations tab](https://charmhub.io/postgresql/configurations) for specific PostgreSQL parameters. -The expected result from `juju status` is something similar to: +Sample output of `juju status --watch 1s`: ```shell Model Controller Cloud/Region Version SLA Timestamp postgresql overlord localhost/localhost 2.9.42 unsupported 09:41:53+01:00 @@ -27,4 +39,6 @@ Machine State Address Inst id Series AZ Message 0 started 10.89.49.129 juju-a8a31d-0 jammy Running ``` -Check the [Testing](/t/11773) reference to test your deployment. \ No newline at end of file +[note] +If you expect having several concurrent connections frequently, it is highly recommended to deploy [PgBouncer](https://charmhub.io/pgbouncer?channel=1/stable) alongside PostgreSQL. For more information, read our explanation about [Connection pooling](/t/15777). +[/note] \ No newline at end of file diff --git a/docs/how-to/h-deploy-maas.md b/docs/how-to/h-deploy-maas.md index be5bd8572e..01a80fdf6f 100644 --- a/docs/how-to/h-deploy-maas.md +++ b/docs/how-to/h-deploy-maas.md @@ -4,6 +4,8 @@ This guide aims to provide a quick start to deploying Charmed PostgreSQL on MAAS If you want to deploy PostgreSQL on MAAS in a **production environment**, refer to the official [Bootstrap MAAS Tutorial](https://maas.io/docs/tutorial-bootstrapping-maas) followed by the [Charmed PostgreSQL Tutorial](/t/9707). + + ## Summary * [Bootstrap a Multipass VM](#heading--bootstrap-multipass-vm) * [Configure MAAS](#heading--configure-maas) @@ -159,4 +161,8 @@ For more information, see the docs for [`multipass delete`](https://multipass.ru To completely delete your VM and all its data, run: ```shell multipass delete --purge maas -``` \ No newline at end of file +``` + +[note] +If you expect having several concurrent connections frequently, it is highly recommended to deploy [PgBouncer](https://charmhub.io/pgbouncer?channel=1/stable) alongside PostgreSQL. For more information, read our explanation about [Connection pooling](/t/15777). +[/note] \ No newline at end of file diff --git a/docs/how-to/h-deploy-terraform.md b/docs/how-to/h-deploy-terraform.md index e16facef69..5384f21488 100644 --- a/docs/how-to/h-deploy-terraform.md +++ b/docs/how-to/h-deploy-terraform.md @@ -170,7 +170,10 @@ Destroy complete! Resources: 1 destroyed. ``` --- [note] +If you expect having several concurrent connections frequently, it is highly recommended to deploy [PgBouncer](https://charmhub.io/pgbouncer?channel=1/stable) alongside PostgreSQL. For more information, read our explanation about [Connection pooling](/t/15777). + For more examples of Terraform modules for VM, including PostgreSQL HA and PostgreSQL + PgBouncer, see the other directories in the [`terraform-modules` repository](https://github.com/canonical/terraform-modules/tree/main/modules/machine). [/note] + Feel free to [contact us](/t/11863) if you have any question and [collaborate with us on GitHub](https://github.com/canonical/terraform-modules)! \ No newline at end of file diff --git a/docs/how-to/h-integrate-db-with-your-charm.md b/docs/how-to/h-development-integrate.md similarity index 100% rename from docs/how-to/h-integrate-db-with-your-charm.md rename to docs/how-to/h-development-integrate.md diff --git a/docs/how-to/h-enable-alert-rules.md b/docs/how-to/h-enable-alert-rules.md index 7819fccb9c..5f034055d7 100644 --- a/docs/how-to/h-enable-alert-rules.md +++ b/docs/how-to/h-enable-alert-rules.md @@ -1,16 +1,20 @@ # How to enable COS Alert Rules -Charmed PostgreSQL VM [ships](https://github.com/canonical/postgresql-operator/tree/main/src/prometheus_alert_rules) a pre-configured and pre-enabled list of [Awesome Alert Rules](https://samber.github.io/awesome-prometheus-alerts/): -![Screenshot from 2024-01-18 20-05-52|690x439](upload://j6WSPQ1BzoFzqIg2jm1mTq79SMo.png) +This guide will show how to set up [Pushover](https://pushover.net/) to receive alert notifications from the COS Alert Manager with [Awesome Alert Rules](https://samber.github.io/awesome-prometheus-alerts/). + +Charmed PostgreSQL VM ships a pre-configured and pre-enabled [list of Awesome Alert Rules](https://github.com/canonical/postgresql-operator/tree/main/src/prometheus_alert_rules). -This guide will show how to set up [Pushover](https://pushover.net/) to receive alert notifications from the COS Alert Manager. +
Screenshot of alert rules in the Grafana web interface + +![Screenshot from 2024-01-18 20-05-52|690x439](upload://j6WSPQ1BzoFzqIg2jm1mTq79SMo.png) +
For information about accessing and managing COS Alert Rules, refer to the [COS documentation](https://charmhub.io/cos-lite). ## Prerequisites -* A deployed [Charmed PostgreSQL VM operator](/t/9697) +* A deployed [Charmed PostgreSQL VM operator] * A deployed [`cos-lite` bundle in a Kubernetes environment](https://charmhub.io/topics/canonical-observability-stack/tutorials/install-microk8s) -* Fully configured [COS Monitoring](/t/10600) +* Fully configured [COS Monitoring] ## Enable COS alerts for Pushover The following section is an example of the [Pushover](https://pushover.net/) alerts aggregator. @@ -64,4 +68,9 @@ The image below shows an example of the Pushover web client: The similar way as above, COS alerts can be send to the long [list of supported receivers](https://prometheus.io/docs/alerting/latest/configuration/#receiver-integration-settings). -Do you have questions? [Contact us](/t/11852)! \ No newline at end of file +Do you have questions? [Contact us]! + + +[Contact us]: /t/11852 +[Charmed PostgreSQL VM operator]: /t/9697 +[COS Monitoring]: /t/10600 \ No newline at end of file diff --git a/docs/how-to/h-enable-tls.md b/docs/how-to/h-enable-tls.md index 1b9b31363f..ee279bbec1 100644 --- a/docs/how-to/h-enable-tls.md +++ b/docs/how-to/h-enable-tls.md @@ -11,7 +11,7 @@ This guide will show how to enable TLS using the [`self-signed-certificates` ope [note type="caution"] **[Self-signed certificates](https://en.wikipedia.org/wiki/Self-signed_certificate) are not recommended for a production environment.** -Check [this guide](/t/11664) for an overview of the signed and self-signed certificate charms available. +Check [this guide](/t/11664) for an overview of the TLS certificate charms available. [/note] ## Summary diff --git a/docs/how-to/h-enable-tracing.md b/docs/how-to/h-enable-tracing.md index b1802350b8..da7bdfb927 100644 --- a/docs/how-to/h-enable-tracing.md +++ b/docs/how-to/h-enable-tracing.md @@ -10,8 +10,8 @@ This guide contains the steps to enable tracing with [Grafana Tempo](https://gra To summarize: * [Deploy the Tempo charm in a COS K8s environment](#heading--deploy) -* [Integrate it with the COS charms](#heading--integrate) * [Offer interfaces for cross-model integrations](#heading--offer) +* [Consume and integratre cross-model integrations](#heading--consume) * [View PostgreSQL traces on Grafana](#heading--view) [note type="caution"] @@ -36,31 +36,13 @@ First, switch to the Kubernetes controller where the COS model is deployed: ```shell juju switch : ``` -Then, deploy the [`tempo-k8s`](https://charmhub.io/tempo-k8s) charm: -```shell -juju deploy -n 1 tempo-k8s --channel latest/edge -``` - -

Integrate with the COS charms

- -Integrate `tempo-k8s` with the COS charms as follows: +Then, deploy the dependencies of Tempo following [this tutorial](https://discourse.charmhub.io/t/tutorial-deploy-tempo-ha-on-top-of-cos-lite/15489). In particular, we would want to: +- Deploy the minio charm +- Deploy the s3 integrator charm +- Add a bucket into minio using a python script +- Configure s3 integrator with the minio credentials -```shell -juju integrate tempo-k8s:grafana-dashboard grafana:grafana-dashboard -juju integrate tempo-k8s:grafana-source grafana:grafana-source -juju integrate tempo-k8s:ingress traefik:traefik-route -juju integrate tempo-k8s:metrics-endpoint prometheus:metrics-endpoint -juju integrate tempo-k8s:logging loki:logging -``` -If you would like to instrument traces from the COS charms as well, create the following integrations: -```shell -juju integrate tempo-k8s:tracing alertmanager:tracing -juju integrate tempo-k8s:tracing catalogue:tracing -juju integrate tempo-k8s:tracing grafana:tracing -juju integrate tempo-k8s:tracing loki:tracing -juju integrate tempo-k8s:tracing prometheus:tracing -juju integrate tempo-k8s:tracing traefik:tracing -``` +Finally, deploy and integrate with Tempo HA in [a monolithic setup](https://discourse.charmhub.io/t/tutorial-deploy-tempo-ha-on-top-of-cos-lite/15489#heading--deploy-monolithic-setup).

Offer interfaces

@@ -69,7 +51,7 @@ Next, offer interfaces for cross-model integrations from the model where Charmed To offer the Tempo integration, run ```shell -juju offer tempo-k8s:tracing +juju offer :tracing ``` Then, switch to the Charmed PostgreSQL model, find the offers, and integrate (relate) with them: @@ -85,45 +67,59 @@ Below is a sample output where `k8s` is the K8s controller name and `cos` is the ```shell Store URL Access Interfaces -k8s admin/cos.tempo-k8s admin tracing:tracing +k8s admin/cos.tempo admin tracing:tracing ``` Next, consume this offer so that it is reachable from the current model: ```shell -juju consume k8s:admin/cos.tempo-k8s +juju consume k8s:admin/cos.tempo ``` -Relate Charmed PostgreSQL with the above consumed interface: +

Consume interfaces

+First, deploy [Grafana Agent](https://charmhub.io/grafana-agent) from the `latest/edge` channel. ```shell -juju integrate postgresql:tracing tempo-k8s:tracing +juju deploy grafana-agent --channel latest/edge +``` + +Then, integrate Grafana Agent with Charmed PostgreSQL: +```shell +juju integrate postgresql:cos-agent grafana-agent:cos-agent +``` + +Finally, integrate Grafana Agent with the consumed interface from the previous section: +```shell +juju integrate grafana-agent:tracing tempo:tracing ``` Wait until the model settles. The following is an example of the `juju status --relations` on the Charmed PostgreSQL model: ```shell Model Controller Cloud/Region Version SLA Timestamp -database lxd localhost/localhost 3.4.3 unsupported 19:32:36Z - -SAAS Status Store URL -tempo-k8s active k8s admin/cos.tempo-k8s +database lxd localhost/localhost 3.5.4 unsupported 21:43:34Z -App Version Status Scale Charm Channel Rev Exposed Message -postgresql 14.11 active 1 postgresql 14/candidate 422 no +SAAS Status Store URL +tempo active uk8s admin/cos.tempo -Unit Workload Agent Machine Public address Ports Message -postgresql/1* active idle 1 10.205.193.156 5432/tcp Primary +App Version Status Scale Charm Channel Rev Exposed Message +grafana-agent blocked 1 grafana-agent latest/edge 286 no Missing ['grafana-cloud-config']|['grafana-dashboards-provider']|['logging-consumer']|['send-remote-write'] for cos-a... +postgresql 14.13 active 1 postgresql 0 no -Machine State Address Inst id Base AZ Message -1 started 10.205.193.156 juju-ceca46-1 ubuntu@22.04 Running +Unit Workload Agent Machine Public address Ports Message +postgresql/0* active idle 0 10.205.193.87 5432/tcp Primary + grafana-agent/0* blocked idle 10.205.193.87 Missing ['grafana-cloud-config']|['grafana-dashboards-provider']|['logging-consumer']|['send-remote-write'] for cos-a... -Integration provider Requirer Interface Type Message -postgresql:database-peers postgresql:database-peers postgresql_peers peer -postgresql:restart postgresql:restart rolling_op peer -postgresql:upgrade postgresql:upgrade upgrade peer -tempo-k8s:tracing postgresql:tracing tracing regular +Machine State Address Inst id Base AZ Message +0 started 10.205.193.87 juju-1fee5d-0 ubuntu@22.04 Running +Integration provider Requirer Interface Type Message +grafana-agent:peers grafana-agent:peers grafana_agent_replica peer +postgresql:cos-agent grafana-agent:cos-agent cos_agent subordinate +postgresql:database-peers postgresql:database-peers postgresql_peers peer +postgresql:restart postgresql:restart rolling_op peer +postgresql:upgrade postgresql:upgrade upgrade peer +tempo:tracing grafana-agent:tracing tracing regular ``` [note] @@ -138,4 +134,4 @@ Below is a screenshot demonstrating a Charmed PostgreSQL trace: ![Example PostgreSQL trace with Grafana Tempo|690x382](upload://2ts8eOnNODTJnFrfeGGD20Lj5lR.jpeg) -Feel free to read through the [Tempo documentation](https://discourse.charmhub.io/t/tempo-k8s-docs-index/14005) at your leisure to explore its deployment and its integrations. \ No newline at end of file +Feel free to read through the [Tempo HA documentation](https://discourse.charmhub.io/t/charmed-tempo-ha/15531) at your leisure to explore its deployment and its integrations. \ No newline at end of file diff --git a/docs/how-to/h-external-access.md b/docs/how-to/h-external-access.md new file mode 100644 index 0000000000..fe51d358aa --- /dev/null +++ b/docs/how-to/h-external-access.md @@ -0,0 +1,13 @@ +# How to connect DB from outside of DB LAN + +## External application (non-Juju) + +[u]Use case[/u]: the client application is a non-Juju application outside of Juju / DB LAN. + +There are many possible ways to connect the Charmed PostgreSQL database from outside of the LAN the DB cluster is located. The available options are heavily depend on the cloud/hardware/virtualization in use. One of the possible options is to use [virtual IP addresses (VIP)](https://en.wikipedia.org/wiki/Virtual_IP_address) which the charm PgBouncer provides with assist of the charm/interface `hacluster`. Please follow the [PgBouncer documentation](https://charmhub.io/pgbouncer/docs/h-external-access?channel=1/stable) for such configuration. + +## External relation (Juju) + +[u]Use case[/u]: the client application is a Juju application outside of DB deployment (e.g. hybrid Juju deployment with different VM clouds/controllers). + +In this case the the cross-controllers-relation is necessary. Please [contact](/t/11863) Data team to discuss the possible option for your use case. \ No newline at end of file diff --git a/docs/how-to/h-integrate.md b/docs/how-to/h-integrate.md new file mode 100644 index 0000000000..10e07bc872 --- /dev/null +++ b/docs/how-to/h-integrate.md @@ -0,0 +1,96 @@ +[note] +**Note**: All commands are written for `juju >= v.3.0` + +If you are using an earlier version, check the [Juju 3.0 Release Notes](https://juju.is/docs/juju/roadmap#heading--juju-3-0-0---22-oct-2022). +[/note] + +# How to integrate with another application + +[Integrations](https://juju.is/docs/juju/relation) (formerly “relations”) are connections between two applications with compatible endpoints. These connections simplify the creation and management of users, passwords, and other shared data. + +This guide shows how to integrate Charmed PostgreSQL with both charmed and non-charmed applications. + +> For developer information about how to integrate your own charmed application with PostgreSQL, see [Development > How to integrate with your charm](/t/11865). + +## Summary +* [Integrate with a charmed application](#integrate-with-a-charmed-application) + * [Modern interface](#modern-interface) + * [Legacy interface](#legacy-interface) +* [Integrate with a non-charmed application](#integrate-with-a-non-charmed-application) +* [Rotate application passwords](#rotate-application-passwords) + +--- + +## Integrate with a charmed application + +Integrations with charmed applications are supported via the modern [`postgresql_client`](https://github.com/canonical/charm-relation-interfaces/blob/main/interfaces/postgresql_client/v0/README.md) interface, and the legacy `psql` interface from the [original version](https://launchpad.net/postgresql-charm) of the charm. + +> You can see which existing charms are compatible with PostgreSQL in the [Integrations](https://charmhub.io/postgresql/integrations) tab. + +### Modern `postgresql_client` interface +To integrate with a charmed application that supports the `postgresql_client` interface, run +```shell +juju integrate postgresql +``` + +To remove the integration, run +```shell +juju remove-relation postgresql +``` + +### Legacy `pgsql` interface +[note type="caution"] +Note that this interface is **deprecated**. +See more information in [Explanation > Legacy charm](/t/10690). +[/note] + +To integrate via the legacy interface, run + ```shell +juju integrate postgresql:db +``` + +Extended permissions can be requested using the `db-admin` endpoint: +```shell +juju integrate postgresql:db-admin +``` + +## Integrate with a non-charmed application + +To integrate with an application outside of Juju, you must use the [`data-integrator` charm](https://charmhub.io/data-integrator) to create the required credentials and endpoints. + +Deploy `data-integrator`: +```shell +juju deploy data-integrator --config database-name= +``` + +Integrate with PostgreSQL: +```shell +juju integrate data-integrator postgresql +``` + +Use the `get-credentials` action to retrieve credentials from `data-integrator`: +```shell +juju run data-integrator/leader get-credentials +``` + +## Rotate application passwords +To rotate the passwords of users created for integrated applications, the integration should be removed and integrated again. This process will generate a new user and password for the application. + +```shell +juju remove-relation postgresql +juju integrate postgresql +``` +>`` can be `data-integrator` in the case of connecting with a non-charmed application. + +### Internal operator user +The operator user is used internally by the Charmed PostgreSQL application. The `set-password` action can be used to rotate its password. + +To set a specific password for the operator user, run +```shell +juju run postgresql/leader set-password password= +``` + +To randomly generate a password for the `operator` user, run +```shell +juju run postgresql/leader set-password +``` \ No newline at end of file diff --git a/docs/how-to/h-manage-applications.md b/docs/how-to/h-manage-applications.md deleted file mode 100644 index 2c4b57ca37..0000000000 --- a/docs/how-to/h-manage-applications.md +++ /dev/null @@ -1,58 +0,0 @@ -[note] -**Note**: All commands are written for `juju >= v.3.0` - -If you are using an earlier version, check the [Juju 3.0 Release Notes](https://juju.is/docs/juju/roadmap#heading--juju-3-0-0---22-oct-2022). -[/note] - -# How to manage client applications - -[Integrations](https://juju.is/docs/juju/relation) (formerly “relations”) are connections between two applications with compatible endpoints. These connections simplify the creation and management of users, passwords, and other shared data. - -## Create an integration - -**Integrations with new applications are supported via the [postgresql_client](https://github.com/canonical/charm-relation-interfaces/blob/main/interfaces/postgresql_client/v0/README.md) interface.** - -To create an integration, run -```shell -juju integrate postgresql -``` - -To remove an integration to an application: - -```shell -juju remove-relation postgresql -``` - -### Legacy `pgsql` interface -We have also added support for the database legacy relation from the [original version](https://launchpad.net/postgresql-charm) of the charm via the `pgsql` interface. Note that **this interface is deprecated**. - - ```shell -juju integrate postgresql:db -``` - -Extended permissions can be requested using the `db-admin` endpoint: -```shell -juju integrate postgresql:db-admin -``` - - -## Rotate application passwords -To rotate the passwords of users created for integrated applications, the integration should be removed and integrated again. This process will generate a new user and password for the application. - -```shell -juju remove-relation postgresql -juju integrate postgresql -``` - -### Internal operator user -The operator user is used internally by the Charmed PostgreSQL VM Operator. The `set-password` action can be used to rotate its password. - -To set a specific password for the operator user, run -```shell -juju run postgresql/leader set-password password= -``` - -To randomly generate a password for the `operator` user, run -```shell -juju run postgresql/leader set-password -``` \ No newline at end of file diff --git a/docs/how-to/h-restore-backup.md b/docs/how-to/h-restore-backup.md index b6ba3402d7..c1bbb00c07 100644 --- a/docs/how-to/h-restore-backup.md +++ b/docs/how-to/h-restore-backup.md @@ -4,18 +4,25 @@ If you are using an earlier version, check the [Juju 3.0 Release Notes](https://juju.is/docs/juju/roadmap#heading--juju-3-0-0---22-oct-2022). [/note] -# How to restore a local backup +# How to restore a local backup This is a guide on how to restore a locally made backup. To restore a backup that was made from a *different* cluster, (i.e. cluster migration via restore), see [How to migrate cluster using backups](/t/charmed-postgresql-how-to-migrate-cluster-via-restore/9691?channel=14/stable): - ## Prerequisites - Deployments have been [scaled-down](/t/charmed-postgresql-how-to-manage-units/9689?channel=14/stable) to a single PostgreSQL unit (scale it up after the backup is restored) - Access to S3 storage - [Configured settings for S3 storage](/t/charmed-postgresql-how-to-configure-s3/9681?channel=14/stable) -- [Existing backups in your S3storage](/t/charmed-postgresql-how-to-create-and-list-backups/9683?channel=14/stable) +- [Existing backups in your S3 storage](/t/charmed-postgresql-how-to-create-and-list-backups/9683?channel=14/stable) +- [Point-in-time recovery](#point-in-time-recovery) requires the following PostgreSQL charm revisions: + - 334+ for `arm64` + - 33+ for `amd64` + +## Summary +* [List backups](#list-backups) +* [Point-in-time recovery](#point-in-time-recovery) +* [Restore backup](#restore-backup) --- @@ -26,16 +33,49 @@ juju run postgresql/leader list-backups ``` This should show your available backups like in the sample output below: + ```shell - backups: |- - backup-id | backup-type | backup-status - ---------------------------------------------------- - YYYY-MM-DDTHH:MM:SSZ | physical | finished +list-backups: |- + Storage bucket name: canonical-postgres + Backups base path: /test/backup/ + + backup-id | action | ... | timeline + --------------------------------------------------------------------------- + 2024-07-22T13:11:56Z | full backup | ... | 1 + 2024-07-22T14:12:45Z | incremental backup | ... | 1 + 2024-07-22T15:34:24Z | restore | ... | 2 + 2024-07-22T16:26:48Z | incremental backup | ... | 2 + 2024-07-22T17:17:59Z | full | ... | 2 + 2024-07-22T18:05:32Z | restore | ... | 3 ``` + +Below is a complete list of parameters shown for each backup/restore operation: +* `backup-id`: unique identifier of the backup. +* `action`: indicates the action performed by the user through one of the charm action; can be any of full backup, incremental backup, differential backup or restore. +* `status`: either finished (successfully) or failed. +* `reference-backup-id` +* `LSN start/stop`: a database specific number (or timestamp) to identify its state. +* `start-time`: records start of the backup operation. +* `finish-time`: records end of the backup operation. +* `backup-path`: path of the backup related files in the S3 repository. +* `timeline`: number which identifies different branches in the database transactions history; every time a restore or PITR is made, this number is incremented by 1. + +## Point-in-time recovery +Point-in-time recovery (PITR) is a PostgreSQL feature that enables restorations to the database state at specific points in time. + +After performing a PITR in a PostgreSQL cluster, a new timeline is created to track from the point to where the database was restored. They can be tracked via the `timeline` parameter in the `list-backups` output. + ## Restore backup -To restore a backup from that list, run the `restore` command and pass the corresponding `backup-id`: +To restore a backup from that list, run the `restore` command and pass the parameter corresponding to the backup type. + +When the user needs to restore a specific backup that was made, they can use the `backup-id` that is listed in the `list-backups` output. ```shell juju run postgresql/leader restore backup-id=YYYY-MM-DDTHH:MM:SSZ ``` +However, if the user needs to restore to a specific point in time between different backups (e.g. to restore only specific transactions made between those backups), they can use the `restore-to-time` parameter to pass a timestamp related to the moment they want to restore. + ```shell +juju run postgresql/leader restore restore-to-time="YYYY-MM-DDTHH:MM:SSZ" +``` + Your restore will then be in progress. \ No newline at end of file diff --git a/docs/how-to/h-rollback-major.md b/docs/how-to/h-rollback-major.md deleted file mode 100644 index 171cde56e7..0000000000 --- a/docs/how-to/h-rollback-major.md +++ /dev/null @@ -1,9 +0,0 @@ -# Perform a major rollback - -**Example**: PostgreSQL 15 -> PostgreSQL 14 - -[note type="negative"] -Currently, this charm only supports PostgreSQL 14. Therefore, only [minor rollbacks](/t/12090) are possible. - -Canonical is **NOT** planning to support in-place rollbacks for the major PostgreSQL version change as the old PostgreSQL cluster installation will stay nearby and can be reused for the rollback. -[/note] \ No newline at end of file diff --git a/docs/how-to/h-rollback-minor.md b/docs/how-to/h-rollback-minor.md index f64a9cdfe5..5989080da7 100644 --- a/docs/how-to/h-rollback-minor.md +++ b/docs/how-to/h-rollback-minor.md @@ -1,38 +1,31 @@ -# Minor Rollback - -**Example**: PostgreSQL 14.9 -> PostgreSQL 14.8
-(including simple charm revision bump: from revision 43 to revision 42) - -[note type="caution"] -**Warning:** Do NOT trigger `rollback` during the running `upgrade` action! It may cause an unpredictable PostgreSQL cluster state! -[/note] - [note] **Note**: All commands are written for `juju >= v.3.0` -If you are using an earlier version, be aware that: - - - `juju run` replaces `juju run-action --wait` in `juju v.2.9` - - `juju integrate` replaces `juju relate` and `juju add-relation` in `juju v.2.9` - -For more information, check the [Juju 3.0 Release Notes](https://juju.is/docs/juju/roadmap#heading--juju-3-0-0---22-oct-2022). +If you are using an earlier version, check the [Juju 3.0 Release Notes](https://juju.is/docs/juju/roadmap#heading--juju-3-0-0---22-oct-2022). [/note] -## Manual rollback +# Perform a minor rollback +**Example**: PostgreSQL 14.9 -> PostgreSQL 14.8
+(including simple charm revision bump: from revision 43 to revision 42) + After a `juju refresh`, if there are any version incompatibilities in charm revisions, its dependencies, or any other unexpected failure in the upgrade process, the process will be halted and enter a failure state. Even if the underlying PostgreSQL cluster continues to work, it’s important to roll back the charm to a previous revision so that an update can be attempted after further inspection of the failure. -## Minor rollback steps +[note type="caution"] +**Warning:** Do NOT trigger `rollback` during the running `upgrade` action! It may cause an unpredictable PostgreSQL cluster state! +[/note] + +## Summary 1. **Prepare** the Charmed PostgreSQL VM application for the in-place rollback. 2. **Rollback**. Once started, all units in a cluster will be executed sequentially. The rollback will be aborted (paused) if the unit rollback has failed. 3. **Check**. Make sure the charm and cluster are in a healthy state again. -To execute a rollback, we use a similar procedure to the upgrade. The difference is the charm revision to upgrade to. In this guide's example, we will refresh the charm back to revision `182`. - ## Step 1: Prepare +To execute a rollback, we use a similar procedure to the upgrade. The difference is the charm revision to upgrade to. In this guide's example, we will refresh the charm back to revision `182`. + It is necessary to re-run `pre-upgrade-check` action on the leader unit, to enter the upgrade recovery state: ```shell juju run postgresql/leader pre-upgrade-check diff --git a/docs/how-to/h-upgrade-intro.md b/docs/how-to/h-upgrade-intro.md deleted file mode 100644 index 8e1e569828..0000000000 --- a/docs/how-to/h-upgrade-intro.md +++ /dev/null @@ -1,17 +0,0 @@ -# Charmed PostgreSQL Upgrade - -Please choose the appropriate upgrade/rollback tutorial. - -Migration: - -* [Major upgrade](/t/12087), e.g. PostgreSQL 14 -> PostgreSQL 15. - -* [Major rollback](/t/12088), e.g. PostgreSQL 15 -> PostgreSQL 14. - -In-place minor upgrade: - -* [Minor upgrade](/t/12089), e.g. PostgreSQL 14.8 -> PostgreSQL 14.9
-(including charm revision bump 42 -> 43). - -* [Minor rollback](/t/12090), e.g. PostgreSQL 14.9 -> PostgreSQL 14.8
-(including charm revision return 43 -> 42). \ No newline at end of file diff --git a/docs/how-to/h-upgrade-major.md b/docs/how-to/h-upgrade-major.md deleted file mode 100644 index 78e845bbde..0000000000 --- a/docs/how-to/h-upgrade-major.md +++ /dev/null @@ -1,9 +0,0 @@ -# Perform a major upgrade - -**Example**: PostgreSQL 14 -> PostgreSQL 15 - -[note type="negative"] -Currently, this charm only supports PostgreSQL 14. Therefore, only [minor upgrades](/t/12089) are possible. - -Canonical is **NOT** planning to support in-place upgrades for the major version change. The new PostgreSQL cluster will have to be installed nearby, and the data will be copied from the old to the new installation. After announcing the next PostgreSQL major version support, the appropriate manual will be published here. -[/note] \ No newline at end of file diff --git a/docs/how-to/h-upgrade-minor.md b/docs/how-to/h-upgrade-minor.md index cb5130077d..ffcf34ff92 100644 --- a/docs/how-to/h-upgrade-minor.md +++ b/docs/how-to/h-upgrade-minor.md @@ -1,17 +1,28 @@ +[note] +**Note**: All commands are written for `juju >= v.3.0` + +If you are using an earlier version, check the [Juju 3.0 Release Notes](https://juju.is/docs/juju/roadmap#heading--juju-3-0-0---22-oct-2022). +[/note] + # Perform a minor upgrade **Example**: PostgreSQL 14.8 -> PostgreSQL 14.9
(including simple charm revision bump: from revision 193 to revision 196). -[note] This guide is part of [Charmed PostgreSQL Upgrades](/t/12086). Please refer to this page for more information and an overview of the content. -[/note] -[note] -**Note**: All commands are written for `juju >= v.3.0` -If you are using an earlier version, check the [Juju 3.0 Release Notes](https://juju.is/docs/juju/roadmap#heading--juju-3-0-0---22-oct-2022). -[/note] -## Before upgrading +## Summary +- [**Pre-upgrade checks**](#pre-upgrade-checks): Important information to consider before starting an upgrade. +- [**1. Collect**](#step-1-collect) all necessary pre-upgrade information. It will be necessary for a rollback, if needed. **Do not skip this step**; better to be safe than sorry! +- [**2. Prepare**](#step-2-prepare) your Charmed PostgreSQL Juju application for the in-place upgrade. See the step details for all technical details executed by charm here. +- [**3. Upgrade**](#step-3-upgrade). Once started, all units in a cluster will be executed sequentially. The upgrade will be aborted (paused) if the unit upgrade has failed. +- [**4. (Optional) Consider a rollback**](#step-4-rollback-optional) in case of disaster. + - Please [inform us](/t/11863) about your case scenario troubleshooting to trace the source of the issue and prevent it in the future. +- [**Post-upgrade check**](#step-5-post-upgrade-check). Make sure all units are in the proper state and the cluster is healthy. + +--- + +## Pre-upgrade checks Before performing a minor PostgreSQL upgrade, there are some important considerations to take into account: * Concurrency with other operations during the upgrade * Backing up your data @@ -38,15 +49,6 @@ Guides on how to configure backups with S3-compatible storage can be found [here This will ensure minimal service disruption, if any. -## Minor upgrade steps -Here is a summary of the steps to perform a minor upgrade.: - -1. **Collect** all necessary pre-upgrade information. It will be necessary for a rollback, if needed. Do NOT skip this step, it is better to be safe than sorry! -2. **Prepare** your Charmed PostgreSQL Juju application for the in-place upgrade. See the step description below for all technical details executed by charm here. -3. **Upgrade**. Once started, all units in a cluster will be executed sequentially. The upgrade will be aborted (paused) if the unit upgrade has failed. -4. (optional) Consider a [**rollback**](/t/12090) in case of disaster. Please inform and include us in your case scenario troubleshooting to trace the source of the issue and prevent it in the future. [Contact us](/t/11863)! -5. Do a post-upgrade **check**. Make sure all units are in the proper state and the cluster is healthy. - ## Step 1: Collect [note] @@ -170,7 +172,7 @@ The step must be skipped if the upgrade went well! Although the underlying PostgreSQL Cluster continues to work, it’s important to roll back the charm to a previous revision so that an update can be attempted after further inspection of the failure. Please switch to the dedicated [minor rollback](/t/12090) tutorial if necessary. -## Step 5: Check +## Post-upgrade check Future [improvements are planned](https://warthogs.atlassian.net/browse/DPE-2621) to check the state of a pod/cluster on a low level. diff --git a/docs/overview.md b/docs/overview.md index f3f462c049..b2a68da5e3 100644 --- a/docs/overview.md +++ b/docs/overview.md @@ -1,4 +1,4 @@ -# Charmed PostgreSQL Documentation +# Charmed PostgreSQL documentation Charmed PostgreSQL is an open-source software operator designed to deploy and operate object-relational databases on IAAS/VM. It packages the powerful database management system [PostgreSQL](https://www.postgresql.org/) into a charmed operator for deployment with [Juju](https://juju.is/docs/juju). @@ -8,7 +8,7 @@ Charmed PostgreSQL meets the need of deploying PostgreSQL in a structured and co This charmed operator is made for anyone looking for a comprehensive database management interface, whether for operating a complex production environment or simply as a playground to learn more about databases and charms. -[note type="positive"] +[note] This operator is built for **IAAS/VM**. For deployments in **Kubernetes** environments, see [Charmed PostgreSQL K8s](https://charmhub.io/postgresql-k8s). @@ -47,51 +47,53 @@ PostgreSQL is a trademark or registered trademark of PostgreSQL Global Developme |--------|--------|-------------| | 1 | tutorial | [Tutorial]() | | 2 | t-overview | [Overview](/t/9707) | -| 2 | t-set-up | [1. Set up the environment](/t/9709) | +| 2 | t-set-up | [1. Set up environment](/t/9709) | | 2 | t-deploy | [2. Deploy PostgreSQL](/t/9697) | -| 2 | t-scale | [3. Scale replicas](/t/9705) | -| 2 | t-manage-passwords | [4. Manage passwords](/t/9703) | -| 2 | t-integrate | [5. Integrate with other applications](/t/9701) | -| 2 | t-enable-tls | [6. Enable TLS encryption](/t/9699) | -| 2 | t-clean-up | [7. Clean up environment](/t/9695) | -| 1 | how-to | [How-to guides]() | -| 2 | h-set-up | [Set up]() | -| 3 | h-deploy-lxd | [Deploy on LXD](/t/11861) | -| 3 | h-deploy-maas | [Deploy on MAAS](/t/14293) | -| 3 | h-deploy-terraform | [Deploy via Terraform](/t/14916) | -| 3 | h-scale | [Scale replicas](/t/9689) | -| 3 | h-enable-tls | [Enable TLS](/t/9685) | -| 3 | h-manage-applications | [Manage client applications](/t/9687) | -| 2 | h-backups | [Back up and restore]() | +| 2 | t-access| [3. Access PostgreSQL](/t/15798) | +| 2 | t-scale | [4. Scale replicas](/t/9705) | +| 2 | t-manage-passwords | [5. Manage passwords](/t/9703) | +| 2 | t-integrate | [6. Integrate with other applications](/t/9701) | +| 2 | t-enable-tls | [7. Enable TLS encryption](/t/9699) | +| 2 | t-clean-up | [8. Clean up environment](/t/9695) | +| 1 | how-to | [How to]() | +| 2 | h-deploy | [Deploy]() | +| 3 | h-deploy-lxd | [LXD](/t/11861) | +| 3 | h-deploy-maas | [MAAS](/t/14293) | +| 3 | h-deploy-ec2 | [AWS EC2](/t/15703) | +| 3 | h-deploy-gce | [GCE](/t/15722) | +| 3 | h-deploy-terraform | [Terraform](/t/14916) | +| 3 | h-deploy-airgapped | [Air-gapped](/t/15746) | +| 2 | h-integrate | [Integrate with another application](/t/9687) | +| 2 | h-external-access | [External access](/t/15802) | +| 2 | h-scale | [Scale replicas](/t/9689) | +| 2 | h-enable-tls | [Enable TLS](/t/9685) | +| 2 | h-backup | [Back up and restore]() | | 3 | h-configure-s3-aws | [Configure S3 AWS](/t/9681) | | 3 | h-configure-s3-radosgw | [Configure S3 RadosGW](/t/10313) | | 3 | h-create-backup | [Create a backup](/t/9683) | | 3 | h-restore-backup | [Restore a backup](/t/9693) | | 3 | h-manage-backup-retention | [Manage backup retention](/t/14249) | | 3 | h-migrate-cluster | [Migrate a cluster](/t/9691) | -| 2 | h-monitor | [Monitor (COS)]() | +| 2 | h-monitor | [Monitoring (COS)]() | | 3 | h-enable-monitoring | [Enable monitoring](/t/10600) | -| 3 | h-enable-alert-rules | [Enable Alert Rules](/t/13084) | +| 3 | h-enable-alert-rules | [Enable alert rules](/t/13084) | | 3 | h-enable-tracing | [Enable tracing](/t/14521) | -| 2 | h-upgrade | [Upgrade]() | -| 3 | h-upgrade-intro | [Overview](/t/12086) | -| 3 | h-upgrade-major | [Perform a major upgrade](/t/12087) | -| 3 | h-rollback-major | [Perform a major rollback](/t/12088) | +| 2 | h-upgrade | [Minor upgrades]() | | 3 | h-upgrade-minor | [Perform a minor upgrade](/t/12089) | | 3 | h-rollback-minor | [Perform a minor rollback](/t/12090) | -| 2 | h-integrate-your-charm | [Integrate with your charm]() | -| 3 | h-integrate-db-with-your-charm | [Integrate a database with your charm](/t/11865) | -| 3 | h-migrate-pgdump | [Migrate data via pg_dump](/t/12163) | -| 3 | h-migrate-backup-restore | [Migrate data via backup/restore](/t/12164) | -| 2 | h-async | [Cross-regional async replication]() | +| 2 | h-async | [Cross-regional async replication](/t/15412) | | 3 | h-async-set-up | [Set up clusters](/t/13991) | | 3 | h-async-integrate | [Integrate with a client app](/t/13992) | | 3 | h-async-remove-recover | [Remove or recover a cluster](/t/13994) | | 2 | h-enable-plugins-extensions | [Enable plugins/extensions](/t/10906) | +| 2 | h-development| [Development]() | +| 3 | h-development-integrate | [Integrate with your charm](/t/11865) | +| 3 | h-migrate-pgdump | [Migrate data via pg_dump](/t/12163) | +| 3 | h-migrate-backup-restore | [Migrate data via backup/restore](/t/12164) | | 1 | reference | [Reference]() | | 2 | r-overview | [Overview](/t/13976) | -| 2 | r-releases | [Release Notes]() | -| 3 | r-all-releases | [All releases](/t/11875) | +| 2 | r-releases | [Release Notes](/t/11875) | +| 3 | r-revision-467-468 | [Revision 467/468](/t/15378) | | 3 | r-revision-429-430 | [Revision 429/430](/t/14067) | | 3 | r-revision-363 | [Revision 363](/t/13124) | | 3 | r-revision-351 | [Revision 351](/t/12823) | @@ -100,7 +102,7 @@ PostgreSQL is a trademark or registered trademark of PostgreSQL Global Developme | 2 | r-system-requirements | [System requirements](/t/11743) | | 2 | r-software-testing | [Software testing](/t/11773) | | 2 | r-performance | [Performance and resources](/t/11974) | -| 2 | h-troubleshooting | [Troubleshooting](/t/11864) | +| 2 | r-troubleshooting | [Troubleshooting](/t/11864) | | 2 | r-plugins-extensions | [Plugins/extensions](/t/10946) | | 2 | r-contacts | [Contacts](/t/11863) | | 1 | explanation | [Explanation]() | @@ -111,6 +113,13 @@ PostgreSQL is a trademark or registered trademark of PostgreSQL Global Developme | 2 | e-logs | [Logs](/t/12099) | | 2 | e-juju-details | [Juju](/t/11985) | | 2 | e-legacy-charm | [Legacy charm](/t/10690) | +| 2 | e-connection-pooling| [Connection pooling](/t/15777) | | 1 | search | [Search](https://canonical.com/data/docs/postgresql/iaas) | -[/details] \ No newline at end of file +[/details] + + \ No newline at end of file diff --git a/docs/reference/r-all-releases.md b/docs/reference/r-all-releases.md deleted file mode 100644 index 7f2144528a..0000000000 --- a/docs/reference/r-all-releases.md +++ /dev/null @@ -1,32 +0,0 @@ -# Release Notes -Here you will find release notes for major revisions of this charm that are available in the [Charmhub `stable` channel](https://juju.is/docs/juju/channel#heading--risk). - -To see **all** charm revisions, check the [Charmed PostgreSQL Releases page](https://github.com/canonical/postgresql-operator/releases) on GitHub. - -## At a glance - -The table below is a high-level overview of the architectures and integrations that are supported by each charm revision. - -| Revision | amd64 | arm64 | [TLS encryption](/t/9685)* | [Monitoring (COS, Grafana)](/t/10600) | [Tracing (Tempo K8s)](/t/14521) | -|:--------:|:-----:|:-----:|:--------------------:|:---------------:|:--------------------:| -| [430](/t/14067) | | ![check] | ![check] | ![check] | ![check] | -| [429](/t/14067) | ![check] | | ![check] | ![check] | ![check] | -| [363](/t/13124) | ![check] | | ![check] | ![check] | | -| [351](/t/12823) | ![check] | | | ![check] | | -| [336](/t/11877) | ![check] | | | ![check] | | -| [288](/t/11876) | ![check] | | | | | - - - -**TLS encryption***: Support for **`v2` or higher** of the [`tls-certificates` interface](https://charmhub.io/tls-certificates-interface/libraries/tls_certificates). This means that you can integrate with [modern TLS charms](https://charmhub.io/topics/security-with-x-509-certificates). - -For more details about a particular revision, refer to its dedicated Release Notes page. -For more details about each feature/interface, refer to their dedicated How-To guide. - -### Plugins/extensions - -For a list of all plugins supported for each revision, see the reference page [Plugins/extensions](/t/10946). - - -[check]: https://img.shields.io/badge/%E2%9C%93-brightgreen -[cross]: https://img.shields.io/badge/x-white \ No newline at end of file diff --git a/docs/reference/r-releases.md b/docs/reference/r-releases.md new file mode 100644 index 0000000000..a1f9573916 --- /dev/null +++ b/docs/reference/r-releases.md @@ -0,0 +1,101 @@ +# Release Notes + +This page provides high-level overviews of the dependencies and features that are supported by each revision in every stable release. + +To learn more about the different release tracks and channels, see the [Juju documentation about channels](https://juju.is/docs/juju/channel#heading--risk). + +To see all releases and commits, check the [Charmed PostgreSQL Releases page on GitHub](https://github.com/canonical/postgresql-operator/releases). + +## Dependencies and supported features + +For a given release, this table shows: +* The PostgreSQL version packaged inside +* The minimum Juju 3 version required to reliably operate **all** features of the release + > This charm still supports older versions of Juju down to 2.9. See the [Juju section of the system requirements](/t/11743) for more details. +* Support for specific features + +| Release | PostgreSQL version | Juju 3 version | [TLS encryption](/t/9685)* | [COS monitoring](/t/10600) | [Minor version upgrades](/t/12089) | [Cross-regional async replication](/t/15412) | [Point-in-time recovery](/t/9693) | +|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:| +| [467], [468] | 14.12 | `3.4.3+` | ![check] | ![check] | ![check] | ![check] | ![check] | +| [429], [430] | 14.11 | `3.4.2+` | ![check] | ![check] | ![check] | ![check] | | +| [363] | 14.10 | `3.4.2+` | ![check] | ![check] | ![check] | ![check] | | +| [351] | 14.9 | `3.1.6+` | | ![check] | ![check] | | | +| [336] | 14.9 | `3.1.5+` | | ![check] | ![check] | | | +| [288] | 14.7 | `2.9.32+` | | | | | | + + +\* **TLS encryption**: Support for **`v2` or higher** of the [`tls-certificates` interface](https://charmhub.io/tls-certificates-interface/libraries/tls_certificates). This means that you can integrate with [modern TLS charms](https://charmhub.io/topics/security-with-x-509-certificates). + +For more details about a particular revision, refer to its dedicated Release Notes page. +For more details about each feature/interface, refer to the documentation linked in the column header. + +## Architecture and base +Several [revisions](https://juju.is/docs/sdk/revision) are released simultaneously for different [bases/series](https://juju.is/docs/juju/base) using the same charm code. In other words, one release contains multiple revisions. + +> If you do not specify a revision on deploy time, Juju will automatically choose the revision that matches your base and architecture. + +> If you deploy a specific revision, **you must make sure it matches your base and architecture** via the tables below or with [`juju info`](https://juju.is/docs/juju/juju-info) + +### Release 467-468 (`14/stable`) + +| Revision | amd64 | arm64 | Ubuntu 22.04 LTS +|:--------:|:-----:|:-----:|:-----:| +|[468] |![check] | | ![check] | +|[467] | | ![check]| ![check] | + +[details=Release 429-430] + +| Revision | amd64 | arm64 | Ubuntu 22.04 LTS +|:--------:|:-----:|:-----:|:-----:| +|[430] |![check]| | ![check] | +|[429] | | ![check]| ![check] | +[/details] + +[details=Release 363] + +| Revision | amd64 | arm64 | Ubuntu 22.04 LTS +|:--------:|:-----:|:-----:|:-----:| +|[363] | ![check]| | ![check] | +[/details] + +[details=Release 351] + +| Revision | amd64 | arm64 | Ubuntu 22.04 LTS +|:--------:|:-----:|:-----:|:-----:| +|[351] |![check]| | ![check] | +[/details] + +[details=Release 336] + +| Revision | amd64 | arm64 | Ubuntu 22.04 LTS +|:--------:|:-----:|:-----:|:-----:| +|[336] |![check]| | ![check] | +[/details] + +[details=Release 288] + +| Revision | amd64 | arm64 | Ubuntu 22.04 LTS +|:--------:|:-----:|:-----:|:-----:| +|[288] |![check]| | ![check] | +[/details] + +## Plugins/extensions + +For a list of all plugins supported for each revision, see the reference page [Plugins/extensions](/t/10946). + +[note] + Our release notes are an ongoing work in progress. If there is any additional information about releases that you would like to see or suggestions for other improvements, don't hesitate to contact us on [Matrix ](https://matrix.to/#/#charmhub-data-platform:ubuntu.com) or [leave a comment](https://discourse.charmhub.io/t/charmed-postgresql-reference-release-notes/11875). +[/note] + + +[468]: /t/15378 +[467]: /t/15378 +[430]: /t/14067 +[429]: /t/14067 +[363]: /t/13124 +[351]: /t/12823 +[336]: /t/11877 +[288]: /t/11876 + + +[check]: https://img.icons8.com/color/20/checkmark--v1.png \ No newline at end of file diff --git a/docs/reference/r-revision-467-468.md b/docs/reference/r-revision-467-468.md new file mode 100644 index 0000000000..e3664cbaa3 --- /dev/null +++ b/docs/reference/r-revision-467-468.md @@ -0,0 +1,157 @@ +>Reference > Release Notes > [All revisions] > Revision 467/468 + +# Revision 467/468 +September 11, 2024 + +Dear community, + +Canonical's newest Charmed PostgreSQL operator has been published in the [14/stable channel]. + +Due to the newly added support for `arm64` architecture, the PostgreSQL charm now releases multiple revisions simultaneously: +* Revision 468 is built for `amd64` on Ubuntu 22.04 LTS +* Revision 467 is built for `arm64` on Ubuntu 22.04 LTS + +To make sure you deploy for the right architecture, we recommend setting an [architecture constraint](https://juju.is/docs/juju/constraint#heading--arch) for your entire juju model. + +Otherwise, it can be done at deploy time with the `--constraints` flag: +```shell +juju deploy postgresql --constraints arch= +``` +where `` can be `amd64` or `arm64`. + +--- + +## Highlights +* Upgraded PostgreSQL from v.14.11 → v.14.12 ([PR #530](https://github.com/canonical/postgresql-operator/pull/530)) + * Check the official [PostgreSQL release notes](https://www.postgresql.org/docs/release/14.12/) +* Added support for Point In Time Recovery ([PR #391](https://github.com/canonical/postgresql-operator/pull/391)) ([DPE-2582](https://warthogs.atlassian.net/browse/DPE-2582)) +* Secure Syncobj and Patroni with passwords ([PR #596](https://github.com/canonical/postgresql-operator/pull/596)) ([DPE-5269](https://warthogs.atlassian.net/browse/DPE-5269)) +* Removed deprecated config option `profile-limit-memory` ([PR #564](https://github.com/canonical/postgresql-operator/pull/564)) ([DPE-4889](https://warthogs.atlassian.net/browse/DPE-4889)) + +## Features + +* Added URI connection string to relations ([PR #527](https://github.com/canonical/postgresql-operator/pull/527)) ([DPE-2278](https://warthogs.atlassian.net/browse/DPE-2278)) +* Improve `list-backups` action output ([PR #522](https://github.com/canonical/postgresql-operator/pull/522)) ([DPE-4479](https://warthogs.atlassian.net/browse/DPE-4479)) +* Show start/end time in UTC time in list-backups output ([PR #551](https://github.com/canonical/postgresql-operator/pull/551)) +* Switched to constant snap locales ([PR #559](https://github.com/canonical/postgresql-operator/pull/559)) ([DPE-4198](https://warthogs.atlassian.net/browse/DPE-4198)) +* Moved URI generation to update endpoints ([PR #584](https://github.com/canonical/postgresql-operator/pull/584)) + +## Bugfixes + +* Wait for exact number of units after scale down ([PR #565](https://github.com/canonical/postgresql-operator/pull/565)) ([DPE-5029](https://warthogs.atlassian.net/browse/DPE-5029)) +* Improved test stability by pausing Patroni in the TLS test ([PR #534](https://github.com/canonical/postgresql-operator/pull/534)) ([DPE-4533](https://warthogs.atlassian.net/browse/DPE-4533)) +* Block charm if it detects objects dependent on disabled plugins ([PR #560](https://github.com/canonical/postgresql-operator/pull/560)) ([DPE-4967](https://warthogs.atlassian.net/browse/DPE-4967)) +* Disabled pgBackRest service initialization ([PR #530](https://github.com/canonical/postgresql-operator/pull/530)) ([DPE-4345](https://warthogs.atlassian.net/browse/DPE-4345)) +* Increased timeout and terminate processes that are still up ([PR #514](https://github.com/canonical/postgresql-operator/pull/514)) ([DPE-4532](https://warthogs.atlassian.net/browse/DPE-4532)) +* Fixed GCP backup test ([PR #521](https://github.com/canonical/postgresql-operator/pull/521)) ([DPE-4820](https://warthogs.atlassian.net/browse/DPE-4820)) +* Handled on start secret exception and remove stale test ([PR #550](https://github.com/canonical/postgresql-operator/pull/550)) +* Removed block on failure to get the db version ([PR #578](https://github.com/canonical/postgresql-operator/pull/578)) ([DPE-3562](https://warthogs.atlassian.net/browse/DPE-3562)) +* Updated unit tests after fixing GCP backup test ([PR #528](https://github.com/canonical/postgresql-operator/pull/528)) ([DPE-4820](https://warthogs.atlassian.net/browse/DPE-4820)) +* Ported some `test_self_healing` CI fixes + update check for invalid extra user credentials ([PR #546](https://github.com/canonical/postgresql-operator/pull/546)) ([DPE-4856](https://warthogs.atlassian.net/browse/DPE-4856)) +* Fixed slow bootstrap of replicas ([PR #510](https://github.com/canonical/postgresql-operator/pull/510)) ([DPE-4759](https://warthogs.atlassian.net/browse/DPE-4759)) +* Fixed conditional password ([PR #604](https://github.com/canonical/postgresql-operator/pull/604)) +* Added enforcement of Juju versions ([PR #518](https://github.com/canonical/postgresql-operator/pull/518)) ([DPE-4809](https://warthogs.atlassian.net/browse/DPE-4809)) +* Fixed a missing case for peer to secrets translation. ([PR #533](https://github.com/canonical/postgresql-operator/pull/533)) +* Updated README.md ([PR #538](https://github.com/canonical/postgresql-operator/pull/538)) ([DPE-4901](https://warthogs.atlassian.net/browse/DPE-4901)) +* Increased test coverage ([PR #505](https://github.com/canonical/postgresql-operator/pull/505)) + +## Known limitations + + * The unit action `resume-upgrade` randomly raises a [harmless error message](https://warthogs.atlassian.net/browse/DPE-5420): `terminated`. + * The [charm sysbench](https://charmhub.io/sysbench) may [crash](https://warthogs.atlassian.net/browse/DPE-5436) during a PostgreSQL charm refresh. + * Make sure that [cluster-cluster replication](/t/13991) is requested for the same charm/workload revisions. An automated check is [planned](https://warthogs.atlassian.net/browse/DPE-5418). + * [Contact us](/t/11863) to schedule [the cluster-cluster replication](/t/13991) upgrade with you. + +If you are jumping over several stable revisions, check [previous release notes][All revisions] before upgrading. + +## Requirements and compatibility + +This charm revision features the following changes in dependencies: +* (increased) The minimum Juju version required to reliably operate **all** features of the release is `v3.4.5` + > You can upgrade to this revision on Juju `v2.9.50+`, but it will not support newer features like cross-regional asynchronous replication, point-in-time recovery, and modern TLS certificate charm integrations. +* (increased) PostgreSQL version 14.12 + +Check the [system requirements] page for more details, such as supported minor versions of Juju and hardware requirements. + +### Integration tests +Below are the charm integrations tested with this revision on different Juju environments and architectures: +* Juju `v.2.9.50` on `amd64` +* Juju `v.3.4.5` on `amd64` and `arm64` + +| Software | Version | Notes | +|-----|-----|-----| +| [lxd] | `5.12/stable` | | +| [nextcloud] | `v29.0.5.1`, `rev 26` | | +| [mailman3-core] | `rev 18` | | +| [data-integrator] | `rev 41` | | +| [s3-integrator] | `rev 31` | | +| [postgresql-test-app] | `rev 237` | | + +See the [`/lib/charms` directory on GitHub] for details about all supported libraries. + +See the [`metadata.yaml` file on GitHub] for a full list of supported interfaces. + +### Packaging + +This charm is based on the Charmed PostgreSQL [snap Revision 120/121]. It packages: +* [postgresql `v.14.12`] +* [pgbouncer `v.1.21`] +* [patroni `v.3.1.2 `] +* [pgBackRest `v.2.48`] +* [prometheus-postgres-exporter `v.0.12.1`] + +### Dependencies and automations + +[details=This section contains a list of updates to libs, dependencies, actions, and workflows.] + +* Added jinja2 as a dependency ([PR #520](https://github.com/canonical/postgresql-operator/pull/520)) ([DPE-4816](https://warthogs.atlassian.net/browse/DPE-4816)) +* Switched Jira issue sync from workflow to bot ([PR #586](https://github.com/canonical/postgresql-operator/pull/586)) +* Updated canonical/charming-actions action to v2.6.2 ([PR #523](https://github.com/canonical/postgresql-operator/pull/523)) +* Updated data-platform-workflows to v21.0.1 ([PR #599](https://github.com/canonical/postgresql-operator/pull/599)) +* Updated dependency cryptography to v43 ([PR #539](https://github.com/canonical/postgresql-operator/pull/539)) +* Updated dependency tenacity to v9 ([PR #558](https://github.com/canonical/postgresql-operator/pull/558)) +* Updated Juju agents (patch) ([PR #553](https://github.com/canonical/postgresql-operator/pull/553)) +* Switch to resusable presets ([PR #513](https://github.com/canonical/postgresql-operator/pull/513)) +* Use poetry package-mode=false ([PR #556](https://github.com/canonical/postgresql-operator/pull/556)) +* Switched test-app interface ([PR #557](https://github.com/canonical/postgresql-operator/pull/557)) +[/details] + + +[All revisions]: /t/11875 +[system requirements]: /t/11743 + + +[`/lib/charms` directory on GitHub]: https://github.com/canonical/postgresql-operator/tree/rev468/lib/charms +[`metadata.yaml` file on GitHub]: https://github.com/canonical/postgresql-operator/blob/rev468/metadata.yaml + + +[14/stable channel]: https://charmhub.io/postgresql?channel=14/stable + + +[`charmed-postgresql` packaging]: https://github.com/canonical/charmed-postgresql-snap +[snap Revision 120/121]: https://github.com/canonical/charmed-postgresql-snap/releases/tag/rev121 +[rock image]: ghcr.io/canonical/charmed-postgresql@sha256:7ef86a352c94e2a664f621a1cc683d7a983fd86e923d98c32b863f717cb1c173 + +[postgresql `v.14.12`]: https://launchpad.net/ubuntu/+source/postgresql-14/14.12-0ubuntu0.22.04.1 +[pgbouncer `v.1.21`]: https://launchpad.net/~data-platform/+archive/ubuntu/pgbouncer +[patroni `v.3.1.2 `]: https://launchpad.net/~data-platform/+archive/ubuntu/patroni +[pgBackRest `v.2.48`]: https://launchpad.net/~data-platform/+archive/ubuntu/pgbackrest +[prometheus-postgres-exporter `v.0.12.1`]: https://launchpad.net/~data-platform/+archive/ubuntu/postgres-exporter + + +[juju]: https://juju.is/docs/juju/ +[lxd]: https://documentation.ubuntu.com/lxd/en/latest/ +[nextcloud]: https://charmhub.io/nextcloud +[mailman3-core]: https://charmhub.io/mailman3-core +[data-integrator]: https://charmhub.io/data-integrator +[s3-integrator]: https://charmhub.io/s3-integrator +[postgresql-test-app]: https://charmhub.io/postgresql-test-app +[discourse-k8s]: https://charmhub.io/discourse-k8s +[indico]: https://charmhub.io/indico +[microk8s]: https://charmhub.io/microk8s +[tls-certificates-operator]: https://charmhub.io/tls-certificates-operator +[self-signed-certificates]: https://charmhub.io/self-signed-certificates + + +[amd64]: https://img.shields.io/badge/amd64-darkgreen +[arm64]: https://img.shields.io/badge/arm64-blue \ No newline at end of file diff --git a/docs/how-to/h-troubleshooting.md b/docs/reference/r-troubleshooting.md similarity index 100% rename from docs/how-to/h-troubleshooting.md rename to docs/reference/r-troubleshooting.md diff --git a/docs/tutorial/t-access.md b/docs/tutorial/t-access.md new file mode 100644 index 0000000000..7e02a57e2b --- /dev/null +++ b/docs/tutorial/t-access.md @@ -0,0 +1,162 @@ +> [Charmed PostgreSQL Tutorial](/t/9707) > 3. Access PostgreSQL + +# Access PostgreSQL + +In this section, you will learn how to get the credentials of your deployment, connect to the PostgreSQL instance, view its default databases, and finally, create your own new database. +[note type="caution"] +This part of the tutorial accesses PostgreSQL via the `operator` user. + +**Do not directly interface with the `operator` user in a production environment.** + +In a later section about [Integrations,](https://charmhub.io/postgresql-k8s/docs/t-integrations) we will cover how to safely access PostgreSQL by creating a separate user via the [Data Integrator charm](https://charmhub.io/data-integrator) +[/note] + + +## Summary + +- [Retrieve credentials](#retrieve-credentials) +- [Access PostgreSQL via `psql`](#access-postgresql-via-psql) + - [ Create a new database](#heading--new-database) + +--- +## Retrieve credentials + +Connecting to the database requires that you know three pieces of information: The internal postgreSQL database's username and password, and the host machine's IP address. + +The IP addresses associated with each application unit can be found using the `juju status` command. Since we will use the leader unit to connect to PostgreSQL, we are interested in the IP address for the unit marked with `*`, like shown in the output below: +```shell +Unit Workload Agent Address Ports Message +postgresql/0* active idle 10.1.110.80 Primary +``` + +The user we will connect to in this tutorial will be 'operator'. To retrieve its associated password, run the Charmed PostgreSQL action get-password: +```shell +juju run postgresql/leader get-password +``` +The command above should output something like this: +```shell +Running operation 1 with 1 task + - task 2 on unit-postgresql-0 + +Waiting for task 2... +password: 66hDfCMm3ofT0yrG +``` +In order to retrieve the password of a user other than 'operator', use the option username: +```shell +juju run postgresql/leader get-password username=replication +``` + +At this point, we have all the information required to access PostgreSQL. Run the command below to enter the leader unit's shell as root: + +```shell +juju ssh --container postgresql postgresql/leader bash +``` +which should bring you to a prompt like this: + +```shell + root@postgresql-0:/# +``` +The following commands should be executed from this remote shell you just logged into. + +>If you’d like to leave the unit's shell and return to your local terminal, enter `Ctrl+D` or type `exit`. + +## Access PostgreSQL via `psql` + +The easiest way to interact with PostgreSQL is via [PostgreSQL interactive terminal `psql`](https://www.postgresql.org/docs/14/app-psql.html), which is already installed on the host you're connected to. + +For example, to list all databases currently available, run the command below. When requested, enter the password that you obtained earlier. +```shell +psql --host=10.1.110.80 --username=operator --password --list +``` + +You can see below the output for the list of databases. `postgres` is the default database we are connected to and is used for administrative tasks and for creating other databases. +```shell + Name | Owner | Encoding | Collate | Ctype | Access privileges +-----------+----------+----------+-------------+-------------+----------------------- + postgres | operator | UTF8 | en_US.UTF-8 | en_US.UTF-8 | + template0 | operator | UTF8 | en_US.UTF-8 | en_US.UTF-8 | =c/operator + + | | | | | operator=CTc/operator + template1 | operator | UTF8 | en_US.UTF-8 | en_US.UTF-8 | =c/operator + + | | | | | operator=CTc/operator +(3 rows) +``` + +In order to execute queries, we should enter psql's interactive terminal by running the following command, again typing password when requested: +```shell + psql --host=10.1.110.80 --username=operator --password postgres +``` + +The output should be something like this: + +```shell +psql (14.10 (Ubuntu 14.10-0ubuntu0.22.04.1)) +Type "help" for help. + +postgres=# +``` +Now you are successfully logged in the interactive terminal. Here it is possible to execute commands to PostgreSQL directly using PostgreSQL SQL Queries. For example, to show which version of PostgreSQL is installed, run the following command: + +```shell +postgres=# SELECT version(); + version +--------------------------------------------------------------------------------------------------------------------------------- + PostgreSQL 14.10 (Ubuntu 14.10-0ubuntu0.22.04.1) on x86_64-pc-linux-gnu, compiled by gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0, 64-bit +(1 row) +``` + +We can see that PostgreSQL version 14.10 is installed. From this prompt, to print the list of available databases, we can simply run this command: + +```shell +postgres=# \l +``` + +The output should be the same as the one obtained before with `psql`, but this time we did not need to specify any parameters since we are already connected to the PostgreSQL application. + +### Create a new database +For creating and connecting to a new sample database, we can run the following commands: +```shell +postgres=# CREATE DATABASE mynewdatabase; +postgres=# \c mynewdatabase + +You are now connected to database "mynewdatabase" as user "operator". +``` + +We can now create a new table inside this database: + +```shell +postgres=# CREATE TABLE mytable ( + id SERIAL PRIMARY KEY, + name VARCHAR(50), + age INT +); +``` + +And insert an element into it: + +```shell +postgres=# INSERT INTO mytable (name, age) VALUES ('John', 30); +``` + +We can see our new table element by submitting a query: + +```shell +postgres=# SELECT * FROM mytable; + + id | name | age +----+------+----- + 1 | John | 30 +(1 row) +``` + +You can try multiple SQL commands inside this environment. Once you're ready, reconnect to the default postgres database and drop the sample database we created: + +```shell +postgres=# \c postgres + +You are now connected to database "postgres" as user "operator". +postgres=# DROP DATABASE mynewdatabase; +``` + +When you’re ready to leave the PostgreSQL shell, you can just type `exit`. This will take you back to the host of Charmed PostgreSQL (`postgresql/0`). Exit this host by once again typing exit. Now you will be in your original shell where you first started the tutorial. Here you can interact with Juju and LXD. + +**Next step**: [4. Scale replicas](/t/9705) \ No newline at end of file diff --git a/docs/tutorial/t-clean-up.md b/docs/tutorial/t-clean-up.md index aba0cdb7e3..62de6ef958 100644 --- a/docs/tutorial/t-clean-up.md +++ b/docs/tutorial/t-clean-up.md @@ -1,4 +1,4 @@ -> [Charmed PostgreSQL VM Tutorial](https://discourse.charmhub.io/t/9707) > 7. Clean up environment +> [Charmed PostgreSQL VM Tutorial](/t/9707) > 7. Clean up environment # Clean up your environment @@ -22,7 +22,7 @@ If you're done with testing and would like to free up resources on your machine, For more information, see the docs for [`multipass delete`](https://multipass.run/docs/delete-command). [/note] -**Delete your VM and its data** with +**Delete your VM and its data** by running ```shell multipass delete --purge my-vm ``` diff --git a/docs/tutorial/t-deploy.md b/docs/tutorial/t-deploy.md index 6ad6594b3e..7799f482bc 100644 --- a/docs/tutorial/t-deploy.md +++ b/docs/tutorial/t-deploy.md @@ -4,16 +4,9 @@ In this section, you will deploy Charmed PostgreSQL VM, access a unit, and interact with the PostgreSQL databases that exist inside the application. -## Summary -- [Deploy PostgreSQL](#heading--deploy) -- [Access PostgreSQL](#heading--access) - - [Retrieve credentials](#heading--retrieve-credentials) - - [Access PostgreSQL via `psql`](#heading--psql) ---- +## Deploy PostgreSQL -

Deploy PostgreSQL

- -To deploy Charmed PostgreSQL, all you need to do is run +To deploy Charmed PostgreSQL, run ```shell juju deploy postgresql ``` @@ -42,100 +35,5 @@ Machine State Address Inst id Series AZ Message 0 started 10.89.49.129 juju-a8a31d-0 jammy Running ``` -You can also watch juju logs with the [`juju debug-log`](https://juju.is/docs/juju/juju-debug-log) command. -More info on logging in the [juju logs documentation](https://juju.is/docs/olm/juju-logs). - -

Access PostgreSQL

- -[note type="caution"] - **Warning**: This part of the tutorial accesses PostgreSQL via the `operator` user. - -**Do not directly interface with the `operator` user in a production environment.** - -In a later section about [Integrations,](https://charmhub.io/postgresql-k8s/docs/t-integrations) we will cover how to safely access PostgreSQL by creating a separate user via the [Data Integrator charm](https://charmhub.io/data-integrator) -[/note] - -

Retrieve credentials

- -Connecting to the database requires that you know the values for `host`, `username` and `password`. - -To retrieve these values, run the Charmed PostgreSQL action `get-password`: -```shell -juju run postgresql/leader get-password -``` -Running the command above should output: -```yaml -unit-postgresql-0: - UnitId: postgresql/0 - id: "2" - results: - operator-password: - status: completed - timing: - completed: 2023-03-20 08:42:22 +0000 UTC - enqueued: 2023-03-20 08:42:19 +0000 UTC - started: 2023-03-20 08:42:21 +0000 UTC -``` - -To request a password for a different user, use the option `username`: -```shell -juju run postgresql/leader get-password username=replication -``` - -The IP address of the unit hosting the PostgreSQL application, also referred to as the "host", can be found with `juju status`: -``` -... -Unit Workload Agent Machine Public address Ports Message -postgresql/0* active idle 0 10.89.49.129 -... -``` -

Access PostgreSQL via psql

- - -To access the units hosting Charmed PostgreSQL, run -```shell -juju ssh postgresql/leader -``` - ->If at any point you'd like to leave the unit hosting Charmed PostgreSQL K8s, enter `Ctrl+D` or type `exit`. - -The easiest way to access PostgreSQL is via the [PostgreSQL interactive terminal `psql`](https://www.postgresql.org/docs/14/app-psql.html), which is already installed here. - -To list all available databases, run: -```shell -psql --host=10.89.49.129 --username=operator --password --list -``` -When requested, enter the `` for charm user `operator` that you obtained earlier. - -Example output: -``` - List of databases - Name | Owner | Encoding | Collate | Ctype | Access privileges ------------+----------+----------+---------+---------+----------------------- - postgres | operator | UTF8 | C.UTF-8 | C.UTF-8 | - template0 | operator | UTF8 | C.UTF-8 | C.UTF-8 | =c/operator + - | | | | | operator=CTc/operator - template1 | operator | UTF8 | C.UTF-8 | C.UTF-8 | =c/operator + - | | | | | operator=CTc/operator -(3 rows) -``` - -You can now interact with PostgreSQL directly using [PostgreSQL SQL Queries](https://www.postgresql.org/docs/14/queries.html). For example, entering `SELECT version();` should output something like: -``` -> psql --host=10.89.49.129 --username=operator --password postgres -Password: -psql (14.7 (Ubuntu 14.7-0ubuntu0.22.04.1)) -Type "help" for help. - -postgres=# SELECT version(); - version --------------------------------------------------------------------------------------------------------------------------------------- - PostgreSQL 14.7 (Ubuntu 14.7-0ubuntu0.22.04.1) on x86_64-pc-linux-gnu, compiled by gcc (Ubuntu 11.3.0-1ubuntu1~22.04) 11.3.0, 64-bit -(1 row) -``` - -Feel free to test out any other PostgreSQL queries. - -When you’re ready to leave the PostgreSQL shell, you can just type `exit`. This will take you back to the host of Charmed PostgreSQL K8s (`postgresql-k8s/0`). Exit this host by once again typing `exit`. Now you will be in your original shell where you first started the tutorial. Here you can interact with Juju and LXD. - -**Next step:** [3. Scale replicas](/t/9705) \ No newline at end of file +> You can also watch juju logs with the [`juju debug-log`](https://juju.is/docs/juju/juju-debug-log) command. +More info on logging in the [juju logs documentation](https://juju.is/docs/olm/juju-logs). \ No newline at end of file diff --git a/docs/tutorial/t-enable-tls.md b/docs/tutorial/t-enable-tls.md index 1f279df490..0a425966fb 100644 --- a/docs/tutorial/t-enable-tls.md +++ b/docs/tutorial/t-enable-tls.md @@ -57,6 +57,7 @@ To enable TLS on Charmed PostgreSQL VM, integrate the two applications: ```shell juju integrate postgresql self-signed-certificates ``` +PostgreSQL is now using TLS certificate generated by the `self-signed-certificates` charm. ### Check the TLS certificate in use Use `openssl` to connect to the PostgreSQL and check the TLS certificate in use. Note that your leader unit's IP address will likely be different to the one shown below: @@ -67,8 +68,6 @@ depth=1 C = US, CN = Tutorial CA verify error:num=19:self-signed certificate in certificate chain ... ``` -Congratulations! PostgreSQL is now using TLS certificate generated by the external application `self-signed-certificates`. -

Remove TLS certificate

 To remove the external TLS, remove the integration: diff --git a/docs/tutorial/t-integrate.md b/docs/tutorial/t-integrate.md index 81cfb3de31..72db1dae01 100644 --- a/docs/tutorial/t-integrate.md +++ b/docs/tutorial/t-integrate.md @@ -17,20 +17,20 @@ In this section, you will integrate your Charmed PostgreSQL to another charmed a

Deploy data-integrator

-Before relating to a charmed application, we must first deploy our charmed application. In this tutorial, we will relate to the [Data Integrator charm](https://charmhub.io/data-integrator). This is a bare-bones charm that allows for central management of database users, providing support for different kinds of data platforms (e.g. PostgreSQL, MySQL, MongoDB, Kafka, etc) with a consistent, opinionated and robust user experience. +In this tutorial, we will relate to the [Data Integrator charm](https://charmhub.io/data-integrator). This is a bare-bones charm that allows for central management of database users. It automatically provides credentials and endpoints that are needed to connect with a charmed database application. To deploy `data-integrator`, run ```shell juju deploy data-integrator --config database-name=test-database ``` -The expected output: +Example output: ``` Located charm "data-integrator" in charm-hub, revision 11 Deploying "data-integrator" from charm-hub charm "data-integrator", revision 11 in channel stable on jammy ``` -Checking the deployment progress using `juju status` will show a `blocked` state for the newly deployed charm: +Running `juju status` will show you `data-integrator` in a `blocked` state. This state is expected due to not-yet established relation (integration) between applications. ``` Model Controller Cloud/Region Version SLA Timestamp tutorial overlord localhost/localhost 3.1.7 unsupported 10:22:13+01:00 @@ -49,7 +49,6 @@ Machine State Address Inst id Series AZ Message 1 started 10.89.49.197 juju-a8a31d-1 jammy Running 3 started 10.89.49.179 juju-a8a31d-3 jammy Running ``` -The `blocked` state is expected due to not-yet established relation (integration) between applications.

Integrate with PostgreSQL

@@ -79,11 +78,12 @@ Machine State Address Inst id Series AZ Message 3 started 10.89.49.179 juju-a8a31d-3 jammy Running ``` -To retrieve information such as the username, password, and database, run the command +To retrieve the username, password and database name, run the command ```shell juju run data-integrator/leader get-credentials ``` -This should output something like: + +Example output: ```yaml unit-data-integrator-0: UnitId: data-integrator/0 diff --git a/docs/tutorial/t-overview.md b/docs/tutorial/t-overview.md index 2d24731e03..d8a7a11511 100644 --- a/docs/tutorial/t-overview.md +++ b/docs/tutorial/t-overview.md @@ -1,23 +1,24 @@ -# Charmed PostgreSQL VM Tutorial +# Charmed PostgreSQL Tutorial -This section of our documentation contains comprehensive, hands-on tutorials to help you learn how to deploy Charmed PostgreSQL K8s and become familiar with its available operations. +This section of our documentation contains comprehensive, hands-on tutorials to help you learn how to deploy Charmed PostgreSQL on machines and become familiar with its available operations. ## Prerequisites -While this tutorial intends to guide you as you deploy Charmed PostgreSQL K8s for the first time, it will be most beneficial if: +While this tutorial intends to guide you as you deploy Charmed PostgreSQL for the first time, it will be most beneficial if: - You have some experience using a Linux-based CLI - You are familiar with PostgreSQL concepts such as replication and users. -- Your computer fulfills the [minimum system requirements](/t/11743) +- Your computer fulfils the [minimum system requirements](/t/11743) ## Tutorial contents This Charmed PostgreSQL tutorial has the following parts: | Step | Details | | ------- | ---------- | -| 1. [**Set up the environment**](/t/9709) | Set up a cloud environment for your deployment using [Multipass](https://multipass.run/) with [LXD](https://ubuntu.com/lxd) and [Juju](https://juju.is/). -| 2. [**Deploy PostgreSQL**](/t/9697) | Learn to deploy Charmed PostgreSQL K8s using a single command and access the database directly. -| 3. [**Scale the amount of replicas**](/t/9705) | Learn how to enable high availability with a [Patroni](https://patroni.readthedocs.io/en/latest/)-based cluster. -| 4. [**Manage passwords**](/t/9703) | Learn how to request and change passwords. -| 5. [**Integrate PostgreSQL with other applications**](/t/9701) | Learn how to integrate with other applications using the Data Integrator Charm, access the integrated database, and manage users. -| 6. [**Enable TLS encryption**](/t/9699) | Learn how to enable security in your PostgreSQL deployment via TLS. -| 7. [**Clean-up your environment**](/t/9695) | Free up your machine's resources. \ No newline at end of file +| 1. [**Set up your environment**](/t/9709) | Set up a cloud environment for your deployment using [Multipass](https://multipass.run/) with [LXD](https://ubuntu.com/lxd) and [Juju](https://juju.is/). +| 2. [**Deploy PostgreSQL**](/t/9697) | Learn to deploy Charmed PostgreSQL with Juju +| 3. [**Access PostgreSQL**](/t/15798) | Access a PostgreSQL instance directly +| 4. [**Scale your replicas**](/t/9705) | Learn how to enable high availability with a [Patroni](https://patroni.readthedocs.io/en/latest/)-based cluster. +| 5. [**Manage passwords**](/t/9703) | Learn how to request and change passwords. +| 6. [**Integrate PostgreSQL with other applications**](/t/9701) | Learn how to integrate with other applications using the Data Integrator charm, access the integrated database, and manage users. +| 7. [**Enable TLS encryption**](/t/9699) | Learn how to enable security in your PostgreSQL deployment via TLS. +| 8. [**Clean up your environment**](/t/9695) | Free up your machine's resources. \ No newline at end of file