-
Notifications
You must be signed in to change notification settings - Fork 107
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
* Removed auto generated vars * Updated postgresql default configuration with correct settings * Updated tests according to the new version * Increased postgresql, pgaudit, pgpool, repmgr versions * Postgres installation * Repmgr RedHat installation * Commented pgbouncer package not to fail installation * Ubuntu installation except pgbouncer * Updated components * Added molecule tests and documentation * Modified vars * Asciidoc to markdown * Update core/src/epicli/data/common/ansible/playbooks/roles/postgresql/molecule/debian-repmgr/prepare.yml Co-authored-by: to-bar <[email protected]> * Update core/src/epicli/data/common/ansible/playbooks/roles/postgresql/molecule/debian-repmgr/verify.yml Co-authored-by: to-bar <[email protected]> * Update core/src/epicli/data/common/ansible/playbooks/roles/postgresql/molecule/redhat-repmgr/prepare.yml Co-authored-by: to-bar <[email protected]> * Update core/src/epicli/data/common/ansible/playbooks/roles/postgresql/templates/pg_hba.conf.j2 Co-authored-by: to-bar <[email protected]> * Update core/src/epicli/data/common/ansible/playbooks/roles/postgresql/tasks/extensions/replication/extension.yml Co-authored-by: to-bar <[email protected]> * Update core/src/epicli/data/common/ansible/playbooks/roles/postgresql/tasks/extensions/replication/extension.yml Co-authored-by: to-bar <[email protected]> * Update core/src/epicli/data/common/ansible/playbooks/roles/postgresql/tasks/extensions/replication/extension.yml Co-authored-by: to-bar <[email protected]> * Applied requested changes after review * Update core/src/epicli/data/common/ansible/playbooks/roles/postgresql/tasks/main.yml Co-authored-by: to-bar <[email protected]> * Fixed repmgr check * Require 2 hosts for replication * Improve molecule tests * Add {{ ansible_managed }} to templates * Remove unused templates * Improve task names * Do not hide all failures of "Get cluster status" task Co-authored-by: atsikham <[email protected]> Co-authored-by: to-bar <[email protected]>
- Loading branch information
1 parent
5bbd803
commit 197fe44
Showing
48 changed files
with
1,910 additions
and
290 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
26 changes: 13 additions & 13 deletions
26
core/src/epicli/data/common/ansible/playbooks/roles/postgresql/defaults/main.yml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
32 changes: 32 additions & 0 deletions
32
core/src/epicli/data/common/ansible/playbooks/roles/postgresql/molecule/README.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,32 @@ | ||
# Installation guide | ||
|
||
## Requirements | ||
|
||
- Internet connection | ||
- Molecule [dependencies](https://molecule.readthedocs.io/en/latest/installation.html) | ||
- Docker | ||
- ansible-lint | ||
|
||
## Installation | ||
|
||
```bash | ||
pip install 'molecule[docker]' | ||
pip install ansible-lint | ||
``` | ||
|
||
## Execution | ||
|
||
### General | ||
|
||
```bash | ||
molecule test -s <scenario-name> | ||
``` | ||
|
||
### Separate steps | ||
|
||
```bash | ||
molecule lint -s <scenario-name> | ||
molecule converge -s <scenario-name> | ||
molecule idempotence -s <scenario-name> | ||
molecule verify -s <scenario-name> | ||
``` |
11 changes: 11 additions & 0 deletions
11
core/src/epicli/data/common/ansible/playbooks/roles/postgresql/molecule/add-repos-debian.sh
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,11 @@ | ||
#!/bin/bash -eu | ||
|
||
apt update && apt -y install wget gpg-agent | ||
|
||
wget -qO - https://www.postgresql.org/media/keys/ACCC4CF8.asc | sudo apt-key add - | ||
echo "deb http://apt.postgresql.org/pub/repos/apt bionic-pgdg main" | tee /etc/apt/sources.list.d/pgdg.list | ||
|
||
wget -qO - https://dl.2ndquadrant.com/gpg-key.asc | apt-key add - | ||
echo "deb https://dl.2ndquadrant.com/default/release/apt bionic-2ndquadrant main" | tee /etc/apt/sources.list.d/2ndquadrant-dl-default-release.list | ||
|
||
apt update |
143 changes: 143 additions & 0 deletions
143
core/src/epicli/data/common/ansible/playbooks/roles/postgresql/molecule/add-repos-redhat.sh
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,143 @@ | ||
#!/usr/bin/env bash | ||
|
||
set -euo pipefail | ||
|
||
CREATE_LOGFILE='no' | ||
|
||
# params: <repo_id> <config_file_content> | ||
add_repo_as_file() { | ||
local repo_id="$1" | ||
local config_file_content="$2" | ||
local config_file_name="$repo_id.repo" | ||
|
||
if ! is_repo_enabled "$repo_id"; then | ||
echol "Adding repository: $repo_id" | ||
cat <<<"$config_file_content" >"/etc/yum.repos.d/$config_file_name" || | ||
exit_with_error "Function add_repo_as_file failed for repo: $repo_id" | ||
local -a gpg_key_urls | ||
IFS=" " read -r -a gpg_key_urls \ | ||
<<<"$(grep -i --only-matching --perl-regexp '(?<=^gpgkey=)http[^#\n]+' <<<"$config_file_content")" | ||
if ((${#gpg_key_urls[@]} > 0)); then | ||
import_repo_gpg_keys "${gpg_key_urls[@]}" 3 | ||
fi | ||
# to accept import of repo's GPG key (for repo_gpgcheck=1) | ||
yum -y repolist >/dev/null || exit_with_error "Command failed: yum -y repolist" | ||
fi | ||
} | ||
|
||
# params: <script_url> | ||
add_repo_from_script() { | ||
local script_url="$1" | ||
|
||
echol "Running: curl $script_url | bash" | ||
curl $script_url | bash | ||
} | ||
|
||
# params: <repo_id> | ||
disable_repo() { | ||
local repo_id="$1" | ||
|
||
if yum repolist enabled | grep --quiet "$repo_id"; then | ||
echol "Disabling repository: $repo_id" | ||
yum-config-manager --disable "$repo_id" || | ||
exit_with_error "Command failed: yum-config-manager --disable \"$repo_id\"" | ||
fi | ||
} | ||
|
||
echol() { | ||
echo -e "$@" | ||
if [[ $CREATE_LOGFILE == 'yes' ]]; then | ||
local timestamp=$(date +"%b %e %H:%M:%S") | ||
echo -e "${timestamp}: $@" >>"$LOG_FILE_PATH" | ||
fi | ||
} | ||
|
||
# params: <url(s)> <retries> | ||
import_repo_gpg_keys() { | ||
local retries=${!#} # get last arg | ||
local urls=("${@:1:$#-1}") # remove last arg | ||
|
||
for url in "${urls[@]}"; do | ||
run_cmd_with_retries rpm --import "$url" "$retries" | ||
done | ||
} | ||
|
||
# params: <repo_id> | ||
is_repo_enabled() { | ||
local repo_id="$1" | ||
|
||
if yum repolist | grep --quiet "$repo_id"; then | ||
echol "Repository $repo_id already enabled" | ||
return 0 | ||
else | ||
return 1 | ||
fi | ||
} | ||
|
||
# Runs command as array with printing it, doesn't support commands with shell operators (such as pipe or redirection) | ||
# params: <command to execute> [--no-exit-on-error] | ||
run_cmd() { | ||
local cmd_arr=("$@") | ||
|
||
local exit_on_error=1 | ||
if [[ ${cmd_arr[-1]} == '--no-exit-on-error' ]]; then | ||
exit_on_error=0 | ||
cmd_arr=("${cmd_arr[@]:0:$#-1}") # remove last item | ||
fi | ||
|
||
local escaped_string return_code | ||
escaped_string=$(_print_array_as_shell_escaped_string "${cmd_arr[@]}") | ||
echol "Executing: ${escaped_string}" | ||
"${cmd_arr[@]}" | ||
return_code=$? | ||
if ((return_code != 0)) && ((exit_on_error)); then | ||
exit_with_error "Command failed: ${escaped_string}" | ||
else | ||
return $return_code | ||
fi | ||
} | ||
|
||
# Runs command with retries, doesn't support commands with shell operators (such as pipe or redirection) | ||
# params: <command to execute> <retries> | ||
run_cmd_with_retries() { | ||
# pop 'retries' argument | ||
local retries=${!#} # get last arg (indirect expansion) | ||
set -- "${@:1:$#-1}" # set new "$@" | ||
|
||
local cmd_arr=("$@") | ||
(# sub-shell is used to limit scope for 'set +e' | ||
set +e | ||
trap - ERR # disable global trap locally | ||
for ((i = 0; i <= retries; i++)); do | ||
run_cmd "${cmd_arr[@]}" '--no-exit-on-error' | ||
return_code=$? | ||
if ((return_code == 0)); then | ||
break | ||
elif ((i < retries)); then | ||
sleep 1 | ||
echol "retrying ($((i + 1))/${retries})" | ||
else | ||
echol "ERROR: all attempts failed" | ||
local escaped_string | ||
escaped_string=$(_print_array_as_shell_escaped_string "${cmd_arr[@]}") | ||
exit_with_error "Command failed: ${escaped_string}" | ||
fi | ||
done | ||
return $return_code | ||
) | ||
} | ||
|
||
POSTGRESQL_REPO_CONF=$( | ||
cat <<'EOF' | ||
[pgdg13] | ||
name=PostgreSQL 13 for RHEL/CentOS $releasever - $basearch | ||
baseurl=https://download.postgresql.org/pub/repos/yum/13/redhat/rhel-$releasever-$basearch | ||
enabled=1 | ||
gpgcheck=1 | ||
gpgkey=https://download.postgresql.org/pub/repos/yum/RPM-GPG-KEY-PGDG | ||
EOF | ||
) | ||
|
||
add_repo_as_file 'postgresql-13' "$POSTGRESQL_REPO_CONF" | ||
add_repo_from_script 'https://dl.2ndquadrant.com/default/release/get/13/rpm' # for repmgr | ||
disable_repo '2ndquadrant-dl-default-release-pg13-debug' # script adds 2 repositories, only 1 is required |
19 changes: 19 additions & 0 deletions
19
...epicli/data/common/ansible/playbooks/roles/postgresql/molecule/debian-repmgr/converge.yml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,19 @@ | ||
--- | ||
- name: Converge | ||
hosts: postgresql | ||
become: true | ||
become_method: sudo | ||
module_defaults: | ||
shell: | ||
executable: /bin/bash | ||
tasks: | ||
- name: Include 'configuration/postgresql' variables | ||
include_vars: | ||
file: ../vars-repmgr.yml | ||
|
||
- name: Include postgresql role | ||
include_role: | ||
name: postgresql | ||
vars: | ||
yum_lock_timeout: 300 # https://github.com/ansible/ansible/issues/57189 | ||
roles_with_generated_vars: [ ] # don't execute PgPool related tasks |
45 changes: 45 additions & 0 deletions
45
...epicli/data/common/ansible/playbooks/roles/postgresql/molecule/debian-repmgr/molecule.yml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,45 @@ | ||
--- | ||
dependency: | ||
enabled: false | ||
lint: ansible-lint --exclude molecule | ||
driver: | ||
name: docker | ||
platforms: | ||
- name: instance-1 | ||
groups: | ||
- postgresql | ||
image: geerlingguy/docker-ubuntu1804-ansible:python2 | ||
pre_build_image: true | ||
capabilities: | ||
- SYS_ADMIN | ||
command: /lib/systemd/systemd | ||
tmpfs: | ||
- /run | ||
- /tmp | ||
volumes: | ||
- /sys/fs/cgroup:/sys/fs/cgroup:ro | ||
- name: instance-2 | ||
groups: | ||
- postgresql | ||
image: geerlingguy/docker-ubuntu1804-ansible:python2 | ||
pre_build_image: true | ||
capabilities: | ||
- SYS_ADMIN | ||
command: /lib/systemd/systemd | ||
tmpfs: | ||
- /run | ||
- /tmp | ||
volumes: | ||
- /sys/fs/cgroup:/sys/fs/cgroup:ro | ||
provisioner: | ||
name: ansible | ||
playbooks: | ||
prepare: prepare.yml | ||
converge: converge.yml | ||
verify: ../verify-repmgr.yml | ||
inventory: | ||
group_vars: | ||
postgresql: | ||
ansible_python_interpreter: /usr/bin/python2 | ||
verifier: | ||
name: ansible |
6 changes: 6 additions & 0 deletions
6
.../epicli/data/common/ansible/playbooks/roles/postgresql/molecule/debian-repmgr/prepare.yml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,6 @@ | ||
--- | ||
- name: Prepare | ||
hosts: postgresql | ||
tasks: | ||
- name: Add repositories for downloading dependencies | ||
script: ../add-repos-debian.sh |
19 changes: 19 additions & 0 deletions
19
...i/data/common/ansible/playbooks/roles/postgresql/molecule/debian-single-node/converge.yml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,19 @@ | ||
--- | ||
- name: Converge | ||
hosts: postgresql | ||
become: true | ||
become_method: sudo | ||
module_defaults: | ||
shell: | ||
executable: /bin/bash | ||
tasks: | ||
- name: Include 'configuration/postgresql' variables | ||
include_vars: | ||
file: ../vars-single-node.yml | ||
|
||
- name: Include postgresql role | ||
include_role: | ||
name: postgresql | ||
vars: | ||
yum_lock_timeout: 300 # https://github.com/ansible/ansible/issues/57189 | ||
roles_with_generated_vars: [ ] # don't execute PgPool related tasks |
45 changes: 45 additions & 0 deletions
45
...i/data/common/ansible/playbooks/roles/postgresql/molecule/debian-single-node/molecule.yml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,45 @@ | ||
--- | ||
dependency: | ||
enabled: false | ||
lint: ansible-lint --exclude molecule | ||
driver: | ||
name: docker | ||
platforms: | ||
- name: instance-1 | ||
groups: | ||
- postgresql | ||
image: geerlingguy/docker-ubuntu1804-ansible:python2 | ||
pre_build_image: true | ||
capabilities: | ||
- SYS_ADMIN | ||
command: /lib/systemd/systemd | ||
tmpfs: | ||
- /run | ||
- /tmp | ||
volumes: | ||
- /sys/fs/cgroup:/sys/fs/cgroup:ro | ||
- name: instance-2 | ||
groups: | ||
- postgresql | ||
image: geerlingguy/docker-ubuntu1804-ansible:python2 | ||
pre_build_image: true | ||
capabilities: | ||
- SYS_ADMIN | ||
command: /lib/systemd/systemd | ||
tmpfs: | ||
- /run | ||
- /tmp | ||
volumes: | ||
- /sys/fs/cgroup:/sys/fs/cgroup:ro | ||
provisioner: | ||
name: ansible | ||
playbooks: | ||
prepare: prepare.yml | ||
converge: converge.yml | ||
verify: ../verify-single-node.yml | ||
inventory: | ||
group_vars: | ||
postgresql: | ||
ansible_python_interpreter: /usr/bin/python2 | ||
verifier: | ||
name: ansible |
6 changes: 6 additions & 0 deletions
6
...li/data/common/ansible/playbooks/roles/postgresql/molecule/debian-single-node/prepare.yml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,6 @@ | ||
--- | ||
- name: Prepare | ||
hosts: postgresql | ||
tasks: | ||
- name: Add repositories for downloading dependencies | ||
script: ../add-repos-debian.sh |
Oops, something went wrong.