Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add priority class options to high priority pods #862

Merged
merged 4 commits into from
Apr 21, 2022
Merged

Add priority class options to high priority pods #862

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
5372771
Add priority class options to high priority pods
fosterseth Apr 7, 2022
21062f0
Add default resource requests for postgres containers
rooftopcellist Apr 15, 2022
dfa0f6d
Add docs for priority classes & fix typo
rooftopcellist Apr 21, 2022
daf15a9
Reduce the resources requests for CI runs
rooftopcellist Apr 21, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 39 additions & 20 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -434,14 +434,15 @@ If you don't have access to an external PostgreSQL service, the AWX operator can

The following variables are customizable for the managed PostgreSQL service

| Name | Description | Default |
| --------------------------------------------- | --------------------------------------------- | --------------------------------- |
| postgres_image | Path of the image to pull | postgres:12 |
| postgres_init_container_resource_requirements | Database init container resource requirements | requests: {} |
| postgres_resource_requirements | PostgreSQL container resource requirements | requests: {} |
| postgres_storage_requirements | PostgreSQL container storage requirements | requests: {storage: 8Gi} |
| postgres_storage_class | PostgreSQL PV storage class | Empty string |
| postgres_data_path | PostgreSQL data path | `/var/lib/postgresql/data/pgdata` |
| Name | Description | Default |
| --------------------------------------------- | --------------------------------------------- | ---------------------------------- |
| postgres_image | Path of the image to pull | postgres:12 |
| postgres_init_container_resource_requirements | Database init container resource requirements | requests: {cpu: 10m, memory: 64Mi} |
| postgres_resource_requirements | PostgreSQL container resource requirements | requests: {cpu: 10m, memory: 64Mi} |
| postgres_storage_requirements | PostgreSQL container storage requirements | requests: {storage: 8Gi} |
| postgres_storage_class | PostgreSQL PV storage class | Empty string |
| postgres_data_path | PostgreSQL data path | `/var/lib/postgresql/data/pgdata` |
| postgres_priority_class | Priority class used for PostgreSQL pod | Empty string |

Example of customization could be:

Expand Down Expand Up @@ -541,11 +542,11 @@ Again, this is the most relaxed SCC that is provided by OpenShift, so be sure to

The resource requirements for both, the task and the web containers are configurable - both the lower end (requests) and the upper end (limits).

| Name | Description | Default |
| -------------------------- | ------------------------------------------------ | ----------------------------------- |
| web_resource_requirements | Web container resource requirements | requests: {cpu: 1000m, memory: 2Gi} |
| task_resource_requirements | Task container resource requirements | requests: {cpu: 500m, memory: 1Gi} |
| ee_resource_requirements | EE control plane container resource requirements | requests: {cpu: 500m, memory: 1Gi} |
| Name | Description | Default |
| -------------------------- | ------------------------------------------------ | ------------------------------------ |
| web_resource_requirements | Web container resource requirements | requests: {cpu: 100m, memory: 128Mi} |
| task_resource_requirements | Task container resource requirements | requests: {cpu: 100m, memory: 128Mi} |
| ee_resource_requirements | EE control plane container resource requirements | requests: {cpu: 100m, memory: 128Mi} |

Example of customization could be:

Expand All @@ -555,27 +556,45 @@ spec:
...
web_resource_requirements:
requests:
cpu: 1000m
cpu: 250m
memory: 2Gi
limits:
cpu: 2000m
cpu: 1000m
memory: 4Gi
task_resource_requirements:
requests:
cpu: 500m
cpu: 250m
memory: 1Gi
limits:
cpu: 1000m
cpu: 2000m
memory: 2Gi
ee_resource_requirements:
requests:
cpu: 500m
memory: 1Gi
cpu: 250m
memory: 100Mi
limits:
cpu: 1000m
cpu: 500m
memory: 2Gi
```

#### Priority Classes

The AWX and Postgres pods can be assigned a custom PriorityClass to rank their importance compared to other pods in your cluster, which determines which pods get evicted first if resources are running low.
First, [create your PriorityClass](https://kubernetes.io/docs/concepts/scheduling-eviction/pod-priority-preemption/#priorityclass) if needed.
Then set the name of your priority class to the control plane and postgres pods as shown below.

```yaml
---
apiVersion: awx.ansible.com/v1beta1
kind: AWX
metadata:
name: awx-demo
spec:
...
control_plane_priority_class: awx-demo-high-priority
postgres_priority_class: awx-demo-medium-priority
```

#### Assigning AWX pods to specific nodes

You can constrain the AWX pods created by the operator to run on a certain subset of nodes. `node_selector` and `postgres_selector` constrains
Expand Down
8 changes: 7 additions & 1 deletion config/crd/bases/awx.ansible.com_awxs.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -165,6 +165,9 @@ spec:
control_plane_ee_image:
description: Registry path to the Execution Environment container image to use on control plane pods
type: string
control_plane_priority_class:
description: Assign a preexisting priority class to the control plane pods
type: string
ee_pull_credentials_secret:
description: Secret where pull credentials for registered ees can be found
type: string
Expand All @@ -184,7 +187,7 @@ spec:
type: array
items:
type: string
image_pull_secret: # deprecated
image_pull_secret: # deprecated
description: (Deprecated) Image pull secret for app and database containers
type: string
task_resource_requirements:
Expand Down Expand Up @@ -392,6 +395,9 @@ spec:
postgres_storage_class:
description: Storage class to use for the PostgreSQL PVC
type: string
postgres_priority_class:
description: Assign a preexisting priority class to the postgres pod
type: string
postgres_data_path:
description: Path where the PostgreSQL data are located
type: string
Expand Down
12 changes: 7 additions & 5 deletions molecule/default/templates/awx_cr_molecule.yml.j2
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,13 +15,15 @@ spec:
kubernetes.io/ingress.class: nginx
web_resource_requirements:
requests:
cpu: 250m
memory: 128M
cpu: 100m
memory: 32M
task_resource_requirements:
requests:
cpu: 250m
memory: 128M
cpu: 100m
memory: 32M
ee_resource_requirements:
requests:
cpu: 200m
memory: 64M
memory: 32M
postgres_resource_requirements: {}
postgres_init_container_resource_requirements: {}
26 changes: 18 additions & 8 deletions roles/installer/defaults/main.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -169,23 +169,25 @@ web_command: []

task_resource_requirements:
requests:
cpu: 500m
memory: 1Gi
cpu: 100m
memory: 128Mi

web_resource_requirements:
requests:
cpu: 1000m
memory: 2Gi
cpu: 100m
memory: 128Mi

ee_resource_requirements:
requests:
cpu: 500m
memory: 1Gi
cpu: 100m
memory: 64Mi

# Customize CSRF options
csrf_cookie_secure: False
session_cookie_secure: False

# Assign a preexisting priority class to the control plane pods
control_plane_priority_class: ''
# Add extra environment variables to the AWX task/web containers. Specify as
# literal block. E.g.:
# task_extra_env: |
Expand Down Expand Up @@ -226,8 +228,16 @@ postgres_tolerations: ''
postgres_storage_requirements:
requests:
storage: 8Gi
postgres_init_container_resource_requirements: {}
postgres_resource_requirements: {}
postgres_resource_requirements:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@fosterseth I noticed this PR after putting up this other PR that makes a similar change: #876

I think your default request values may be a little too high (requests are guaranteed resources, it doesn't prevent the container from using more when needed).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's sync up next week on how to merge these. Proposal: if you fold in my changes in #876 and fix the conflicts, we can just merge this and close the other.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I pulled your commit into this PR

Copy link
Member

@rooftopcellist rooftopcellist Apr 20, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @fosterseth

I will pull this down and test it out again.

requests:
cpu: 10m
memory: 64Mi
postgres_init_container_resource_requirements:
requests:
cpu: 10m
memory: 64Mi
# Assign a preexisting priority class to the postgres pod
postgres_priority_class: ''
postgres_data_path: '/var/lib/postgresql/data/pgdata'

# Persistence to the AWX project data folder
Expand Down
3 changes: 3 additions & 0 deletions roles/installer/templates/deployment.yaml.j2
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,9 @@ spec:
{% for secret in image_pull_secrets %}
- name: {{ secret }}
{% endfor %}
{% endif %}
{% if control_plane_priority_class is defined %}
priorityClassName: '{{ control_plane_priority_class }}'
{% endif %}
initContainers:
{% if bundle_ca_crt or projects_persistence|bool or init_container_extra_commands %}
Expand Down
3 changes: 3 additions & 0 deletions roles/installer/templates/postgres.yaml.j2
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,9 @@ spec:
{% for secret in image_pull_secrets %}
- name: {{ secret }}
{% endfor %}
{% endif %}
{% if postgres_priority_class is defined %}
priorityClassName: '{{ postgres_priority_class }}'
{% endif %}
initContainers:
- name: database-check
Expand Down