-
Notifications
You must be signed in to change notification settings - Fork 192
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Use GitHub Actions services (for PostgreSQL and RabbitMQ) #3901
Merged
sphuber
merged 6 commits into
aiidateam:develop
from
CasperWA:use_github_actions_services
Apr 8, 2020
Merged
Changes from all commits
Commits
Show all changes
6 commits
Select commit
Hold shift + click to select a range
a77e018
Test CI using postgres through `services`
CasperWA 0b9e53f
Use host auth method `trust`
CasperWA 97feff8
Remove all traces of old postgres action
CasperWA 120c2b1
Completely remove postgres action, move to service
CasperWA 5b2e100
Try moving also RabbitMQ to services
CasperWA d863518
Completely move RabbitMQ to services
CasperWA File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -110,15 +110,27 @@ jobs: | |
| backend: ['django', 'sqlalchemy'] | ||
| python-version: [3.5, 3.8] | ||
|
|
||
| services: | ||
| postgres: | ||
| image: postgres:10 | ||
| env: | ||
| POSTGRES_DB: test_${{ matrix.backend }} | ||
| POSTGRES_PASSWORD: '' | ||
| POSTGRES_HOST_AUTH_METHOD: trust | ||
| options: >- | ||
| --health-cmd pg_isready | ||
| --health-interval 10s | ||
| --health-timeout 5s | ||
| --health-retries 5 | ||
| ports: | ||
| - 5432:5432 | ||
| rabbitmq: | ||
| image: rabbitmq:latest | ||
| ports: | ||
| - 5672:5672 | ||
|
|
||
| steps: | ||
| - uses: actions/checkout@v2 | ||
| - uses: CasperWA/[email protected] | ||
| with: | ||
| postgresql version: '10' | ||
| postgresql db: test_${{ matrix.backend }} | ||
| postgresql user: postgres | ||
| postgresql password: '' | ||
| postgresql auth: trust | ||
|
|
||
| - name: Set up Python ${{ matrix.python-version }} | ||
| uses: actions/setup-python@v1 | ||
|
|
@@ -127,13 +139,9 @@ jobs: | |
|
|
||
| - name: Install system dependencies | ||
| run: | | ||
| wget -O - "https://github.com/rabbitmq/signing-keys/releases/download/2.0/rabbitmq-release-signing-key.asc" | sudo apt-key add - | ||
| echo 'deb https://dl.bintray.com/rabbitmq-erlang/debian bionic erlang' | sudo tee -a /etc/apt/sources.list.d/bintray.rabbitmq.list | ||
| echo 'deb https://dl.bintray.com/rabbitmq/debian bionic main' | sudo tee -a /etc/apt/sources.list.d/bintray.rabbitmq.list | ||
| sudo rm -f /etc/apt/sources.list.d/dotnetdev.list /etc/apt/sources.list.d/microsoft-prod.list | ||
| sudo apt update | ||
| sudo apt install postgresql-10 rabbitmq-server graphviz | ||
| sudo systemctl status rabbitmq-server.service | ||
| sudo apt install postgresql-10 graphviz | ||
|
|
||
| - name: Upgrade pip | ||
| run: | | ||
|
|
||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -98,15 +98,27 @@ jobs: | |
| python-version: [3.5, 3.6, 3.7, 3.8] | ||
| backend: ['django', 'sqlalchemy'] | ||
|
|
||
| services: | ||
| postgres: | ||
| image: postgres:10 | ||
| env: | ||
| POSTGRES_DB: test_${{ matrix.backend }} | ||
| POSTGRES_PASSWORD: '' | ||
| POSTGRES_HOST_AUTH_METHOD: trust | ||
| options: >- | ||
| --health-cmd pg_isready | ||
| --health-interval 10s | ||
| --health-timeout 5s | ||
| --health-retries 5 | ||
| ports: | ||
| - 5432:5432 | ||
| rabbitmq: | ||
| image: rabbitmq:latest | ||
| ports: | ||
| - 5672:5672 | ||
|
|
||
| steps: | ||
| - uses: actions/checkout@v2 | ||
| - uses: CasperWA/[email protected] | ||
| with: | ||
| postgresql version: '10' | ||
| postgresql db: test_${{ matrix.backend }} | ||
| postgresql user: postgres | ||
| postgresql password: '' | ||
| postgresql auth: trust | ||
|
|
||
| - name: Set up Python ${{ matrix.python-version }} | ||
| uses: actions/setup-python@v1 | ||
|
|
@@ -115,13 +127,9 @@ jobs: | |
|
|
||
| - name: Install system dependencies | ||
| run: | | ||
| wget -O - "https://github.com/rabbitmq/signing-keys/releases/download/2.0/rabbitmq-release-signing-key.asc" | sudo apt-key add - | ||
| echo 'deb https://dl.bintray.com/rabbitmq-erlang/debian bionic erlang' | sudo tee -a /etc/apt/sources.list.d/bintray.rabbitmq.list | ||
| echo 'deb https://dl.bintray.com/rabbitmq/debian bionic main' | sudo tee -a /etc/apt/sources.list.d/bintray.rabbitmq.list | ||
| sudo rm -f /etc/apt/sources.list.d/dotnetdev.list /etc/apt/sources.list.d/microsoft-prod.list | ||
| sudo apt update | ||
| sudo apt install postgresql-10 rabbitmq-server graphviz | ||
| sudo systemctl status rabbitmq-server.service | ||
| sudo apt install postgresql-10 graphviz | ||
|
|
||
| - run: pip install --upgrade pip | ||
|
|
||
|
|
||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -17,17 +17,29 @@ jobs: | |
| backend: ['django', 'sqlalchemy'] | ||
| python-version: [3.5, 3.6, 3.7, 3.8] | ||
|
|
||
| services: | ||
| postgres: | ||
| image: postgres:10 | ||
| env: | ||
| POSTGRES_DB: test_${{ matrix.backend }} | ||
| POSTGRES_PASSWORD: '' | ||
| POSTGRES_HOST_AUTH_METHOD: trust | ||
| options: >- | ||
| --health-cmd pg_isready | ||
| --health-interval 10s | ||
| --health-timeout 5s | ||
| --health-retries 5 | ||
| ports: | ||
| - 5432:5432 | ||
| rabbitmq: | ||
| image: rabbitmq:latest | ||
| ports: | ||
| - 5672:5672 | ||
|
|
||
| steps: | ||
| - uses: actions/checkout@v2 | ||
| with: | ||
| ref: ${{ github.event.client_payload.head_ref }} | ||
| - uses: CasperWA/[email protected] | ||
| with: | ||
| postgresql version: '10' | ||
| postgresql db: test_${{ matrix.backend }} | ||
| postgresql user: postgres | ||
| postgresql password: '' | ||
| postgresql auth: trust | ||
|
|
||
| - name: Set up Python ${{ matrix.python-version }} | ||
| uses: actions/setup-python@v1 | ||
|
|
@@ -36,13 +48,9 @@ jobs: | |
|
|
||
| - name: Install system dependencies | ||
| run: | | ||
| wget -O - "https://github.com/rabbitmq/signing-keys/releases/download/2.0/rabbitmq-release-signing-key.asc" | sudo apt-key add - | ||
| echo 'deb https://dl.bintray.com/rabbitmq-erlang/debian bionic erlang' | sudo tee -a /etc/apt/sources.list.d/bintray.rabbitmq.list | ||
| echo 'deb https://dl.bintray.com/rabbitmq/debian bionic main' | sudo tee -a /etc/apt/sources.list.d/bintray.rabbitmq.list | ||
| sudo rm -f /etc/apt/sources.list.d/dotnetdev.list /etc/apt/sources.list.d/microsoft-prod.list | ||
| sudo apt update | ||
| sudo apt install postgresql-10 rabbitmq-server graphviz | ||
| sudo systemctl status rabbitmq-server.service | ||
| sudo apt install postgresql-10 graphviz | ||
|
|
||
| - run: pip install --upgrade pip | ||
|
|
||
|
|
||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why do we still need to install postgresql-10 manually here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Due to the PostgreSQL tests. It may be that we do not need to after @ltalirz has moved these tests out of the package, but I am not sure?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
More specifically, all tests using this class will not run properly - at least that was my prior experience when messing around with these workflows adding the, now obsolete, PostgreSQL GitHub Action.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If you are mentioning the moving out of the
PGSUclass, then that already happened. The PR was merged yesterday. Did you try to remove it and see if the tests still run? If so what was the failure?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ah that makes sense, because there it relies on
pg_testto create a postgres cluster on the localhost. So if the service just exposes a postgres database on some other machine, then we still have to install it manuallyThere was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Concerning the removal of
postgresql-10: It is indeed not possible, see here.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah - there's some weird stuff happening. Hmm, also since the GitHub Pages were having some maintenance, the Docs CI is failing (I'm guessing that's why).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Since there are "only" 3 packages left (although there may be a lot of dependencies) we could download the packages and keep them under
/.ci/or similar and install them from there? I don't know if this is a viable solution, but it would avoid the fragilesudo apt update.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The
aiida-corerepository is already over 100MB, so I would be really hesitant to start vendoring entire system software packages just to make the CI work.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I found this page commenting on the issue. See also actions/runner-images#675