chore: GHA to smoke test against last released exemplar #2983
Conversation
safeer
added
the
run-all
safeer
force-pushed
the
saf/smoketest_gha
branch
23 times, most recently
from
07c135d
to
5725b01
Compare
safeer
changed the title
| cache: true | ||
| - name: Build Cache | ||
| uses: ./.github/actions/build-cache | ||
| - name: Docker Compose |
There was a problem hiding this comment.
Everything is handled by kube, we should not need docker compose or initdb
There was a problem hiding this comment.
Thanks, yeah gonna do some cleanup here
| @@ -0,0 +1,95 @@ | |||
| on: | |||
| pull_request: | |||
There was a problem hiding this comment.
I think we actually want this to be on push, as we are not using merge groups any more.
| jobs: | ||
| smoke-test-upgrade-path: | ||
| name: Smoke Test Upgrade Path | ||
| # if: github.event_name != 'pull_request' || github.event.action == 'enqueued' || contains( github.event.pull_request.labels.*.name, 'run-all') |
There was a problem hiding this comment.
I think we need to uncomment this before merging.
| @@ -375,3 +375,57 @@ jobs: | |||
| else | |||
| echo "Integration tests passed" | |||
| fi | |||
| smoke-test-released-exemplar: | |||
There was a problem hiding this comment.
I think this should be removed from ci.yaml, as it is duplicated in smoketest.yaml, which also runs on PRs?
| fetch-tags: true | ||
| - name: Get last tagged release | ||
| run: | | ||
| latest_release=$(git tag --sort=-v:refname | grep -v v1 | head -n 1) |
There was a problem hiding this comment.
At some point (not now) we are going to need to think about testing older releases. The 'latest release' is often just a few commits ago.
| - name: Replace the tagged release smoketest with the current smoketest | ||
| run: | | ||
| set -euo pipefail | ||
| echo "Replacing tagged release smoketest with current smoketest" |
There was a problem hiding this comment.
This approach will break if the smoke test adds new features that are in the new version but not the old version, as you are testing the new code again old FTL.
This is fine for now but I think but will need some thought going forward.
| run: | | ||
| set -euo pipefail | ||
| echo "Deploying the tagged release to the cluster" | ||
| cd deployment && just deploy-version ${{ env.LATEST_VERSION }} && just wait-for-kube && cd .. |
There was a problem hiding this comment.
wait-for-kube is already in the harness, if you leave it off here it will speed up the test.
…s released binaries
safeer
force-pushed
the
saf/smoketest_gha
branch
from
e1fb708
to
1dda917
Compare
|
Ingress is working, but looks like PubSub is also borked in the kube environment. Working through that. @stuartwdouglas thanks for the review! Definitely addressing these.. I should've marked this as draft again 😂 |
This adds a GitHub Action that smoke tests branches against the exemplar in the latest tagged release.
It will catch regressions that may affect users on FTL upgrades, but it doesn't specifically test the no-downtime upgrade path that users will go through. That's upcoming; in order to do it, the exemplar will first need to be run in FTL's long living cluster.