Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

test: Automatic GQL performance regression tests #1337

Merged
merged 9 commits into from
Feb 15, 2024
Merged

test: Automatic GQL performance regression tests #1337

merged 9 commits into from
Feb 15, 2024

Conversation

bprusinowski
Copy link
Collaborator

@bprusinowski bprusinowski commented Feb 14, 2024 •
edited
Loading

This PR introduces a first iteration of automatic GQL performance regression tests on successful deployment and consolidates the code related to GQL performance testing.

@vercel Vercel
Copy link

vercel bot commented Feb 14, 2024 •
edited
Loading

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
visualization-tool ✅ Ready (Inspect) Visit Preview 💬 Add feedback Feb 15, 2024 3:25pm

@bprusinowski bprusinowski changed the title chore: Automatic GQL performance regression tests test: Automatic GQL performance regression tests Feb 14, 2024
@bprusinowski bprusinowski marked this pull request as ready for review February 15, 2024 12:00
ptbrowne
ptbrowne approved these changes Feb 15, 2024
View reviewed changes
Copy link
Collaborator

@ptbrowne ptbrowne left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am wondering if we could not have the k6 commands as part of our usual tests done via jest (maybe using k6 programmatically) ? This way we stay within our test framework, and maybe we would not have the intermediate step of generating the yml files ?

.github/workflows/performance-tests-pr.yml Outdated
k6 run --tag testid=DataCubePreview --env ENV=PR --env ENDPOINT=${{ github.event.deployment_status.target_url }}/api/graphql --env CUBE_IRI=https://energy.ld.admin.ch/elcom/electricityprice --env CUBE_LABEL=Elcom - </root/k6/performance-tests/graphql/DataCubePreview.js &&
k6 run --tag testid=PossibleFilters --env ENV=PR --env ENDPOINT=${{ github.event.deployment_status.target_url }}/api/graphql --env CUBE_IRI=https://energy.ld.admin.ch/sfoe/bfe_ogd84_einmalverguetung_fuer_photovoltaikanlagen/9 --env CUBE_LABEL=Photovoltaikanlagen/9 - </root/k6/performance-tests/graphql/PossibleFilters.js &&
k6 run --tag testid=PossibleFilters --env ENV=PR --env ENDPOINT=${{ github.event.deployment_status.target_url }}/api/graphql --env CUBE_IRI=https://environment.ld.admin.ch/foen/nfi/nfi_C-20/cube/2023-3 --env CUBE_LABEL=NFI/2023-3 - </root/k6/performance-tests/graphql/PossibleFilters.js &&
k6 run --tag testid=PossibleFilters --env ENV=PR --env ENDPOINT=${{ github.event.deployment_status.target_url }}/api/graphql --env CUBE_IRI=https://energy.ld.admin.ch/elcom/electricityprice --env CUBE_LABEL=Elcom - </root/k6/performance-tests/graphql/PossibleFilters.js
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you extract this part to a script ? This way we have less reliance on github actions & we can run the script locally.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think still we would have problems to run it locally, as it references github.event.deployment_status.target_url, right? Let me know if I missed something again 😅

@bprusinowski
Copy link
Collaborator Author

@ptbrowne thanks for reviewing 🙇‍♂️

I am not sure if it's a good idea to have automatic PR performance tests in jest – we could potentially have "red" status that would prevent us from building / deploying the application if some unit-performance test fails even if it was randomly related to e.g. LINDAS not being up, right? Also, I think the application only deploys to Vercel once unit tests "pass", and here we need to wait for deployment to run the performance tests.

Let me know if I'm missing something, we can also sync on this 👀

@ptbrowne
Copy link
Collaborator

Sorry I was unclear. I was meaning that we would put the test in jests but outside of the test suite required to pass, I think using the "projects" feature from jest would enable to have the main "project" being used for CI and another "performance-e2e" project that could be run only if we want it.

I see the following advantages of using jest:

  • Integration with vscode to run performance test from the editor or the CLI
  • Ability to only focus on one performance test
  • Abilty to have reports (test / fail) per performance test

@bprusinowski bprusinowski force-pushed the chore/update-k6-performance-tests branch from e49a0d6 to 30fc211 Compare February 15, 2024 13:37
@bprusinowski bprusinowski force-pushed the chore/update-k6-performance-tests branch from 39ef604 to 51d0bb6 Compare February 15, 2024 14:22
@bprusinowski bprusinowski force-pushed the chore/update-k6-performance-tests branch from b23dbd4 to e7f93dc Compare February 15, 2024 15:17
@bprusinowski bprusinowski merged commit 8bee42d into main Feb 15, 2024
4 of 7 checks passed
@bprusinowski bprusinowski deleted the chore/update-k6-performance-tests branch February 15, 2024 15:36
@bprusinowski bprusinowski linked an issue Feb 20, 2024 that may be closed by this pull request
Automated regression testing for every GraphQL query #1297
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ptbrowne ptbrowne ptbrowne approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Automated regression testing for every GraphQL query
2 participants
@bprusinowski @ptbrowne