Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reducing flakiness in the tests #47

Merged
merged 11 commits into from
Jun 20, 2022
Merged

Reducing flakiness in the tests #47

merged 11 commits into from
Jun 20, 2022

Conversation

kafkasl
Copy link
Contributor

@kafkasl kafkasl commented Jun 17, 2022

Background

Link to issue

https://biomage.atlassian.net/browse/BIOMAGE-1969

Changes

Code changes

I have tried to decrease tests variability as much:

  • Removing ALL experiments in the testing account before the tests
  • before() hook runs before each test case. Test cases should not rely on ordering, so I removed the tests cases and merged them into a single one (because it's very short anyway)
  • Replaced samples by smaller ones
  • Changed some random stuff that didn't add value
  • Added cache to speed up tests a bit.
  • Removing all tests is something that happens across the whole env, so now integration tests can't run at the same time in a given environment.
  • If a new integration test for a given branch is scheduled, last one is auto-cancelled.

Definition of DONE

Your changes will be ready for merging after each of the steps below have been completed:

Testing

  • Unit tests written
  • Tested locally with Inframock (with latest production data downloaded with biomage experiment pull)
  • Deployed to staging

To set up easy local testing with inframock, follow the instructions here: https://github.com/hms-dbmi-cellenics/inframock
To deploy to the staging environment, follow the instructions here: https://github.com/hms-dbmi-cellenics/biomage-utils

Documentation updates

Is all relevant documentation updated to reflect the proposed changes in this PR?

  • Relevant Github READMEs updated
  • Relevant wiki pages created/updated

Approvers

  • Approved by a member of the core engineering team
  • (UX changes) Approved by vickymorrison (this is her username, tag her if you need approval)

Just before merging:

Optional

  • Photo of a cute animal attached to this PR

@kafkasl kafkasl changed the title Updated deps Reducing flakiness in the tests Jun 17, 2022
@kafkasl kafkasl merged commit 2004717 into master Jun 20, 2022
@kafkasl kafkasl deleted the updated-deps branch June 20, 2022 11:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants