Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Integration Testing Proposal #88

Open
Ian2012 opened this issue Oct 24, 2024 · 5 comments · May be fixed by #80
Open

Integration Testing Proposal #88

Ian2012 opened this issue Oct 24, 2024 · 5 comments · May be fixed by #80

Comments

@Ian2012
Copy link
Contributor

Ian2012 commented Oct 24, 2024

This discussion proposes a solution for the integration testing implementation on Harmony.

Requirements

  • Everything should run in Github Action in a reasonable time (<20 min) without considering OpenedX initialization itself.
  • Every step can be reproduced locally with documentation on how to test it.
  • Run multiple scenarios such as:
    • All helmCharts enabled
    • ElasticSearch
    • OpenSearch
    • Running multiple OpenedX installations in the same cluster
  • Set up different tests for every cluster/environment to verify domains and deployments are working, etc.

Proposed solution

Kind runs k8s on a cluster and works pretty well for our purposes, this tool is already explored on Aspects integration tests. The POC implemented here #80 proposes a folder structure and a GitHub action workflow to test different configurations of the helm chart in parallel, a template for adding new integration testing scenarios, and an initial scenario to setup and test ElasticSearch.
@Ian2012
Copy link
Contributor Author

Ian2012 commented Oct 24, 2024

tagging you here as you are probably the ones more interested in this issue. @bradenmacdonald @gabor-boros @jfavellar90

@bradenmacdonald
Copy link
Contributor

I would also add that the integration test code itself should be something that's easy to maintain. If it's complex and fragile, it may not be worth the effort.

@Ian2012
Copy link
Contributor Author

Ian2012 commented Nov 5, 2024

Following this discussion, are there any specific scenarios we need to cover? The POC already has a test for ElasticSearch but I don't have any other scenarios in mind to test.

@bradenmacdonald
Copy link
Contributor

For Sumac, we'll actually need Meilisearch instead of Elasticsearch :p

@Ian2012
Copy link
Contributor Author

Ian2012 commented Nov 6, 2024

Once support is added I will add the integration testing scenarios. Will the current implementation cover most of our use cases?

I tested ClickHouse and ElasticSearch and found and resolved an issue with ElasticSearch disabled. We can also merge the integration testing PR and start adding scenarios as part of the new features/fixes.

@Ian2012 Ian2012 linked a pull request Nov 6, 2024 that will close this issue
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

test: add integration testing eduNEXT/openedx-k8s-harmony
2 participants
@bradenmacdonald @Ian2012