add initial playwright suite #835
Conversation
✅ Deploy Preview for veda-ui ready!
To edit notification comments on pull requests, go to your Netlify site configuration. |
|
Thank you for taking the initiative of adding E2E tests, I think we should start by asking do we want integration tests? Why do we want integration tests and do we think it's helpful? With the refactor epic coming up, much of this wiring would change since we are moving from the concept of having page / container components that veda-ui currently controls to feature components that instances would compose together. If we were to add integration tests, it would make more sense for integration tests to live on the instance side. But I think we can cover our bases with unit tests if we are just trying to make sure specific components are rendering what they are supposed to. Its best to defer these tests till we start tackling the refactor because unfortunately much of this may end up being throw-away work. |
|
thanks for the feedback @sandrahoang686 . The motivation on this is that smoke testing after upgrades is currently done manually on feature branches. The thought being we could allow the veda-deploy process to have a step to automate the test runs. Originally, I had the tests in the veda-config repo which you can see here prior to the commit to move things into the veda-ui after feedback during review. Would you suggest closing out this PR and merging in the config PR at that previous state? That would allow for some regression testing on the config instance after the UI is restructured as well as permitting the post deploy integration testing to be automated. |
|
closing in favor of placing the test in veda-config to be run by veda-deploy: NASA-IMPACT/veda-config#363 |
This adds a playwright test suite that will work for the veda-ui pages generated with the mock data as well as allow the veda-config repo to use those same tests on all of the stories and datasets in the config repo. It does this by generating a json containing the names of all datasets and stories by parsing the
mdxfiles. All tests are written in a way that they can be run within theveda-uirepo leveraging the mdx files in the mocks directory. A separate PR in theveda-configwill be able to run the same tests against the actual stories and datasets. NASA-IMPACT/veda-config#363Each tests ensures that the pages render with the name and that there are no javascript errors on those pages.
The tests verify that all /catalog and /dataset routes load with tiles for the expected.
There is a test of the explore route for each dataset that ensures each one loads without a javascript error.
There is also a test flow that makes a general search on the /analysis page.
Interactions with the pages are accomplished with locators defined as Page Objects in the
e2e/pagesdirectory.A github workflow is included to run the tests against the mocks on all pull requests. The test run will have a report with trace to allow for debugging of failed tests. That report is retained for 30 days. To successfully complete all tests, a github secret should be added with
MAPBOX_TOKEN. Without this, some pages attempting to render the mapbox maps will cause tests to fail.Assumptions: in veda-ui, the mock data lives in the
/mocksdirectory and all mdx files that are rendered into stories or catalogs have anameproperty and all datasets also have anid.How to test
Run the yarn install to pull in the new dependencies. Run the script yarn test:e2e to verify that the tests run locally.** To verify that tests catch errors, modify pages to break them and ensure that the test suite finds the failure.
** you may be asked to install the playwright browsers on the first run of the test suite.