Add screenshots on E2E test failures (#4898)

* Add test environment to catch errors

* Add jest-circus and jest-html-reporters

This is so we can log the screenshots and errors that happen during E2E tests.

* Use the jest reporters and jest-circus to run the E2E tests

* Force cart test to break

Revert this!

* Upload jest report artifact when tests fail

* Run artifact upload on failure

* Upload entire report directory to artifact

* Revert "Force cart test to break"

This reverts commit c57f2c5.

* Update .gitignore and jest.config

* Force cart test to break

* Force cart test to break

* Remove <rootDir> from jest-html-reporter path

This was causing the artefact not to upload.

* Revert "Force cart test to break"

Co-authored-by: Niels Lange <[email protected]>

.github/workflows/php-js-e2e-tests.yml

@@ -121,6 +121,13 @@ jobs:
                   npm run wp-env clean all
                   npm run test:e2e
 
+            - name: Upload artifacts on failure
+              if: ${{ failure() }}
+              uses: actions/upload-artifact@v2
+              with:
+                name: e2e-with-gutenberg-test-report
+                path: reports/e2e
+
     JSE2ETests:
         name: JavaScript E2E Tests (latest)
         needs: Setup
@@ -182,3 +189,10 @@ jobs:
                   npm run wp-env start
                   npm run wp-env clean all
                   npm run test:e2e
+                
+            - name: Upload artifacts on failure
+              if: ${{ failure() }}
+              uses: actions/upload-artifact@v2
+              with:
+                name: e2e-test-report
+                path: reports/e2e

.gitignore

@@ -38,6 +38,7 @@ tests/cli/vendor
 /tmp
 /tests/bin/tmp
 /tests/e2e-tests/config/local-*.json
+/reports
 
 # Logs
 /logs