-
Notifications
You must be signed in to change notification settings - Fork 915
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Runs GitHub workflow unit tests in band
* Removes caching from GitHub test workflow to simplify.
* Removes "bad apples" check from GitHub workflow since those tests now
pass consistently.
* Runs unit tests in band to reduce execution time from ~90 minutes to
~20 minutes.
* Removes the logic that ignored failing unit tests.
* Adds mocha tests to the workflow.
* Adds ci-specific scripts to simplify the commands.
* `yarn test:jest:ci` and `yarn test:jest_integration:ci` are now used
in the workflow.
Resolves #1231
Signed-off-by: Tommy Markley <[email protected]>- Loading branch information
Tommy Markley
committed
Mar 3, 2022
1 parent
082f44a
commit 79a9a2b
Showing
7 changed files
with
22 additions
and
135 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -11,7 +11,6 @@ on: | |
| branches: [ '**', '!feature/**' ] | ||
|
|
||
| env: | ||
| CACHE_NAME: osd-node-modules | ||
| TEST_BROWSER_HEADLESS: 1 | ||
| CI: 1 | ||
| GCS_UPLOAD_PREFIX: fake | ||
|
|
@@ -26,108 +25,40 @@ jobs: | |
| runs-on: ubuntu-latest | ||
| name: Build and Verify | ||
| steps: | ||
| # Access a cache of set results from a previous run of the job | ||
| # This is to prevent re-running steps that were already successful since it is not native to github actions | ||
| # Can be used to verify flaky steps with reduced times | ||
| - name: Restore the cached run | ||
| uses: actions/cache@v2 | ||
| with: | ||
| path: | | ||
| job_successful | ||
| linter_results | ||
| unit_tests_results | ||
| integration_tests_results | ||
| key: ${{ github.run_id }}-${{ github.job }}-${{ github.sha }} | ||
| restore-keys: | | ||
| ${{ github.run_id }}-${{ github.job }}-${{ github.sha }} | ||
| - name: Get if previous job was successful | ||
| id: job_successful | ||
| run: cat job_successful 2>/dev/null || echo 'false' | ||
|
|
||
| - name: Get the previous linter results | ||
| id: linter_results | ||
| run: cat linter_results 2>/dev/null || echo 'default' | ||
|
|
||
| - name: Get the previous unit tests results | ||
| id: unit_tests_results | ||
| run: cat unit_tests_results 2>/dev/null || echo 'default' | ||
|
|
||
| - name: Get the previous integration tests results | ||
| id: integration_tests_results | ||
| run: cat integration_tests_results 2>/dev/null || echo 'default' | ||
|
|
||
| - name: Checkout code | ||
| if: steps.job_successful.outputs.job_successful != 'true' | ||
| uses: actions/checkout@v2 | ||
|
|
||
| - name: Setup Node | ||
| if: steps.job_successful.outputs.job_successful != 'true' | ||
| uses: actions/setup-node@v2 | ||
| with: | ||
| node-version-file: ".nvmrc" | ||
| registry-url: 'https://registry.npmjs.org' | ||
|
|
||
| - name: Setup Yarn | ||
| if: steps.job_successful.outputs.job_successful != 'true' | ||
| run: | | ||
| npm uninstall -g yarn | ||
| npm i -g [email protected] | ||
| - name: Run bootstrap | ||
| if: steps.job_successful.outputs.job_successful != 'true' | ||
| run: yarn osd bootstrap | ||
|
|
||
| - name: Run linter | ||
| if: steps.linter_results.outputs.linter_results != 'success' | ||
| id: linter | ||
| run: yarn lint | ||
|
|
||
| # Runs unit tests while limiting workers because github actions will spawn more than it can handle and crash | ||
| # Continues on error but will create a comment on the pull request if this step failed. | ||
| - name: Run unit tests | ||
| if: steps.unit_tests_results.outputs.unit_tests_results != 'success' | ||
| id: unit-tests | ||
| continue-on-error: true | ||
| run: node scripts/jest --ci --colors --maxWorkers=10 | ||
| env: | ||
| SKIP_BAD_APPLES: true | ||
|
|
||
| - run: echo Unit tests completed unsuccessfully. However, unit tests are inconsistent on the CI so please verify locally with `yarn test:jest`. | ||
| if: steps.unit_tests_results.outputs.unit_tests_results != 'success' && steps.unit-tests.outcome != 'success' | ||
|
|
||
| # TODO: This gets rejected, we need approval to add this | ||
| # - name: Add comment if unit tests did not succeed | ||
| # if: steps.unit_tests_results.outputs.unit_tests_results != 'success' && steps.unit-tests.outcome != 'success' | ||
| # uses: actions/github-script@v5 | ||
| # with: | ||
| # github-token: ${{ secrets.GITHUB_TOKEN }} | ||
| # script: | | ||
| # github.rest.issues.createComment({ | ||
| # issue_number: context.issue.number, | ||
| # owner: context.repo.owner, | ||
| # repo: context.repo.repo, | ||
| # body: 'Unit tests completed unsuccessfully. However, unit tests are inconsistent on the CI so please verify locally with `yarn test:jest`.' | ||
| # }) | ||
| run: yarn test:jest:ci | ||
|
|
||
| - name: Run mocha tests | ||
| id: mocha-tests | ||
| run: yarn test:mocha | ||
|
|
||
| - name: Run integration tests | ||
| if: steps.integration_tests_results.outputs.integration_tests_results != 'success' | ||
| id: integration-tests | ||
| run: node scripts/jest_integration --ci --colors --max-old-space-size=5120 | ||
|
|
||
| # Set cache if linter, unit tests, and integration tests were successful then the job will be marked successful | ||
| # Sets individual results to empower re-runs of the same build without re-running successful steps. | ||
| - if: | | ||
| (steps.linter.outcome == 'success' || steps.linter.outcome == 'skipped') && | ||
| (steps.unit-tests.outcome == 'success' || steps.unit-tests.outcome == 'skipped') && | ||
| (steps.integration-tests.outcome == 'success' || steps.integration-tests.outcome == 'skipped') | ||
| run: echo "::set-output name=job_successful::true" > job_successful | ||
| - if: steps.linter.outcome == 'success' || steps.linter.outcome == 'skipped' | ||
| run: echo "::set-output name=linter_results::success" > linter_results | ||
| - if: steps.unit-tests.outcome == 'success' || steps.unit-tests.outcome == 'skipped' | ||
| run: echo "::set-output name=unit_tests_results::success" > unit_tests_results | ||
| - if: steps.integration-tests.outcome == 'success' || steps.integration-tests.outcome == 'skipped' | ||
| run: echo "::set-output name=integration_tests_results::success" > integration_tests_results | ||
| run: yarn test:jest_integration:ci | ||
|
|
||
| functional-tests: | ||
| needs: [ build-lint-test ] | ||
| runs-on: ubuntu-latest | ||
|
|
@@ -138,76 +69,39 @@ jobs: | |
| steps: | ||
| - run: echo Running functional tests for ciGroup${{ matrix.group }} | ||
|
|
||
| # Access a cache of set results from a previous run of the job | ||
| # This is to prevent re-running a CI group that was already successful since it is not native to github actions | ||
| # Can be used to verify flaky steps with reduced times | ||
| - name: Restore the cached run | ||
| uses: actions/cache@v2 | ||
| with: | ||
| path: | | ||
| ftr_tests_results | ||
| key: ${{ github.run_id }}-${{ github.job }}-${{ matrix.group }}-${{ github.sha }} | ||
| restore-keys: | | ||
| ${{ github.run_id }}-${{ github.job }}-${{ matrix.group }}-${{ github.sha }} | ||
| - name: Get the cached tests results | ||
| id: ftr_tests_results | ||
| run: cat ftr_tests_results 2>/dev/null || echo 'default' | ||
|
|
||
| - name: Checkout code | ||
| if: steps.ftr_tests_results.outputs.ftr_tests_results != 'success' | ||
| uses: actions/checkout@v2 | ||
|
|
||
| - name: Setup Node | ||
| if: steps.ftr_tests_results.outputs.ftr_tests_results != 'success' | ||
| uses: actions/setup-node@v2 | ||
| with: | ||
| node-version-file: ".nvmrc" | ||
| registry-url: 'https://registry.npmjs.org' | ||
|
|
||
| - name: Setup Yarn | ||
| if: steps.ftr_tests_results.outputs.ftr_tests_results != 'success' | ||
| run: | | ||
| npm uninstall -g yarn | ||
| npm i -g [email protected] | ||
| - name: Get cache path | ||
| if: steps.ftr_tests_results.outputs.ftr_tests_results != 'success' | ||
| id: cache-path | ||
| run: echo "::set-output name=CACHE_DIR::$(yarn cache dir)" | ||
|
|
||
| - name: Setup cache | ||
| if: steps.ftr_tests_results.outputs.ftr_tests_results != 'success' | ||
| uses: actions/cache@v2 | ||
| with: | ||
| path: ${{ steps.cache-path.outputs.CACHE_DIR }} | ||
| key: ${{ runner.os }}-yarn-${{ env.CACHE_NAME }}-${{ hashFiles('**/yarn.lock') }} | ||
| restore-keys: | | ||
| ${{ runner.os }}-yarn-${{ env.CACHE_NAME }}- | ||
| ${{ runner.os }}-yarn- | ||
| ${{ runner.os }}- | ||
| # github virtual env is the latest chrome | ||
| - name: Setup chromedriver | ||
| if: steps.ftr_tests_results.outputs.ftr_tests_results != 'success' | ||
| run: yarn add --dev [email protected] | ||
|
|
||
| - name: Run bootstrap | ||
| if: steps.ftr_tests_results.outputs.ftr_tests_results != 'success' | ||
| run: yarn osd bootstrap | ||
|
|
||
| - name: Build plugins | ||
| if: steps.ftr_tests_results.outputs.ftr_tests_results != 'success' | ||
| run: node scripts/build_opensearch_dashboards_platform_plugins --no-examples --workers 10 | ||
|
|
||
| - if: steps.ftr_tests_results.outputs.ftr_tests_results != 'success' | ||
| - name: Run CI test group ${{ matrix.group }} | ||
| id: ftr-tests | ||
| run: node scripts/functional_tests.js --config test/functional/config.js --include ciGroup${{ matrix.group }} | ||
| env: | ||
| CI_GROUP: ciGroup${{ matrix.group }} | ||
| CI_PARALLEL_PROCESS_NUMBER: ciGroup${{ matrix.group }} | ||
| JOB: ci${{ matrix.group }} | ||
| CACHE_DIR: ciGroup${{ matrix.group }} | ||
|
|
||
| - if: steps.ftr-tests.outcome == 'success' || steps.ftr-tests.outcome == 'skipped' | ||
| run: echo "::set-output name=ftr_tests_results::success" > ftr_tests_results | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters