Skip to content

Commit

Permalink
chore: improve v8 snapshot cache action to be more resilient to windo…
Browse files Browse the repository at this point in the history 
…ws slowness and have one PR per platform (#30551)

* Update update_v8_snapshot_cache.yml

* Update snapshot-meta.json

* Update snapshot-meta.json

* Update snapshot-meta.json

* Apply suggestions from code review

* Update update_v8_snapshot_cache.yml

* Update .github/workflows/update_v8_snapshot_cache.yml

* Update .github/workflows/update_v8_snapshot_cache.yml

* Update update_v8_snapshot_cache.yml

* chore: Update v8 snapshot cache - linux (#30552)

* chore: updating v8 snapshot cache

* chore: updating v8 snapshot cache

---------

Co-authored-by: cypress-bot[bot] <+cypress-bot[bot]@users.noreply.github.com>
Co-authored-by: Ryan Manuel <[email protected]>

* Update update_v8_snapshot_cache.yml

* chore: Update v8 snapshot cache - darwin (#30553)

Co-authored-by: cypress-bot[bot] <+cypress-bot[bot]@users.noreply.github.com>

* chore: Update v8 snapshot cache - windows (#30554)

Co-authored-by: cypress-bot[bot] <+cypress-bot[bot]@users.noreply.github.com>

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: cypress-bot[bot] <+cypress-bot[bot]@users.noreply.github.com>
  • Loading branch information
@ryanthemanuel @github-actions
3 people authored Nov 6, 2024
1 parent 5caacc2 commit 0618c88
Show file tree
Hide file tree
Showing 4 changed files with 75 additions and 37 deletions.
30 changes: 20 additions & 10 deletions .github/workflows/update_v8_snapshot_cache.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ concurrency:
jobs:
update-v8-snapshot-cache:
strategy:
max-parallel: 1
max-parallel: 3
matrix:
platform: [ubuntu-latest, macos-latest, windows-latest]
runs-on: ${{ matrix.platform }}
Expand All @@ -57,14 +57,20 @@ jobs:
steps:
- name: Determine snapshot files - Windows
if: ${{ matrix.platform == 'windows-latest' }}
run: echo "SNAPSHOT_FILES='tooling\v8-snapshot\cache\win32\snapshot-meta.json'" >> $GITHUB_ENV
run: |
echo "SNAPSHOT_FILES='tooling\v8-snapshot\cache\win32\snapshot-meta.json'" >> $GITHUB_ENV
echo "PLATFORM=windows" >> $GITHUB_ENV
shell: bash
- name: Determine snapshot files - Linux
if: ${{ matrix.platform == 'ubuntu-latest' }}
run: echo "SNAPSHOT_FILES='tooling/v8-snapshot/cache/linux/snapshot-meta.json'" >> $GITHUB_ENV
run: |
echo "SNAPSHOT_FILES='tooling/v8-snapshot/cache/linux/snapshot-meta.json'" >> $GITHUB_ENV
echo "PLATFORM=linux" >> $GITHUB_ENV
- name: Determine snapshot files - Mac
if: ${{ matrix.platform == 'macos-latest' }}
run: echo "SNAPSHOT_FILES='tooling/v8-snapshot/cache/darwin/snapshot-meta.json'" >> $GITHUB_ENV
run: |
echo "SNAPSHOT_FILES='tooling/v8-snapshot/cache/darwin/snapshot-meta.json'" >> $GITHUB_ENV
echo "PLATFORM=darwin" >> $GITHUB_ENV
- name: Install setuptools - Mac
if: ${{ matrix.platform == 'macos-latest' }}
run: sudo -H pip install setuptools
Expand All @@ -85,7 +91,8 @@ jobs:
node-version: 18
cache: 'yarn'
- name: Run yarn
run: yarn
# set the timeout here to try and deal with Windows slowness
run: yarn --network-timeout 300000
- name: Run build
run: yarn build
- name: Generate prod snapshot from scratch
Expand All @@ -108,14 +115,17 @@ jobs:
- name: Determine branch name - commit to separate branch
if: ${{ inputs.commit_directly_to_branch != true }}
run: |
echo "BRANCH_NAME=update-v8-snapshot-cache-on-${{ env.BASE_BRANCH }}" >> $GITHUB_ENV
echo "BRANCH_EXISTS=$(git show-ref --verify --quiet refs/remotes/origin/update-v8-snapshot-cache-on-${{ env.BASE_BRANCH }} && echo 'true')" >> $GITHUB_ENV
echo "BRANCH_NAME=update-v8-snapshot-cache-on-${{ env.BASE_BRANCH }}-${{ env.PLATFORM }}" >> $GITHUB_ENV
echo "BRANCH_EXISTS=$(git show-ref --verify --quiet refs/remotes/origin/update-v8-snapshot-cache-on-${{ env.BASE_BRANCH }}-${{ env.PLATFORM }} && echo 'true')" >> $GITHUB_ENV
shell: bash
- name: Check need for PR or branch update
id: check-need-for-pr
run: |
echo "needs_pr=${{ steps.check-for-v8-snapshot-cache-changes.outputs.has_changes == 'true' && env.BRANCH_EXISTS != 'true' }}" >> $GITHUB_OUTPUT
echo "number_of_prs_for_branch=$(gh api '/repos/cypress-io/cypress/pulls?head=cypress-io:${{ env.BRANCH_NAME }}' --jq length)" >> $GITHUB_OUTPUT
echo "needs_pr=${{ steps.check-for-v8-snapshot-cache-changes.outputs.has_changes == 'true' && env.BRANCH_NAME != env.BASE_BRANCH && steps.check-need-for-pr.outputs.number_of_prs_for_branch == '0' }}" >> $GITHUB_OUTPUT
echo "needs_branch_update=${{ steps.check-for-v8-snapshot-cache-changes.outputs.has_changes == 'true' && env.BRANCH_EXISTS == 'true' }}" >> $GITHUB_OUTPUT
env:
GH_TOKEN: ${{ secrets.BOT_GITHUB_ACTION_TOKEN }}
shell: bash
## Update available and a branch/PR already exists
- name: Checkout existing branch
Expand All @@ -128,7 +138,7 @@ jobs:
git merge --squash -Xtheirs stash
## Update available and a PR doesn't already exist
- name: Checkout new branch
if: ${{ steps.check-need-for-pr.outputs.needs_pr == 'true' }}
if: ${{ steps.check-need-for-pr.outputs.needs_branch_update != 'true' }}
run: git checkout -b ${{ env.BRANCH_NAME }} ${{ env.BASE_BRANCH }}
## Commit changes if present
- name: Commit the changes
Expand All @@ -152,7 +162,7 @@ jobs:
github,
baseBranch: '${{ env.BASE_BRANCH }}',
branchName: '${{ env.BRANCH_NAME }}',
description: 'Update v8 snapshot cache',
description: 'Update v8 snapshot cache - ${{ env.PLATFORM }}',
body: 'This PR was automatically generated by the [update-v8-snapshot-cache](https://github.com/cypress-io/cypress/actions/workflows/update_v8_snapshot_cache.yml) github action.',
reviewers: ['ryanthemanuel']
})
42 changes: 27 additions & 15 deletions tooling/v8-snapshot/cache/darwin/snapshot-meta.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -173,7 +173,7 @@
"./node_modules/@cypress/request/index.js",
"./node_modules/@cypress/request/lib/helpers.js",
"./node_modules/@cypress/request/lib/querystring.js",
"./node_modules/@cypress/request/node_modules/form-data/lib/form_data.js",
"./node_modules/@cypress/request/node_modules/tough-cookie/dist/cookie/canonicalDomain.js",
"./node_modules/@cypress/request/request.js",
"./node_modules/@ffmpeg-installer/ffmpeg/index.js",
"./node_modules/@graphql-tools/batch-execute/index.js",
Expand Down Expand Up @@ -280,6 +280,7 @@
"./node_modules/fluent-ffmpeg/lib/utils.js",
"./node_modules/follow-redirects/index.js",
"./node_modules/forever-agent/index.js",
"./node_modules/form-data/lib/form_data.js",
"./node_modules/front-matter/index.js",
"./node_modules/front-matter/node_modules/js-yaml/lib/js-yaml/loader.js",
"./node_modules/front-matter/node_modules/js-yaml/lib/js-yaml/schema/default_full.js",
Expand Down Expand Up @@ -1069,8 +1070,6 @@
"./node_modules/@cypress/commit-info/src/index.js",
"./node_modules/@cypress/commit-info/src/utils.js",
"./node_modules/@cypress/get-windows-proxy/node_modules/debug/src/common.js",
"./node_modules/@cypress/get-windows-proxy/node_modules/registry-js/dist/lib/index.js",
"./node_modules/@cypress/get-windows-proxy/node_modules/registry-js/dist/lib/registry.js",
"./node_modules/@cypress/get-windows-proxy/src/index.js",
"./node_modules/@cypress/parse-domain/build/tries/icann.complete.json",
"./node_modules/@cypress/parse-domain/build/tries/private.complete.json",
Expand All @@ -1091,7 +1090,24 @@
"./node_modules/@cypress/request/lib/multipart.js",
"./node_modules/@cypress/request/lib/redirect.js",
"./node_modules/@cypress/request/lib/tunnel.js",
"./node_modules/@cypress/request/node_modules/form-data/lib/populate.js",
"./node_modules/@cypress/request/node_modules/tough-cookie/dist/cookie/constants.js",
"./node_modules/@cypress/request/node_modules/tough-cookie/dist/cookie/cookie.js",
"./node_modules/@cypress/request/node_modules/tough-cookie/dist/cookie/cookieCompare.js",
"./node_modules/@cypress/request/node_modules/tough-cookie/dist/cookie/cookieJar.js",
"./node_modules/@cypress/request/node_modules/tough-cookie/dist/cookie/defaultPath.js",
"./node_modules/@cypress/request/node_modules/tough-cookie/dist/cookie/domainMatch.js",
"./node_modules/@cypress/request/node_modules/tough-cookie/dist/cookie/formatDate.js",
"./node_modules/@cypress/request/node_modules/tough-cookie/dist/cookie/index.js",
"./node_modules/@cypress/request/node_modules/tough-cookie/dist/cookie/parseDate.js",
"./node_modules/@cypress/request/node_modules/tough-cookie/dist/cookie/permutePath.js",
"./node_modules/@cypress/request/node_modules/tough-cookie/dist/getPublicSuffix.js",
"./node_modules/@cypress/request/node_modules/tough-cookie/dist/memstore.js",
"./node_modules/@cypress/request/node_modules/tough-cookie/dist/pathMatch.js",
"./node_modules/@cypress/request/node_modules/tough-cookie/dist/permuteDomain.js",
"./node_modules/@cypress/request/node_modules/tough-cookie/dist/store.js",
"./node_modules/@cypress/request/node_modules/tough-cookie/dist/utils.js",
"./node_modules/@cypress/request/node_modules/tough-cookie/dist/validators.js",
"./node_modules/@cypress/request/node_modules/tough-cookie/dist/version.js",
"./node_modules/@electron/fuses/dist/config.js",
"./node_modules/@electron/fuses/dist/constants.js",
"./node_modules/@electron/fuses/dist/index.js",
Expand Down Expand Up @@ -1577,6 +1593,7 @@
"./node_modules/binary-extensions/index.js",
"./node_modules/binaryextensions/edition-es5/index.js",
"./node_modules/bindings/bindings.js",
"./node_modules/brace-expansion/index.js",
"./node_modules/braces/index.js",
"./node_modules/braces/lib/compile.js",
"./node_modules/braces/lib/constants.js",
Expand Down Expand Up @@ -1624,6 +1641,8 @@
"./node_modules/chrome-remote-interface/node_modules/ws/lib/sender.js",
"./node_modules/chrome-remote-interface/node_modules/ws/lib/stream.js",
"./node_modules/chrome-remote-interface/node_modules/ws/lib/validation.js",
"./node_modules/ci-info/index.js",
"./node_modules/ci-info/vendors.json",
"./node_modules/cli-truncate/index.js",
"./node_modules/coffeescript/lib/coffeescript/lexer.js",
"./node_modules/coffeescript/lib/coffeescript/parser.js",
Expand Down Expand Up @@ -1907,6 +1926,7 @@
"./node_modules/fluent-ffmpeg/lib/processor.js",
"./node_modules/fluent-ffmpeg/lib/recipes.js",
"./node_modules/follow-redirects/debug.js",
"./node_modules/form-data/lib/populate.js",
"./node_modules/forwarded/index.js",
"./node_modules/fresh/index.js",
"./node_modules/front-matter/node_modules/js-yaml/index.js",
Expand Down Expand Up @@ -2186,9 +2206,6 @@
"./node_modules/ipaddr.js/lib/ipaddr.js",
"./node_modules/is-binary-path/index.js",
"./node_modules/is-buffer/index.js",
"./node_modules/is-ci/index.js",
"./node_modules/is-ci/node_modules/ci-info/index.js",
"./node_modules/is-ci/node_modules/ci-info/vendors.json",
"./node_modules/is-core-module/core.json",
"./node_modules/is-core-module/index.js",
"./node_modules/is-dotfile/index.js",
Expand All @@ -2197,6 +2214,7 @@
"./node_modules/is-fullwidth-code-point/index.js",
"./node_modules/is-glob/index.js",
"./node_modules/is-html/index.js",
"./node_modules/is-obj/index.js",
"./node_modules/is-plain-obj/index.js",
"./node_modules/is-regexp/index.js",
"./node_modules/is-stream/index.js",
Expand Down Expand Up @@ -2460,7 +2478,6 @@
"./node_modules/mime-types/index.js",
"./node_modules/mime/types.json",
"./node_modules/min-indent/index.js",
"./node_modules/minimatch/node_modules/brace-expansion/index.js",
"./node_modules/minimist/index.js",
"./node_modules/mocha-7.0.1/lib/browser/progress.js",
"./node_modules/mocha-7.0.1/lib/context.js",
Expand All @@ -2482,7 +2499,6 @@
"./node_modules/mocha-7.0.1/lib/reporters/markdown.js",
"./node_modules/mocha-7.0.1/lib/stats-collector.js",
"./node_modules/mocha-7.0.1/lib/utils.js",
"./node_modules/mocha-7.0.1/node_modules/brace-expansion/index.js",
"./node_modules/mocha-7.0.1/node_modules/debug/node_modules/ms/index.js",
"./node_modules/mocha-7.0.1/node_modules/debug/src/common.js",
"./node_modules/mocha-7.0.1/node_modules/glob/common.js",
Expand Down Expand Up @@ -3172,8 +3188,6 @@
"./node_modules/recast/parsers/babel.js",
"./node_modules/recast/parsers/esprima.js",
"./node_modules/recast/parsers/typescript.js",
"./node_modules/registry-js/dist/lib/index.js",
"./node_modules/registry-js/dist/lib/registry.js",
"./node_modules/request-promise-core/configure/request2.js",
"./node_modules/request-promise-core/errors.js",
"./node_modules/request-promise-core/lib/errors.js",
Expand Down Expand Up @@ -3313,7 +3327,6 @@
"./node_modules/stream-shift/index.js",
"./node_modules/string-width/index.js",
"./node_modules/stringify-object/index.js",
"./node_modules/stringify-object/node_modules/is-obj/index.js",
"./node_modules/strip-ansi/index.js",
"./node_modules/strip-eof/index.js",
"./node_modules/strip-final-newline/index.js",
Expand Down Expand Up @@ -3354,6 +3367,7 @@
"./node_modules/through/index.js",
"./node_modules/timm/lib/timm.js",
"./node_modules/tinycolor2/cjs/tinycolor.js",
"./node_modules/tldts/dist/cjs/index.js",
"./node_modules/to-fast-properties/index.js",
"./node_modules/to-regex-range/index.js",
"./node_modules/to-regex-range/node_modules/is-number/index.js",
Expand Down Expand Up @@ -3622,7 +3636,6 @@
"./packages/data-context/node_modules/ast-types/lib/shared.js",
"./packages/data-context/node_modules/ast-types/lib/types.js",
"./packages/data-context/node_modules/ast-types/main.js",
"./packages/data-context/node_modules/brace-expansion/index.js",
"./packages/data-context/node_modules/cross-spawn/index.js",
"./packages/data-context/node_modules/cross-spawn/lib/enoent.js",
"./packages/data-context/node_modules/cross-spawn/lib/parse.js",
Expand Down Expand Up @@ -4174,7 +4187,6 @@
"./packages/server/node_modules/p-queue/dist/priority-queue.js",
"./packages/server/node_modules/path-key/index.js",
"./packages/server/node_modules/path-to-regexp/index.js",
"./packages/server/node_modules/pump/index.js",
"./packages/server/node_modules/readable-stream/lib/internal/streams/async_iterator.js",
"./packages/server/node_modules/readable-stream/lib/internal/streams/destroy.js",
"./packages/server/node_modules/readable-stream/lib/internal/streams/end-of-stream.js",
Expand Down Expand Up @@ -4237,5 +4249,5 @@
"./tooling/v8-snapshot/cache/darwin/snapshot-entry.js"
],
"deferredHashFile": "yarn.lock",
"deferredHash": "4c1da762a851baab24ef5d1ba9d878ad207147c7703383f4796c0c9c7910240f"
"deferredHash": "48d21e834f4efdc929d52b19393d9aed9638636c4a64d18b75b70daf3c25511c"
}
2 changes: 1 addition & 1 deletion tooling/v8-snapshot/cache/linux/snapshot-meta.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4252,5 +4252,5 @@
"./tooling/v8-snapshot/cache/linux/snapshot-entry.js"
],
"deferredHashFile": "yarn.lock",
"deferredHash": "3f91247096ae79ca27c13f643c4d4f870ab6aa22f210ce68ddbcce8ea7050380"
"deferredHash": "48d21e834f4efdc929d52b19393d9aed9638636c4a64d18b75b70daf3c25511c"
}
Loading

5 comments on commit 0618c88

@cypress-bot
Copy link
Contributor

@cypress-bot cypress-bot bot commented on 0618c88 Nov 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Circle has built the linux x64 version of the Test Runner.

Learn more about this pre-release build at https://on.cypress.io/advanced-installation#Install-pre-release-version

Run this command to install the pre-release locally:

npm install https://cdn.cypress.io/beta/npm/13.15.3/linux-x64/develop-0618c88538aa0037353c3d79a0de3f1942feb507/cypress.tgz

@cypress-bot
Copy link
Contributor

@cypress-bot cypress-bot bot commented on 0618c88 Nov 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Circle has built the linux arm64 version of the Test Runner.

Learn more about this pre-release build at https://on.cypress.io/advanced-installation#Install-pre-release-version

Run this command to install the pre-release locally:

npm install https://cdn.cypress.io/beta/npm/13.15.3/linux-arm64/develop-0618c88538aa0037353c3d79a0de3f1942feb507/cypress.tgz

@cypress-bot
Copy link
Contributor

@cypress-bot cypress-bot bot commented on 0618c88 Nov 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Circle has built the darwin arm64 version of the Test Runner.

Learn more about this pre-release build at https://on.cypress.io/advanced-installation#Install-pre-release-version

Run this command to install the pre-release locally:

npm install https://cdn.cypress.io/beta/npm/13.15.3/darwin-arm64/develop-0618c88538aa0037353c3d79a0de3f1942feb507/cypress.tgz

@cypress-bot
Copy link
Contributor

@cypress-bot cypress-bot bot commented on 0618c88 Nov 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Circle has built the darwin x64 version of the Test Runner.

Learn more about this pre-release build at https://on.cypress.io/advanced-installation#Install-pre-release-version

Run this command to install the pre-release locally:

npm install https://cdn.cypress.io/beta/npm/13.15.3/darwin-x64/develop-0618c88538aa0037353c3d79a0de3f1942feb507/cypress.tgz

@cypress-bot
Copy link
Contributor

@cypress-bot cypress-bot bot commented on 0618c88 Nov 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Circle has built the win32 x64 version of the Test Runner.

Learn more about this pre-release build at https://on.cypress.io/advanced-installation#Install-pre-release-version

Run this command to install the pre-release locally:

npm install https://cdn.cypress.io/beta/npm/13.15.3/win32-x64/develop-0618c88538aa0037353c3d79a0de3f1942feb507/cypress.tgz

Please sign in to comment.