[Question]: How to cache on github actions? #7249
Comments
|
General recommendation is to not cache them, since they are downloaded in less than 10 seconds. If you are still want to cache them, you find the locations here. The cache key should contain the OS and Playwright version. |
|
It's taking ages to install playwright. But on the actual tests run step, I get the following error: When I list the folder, I can actually see the files but with Is there a way to install a certain webkit version? |
|
Are you using self-hosted runners? If this problem is persistent on the hosted-runners, it might make sense to contact the GitHub Actions support. We run 50-100 jobs daily and didn't experience such big delays yet. Also you are using a private NPM Registry, which might add delay. But the browsers are not getting downloaded over NPM, thats a normal http request then. Running |
|
We are using the gha runners. We have no self hosted stuff. The strange thing is - the Mac machines run consistently slower in everything. The tests run roughly 50% longer and running Anyway, I've managed to solve the issue. On the Mac machine running the tests I've added this code: Now it has a cache for playwright only. I runs a lot faster than the actual After the change: I will check what might be the issue with gha on mac for us as we thought it is a common issue for everyone. Thanks a lot! |
|
@YonatanKra 8 minutes! This is too much! We ourselves run installation on GHA-based macs, and it's never that bad. The whole Is it a public repo? Would you mind sharing a link to the GHA action so that I can look into logs? |
|
Closing since the issue seems stale as part of the triage process. Please create a new issue if you still face any issues. 8 Minutes is definitely not as-designed but sounds more a infra issue. |
Hi, |
|
I extended the above caching logic to support new browser binaries as the version of Playwright changes. This works for - name: Install dependencies
run: yarn
# Figures out the version of playwright that's installed.
# 1. Because we don't know what version yarn will resolve it to, we have
# to use `yarn why` to get the actually installed version.
# 2. Because we're in a workspace, we need to make sure we get the version
# for the root and not any children, hence the `grep`. If not using
# workspaces, this can be skipped.
# 3. jq comes pre-installed in the Ubuntu runner, so we use that to get
# the correct version string.
# 4. Finally, we use sed to extract just the version number (eg; '1.22.0')
# The result is stored in steps.playwright-version.outputs.version
- name: Get installed Playwright version
id: playwright-version
run: echo "::set-output name=version::$(yarn why --json @playwright/test | grep -h 'workspace:.' | jq --raw-output '.children[].locator' | sed -e 's/@playwright\/test@.*://')"
# Attempt to restore the correct Playwright browser binaries based on the
# currently installed version of Playwright (The browser binary versions
# may change with Playwright versions).
# Note: Playwright's cache directory is hard coded because that's what it
# says to do in the docs. There doesn't appear to be a command that prints
# it out for us.
- uses: actions/cache@v3
id: playwright-cache
with:
path: '~/.cache/ms-playwright'
key: '${{ runner.os }}-playwright-${{ steps.playwright-version.outputs.version }}'
# As a fallback, if the Playwright version has changed, try use the
# most recently cached version. There's a good chance that at least one
# of the browser binary versions haven't been updated, so Playwright can
# skip installing that in the next step.
# Note: When falling back to an old cache, `cache-hit` (used below)
# will be `false`. This allows us to restore the potentially out of
# date cache, but still let Playwright decide if it needs to download
# new binaries or not.
restore-keys: |
${{ runner.os }}-playwright-
# If the Playwright browser binaries weren't able to be restored, we tell
# paywright to install everything for us.
- name: Install Playwright's dependencies
if: steps.playwright-cache.outputs.cache-hit != 'true'
run: npx playwright install --with-deps |
|
It might be my version of yarn but echo "::set-output name=version::$(cat yarn.lock | grep "^\"@playwright\/test" --after-context 1 | tail -n 1 | sed 's/\s*version "\([0-9.]\+\)"/\1/')"Furthermore, I found that while the above caches the downloaded versions, the playwright runtime dependencies still need to be installed. Hence, I ended up with the following for the last step: - name: Install Playwright with dependencies
if: steps.playwright-cache.outputs.cache-hit != 'true'
run: npx playwright install --with-deps
- name: Install Playwright's dependencies
if: steps.playwright-cache.outputs.cache-hit == 'true'
run: npx playwright install-deps |
This is crossplatform version of this: node -e "console.log(require('playwright/package.json').version)" |
|
@NikolayMakhonin that only works if |
I think we would need to use
{
"version": "0.1.0",
"name": "your-project-name",
"dependencies": {
"@playwright/test": {
"version": "1.24.0",
"resolved": "https://registry.npmjs.org/@playwright/test/-/test-1.24.0.tgz"
}
}
} |
|
@andykenward |
@chrisvariety Thank you. That's working for me.
|
|
@jesstelford Thanks for sharing your config! Just like @birtles, I'm wondering how this manages installing / caching the ubuntu dependencies. If I just cache |
|
Hmm, I'm not sure about how GitHub Actions caches OS deps. Perhaps if it's done in a seperate step, GitHub will cache the generated Docker image, making subsequent installs faster? |
You can separate the two parts and cache the binaries but still install the host system dependencies. https://justin.poehnelt.com/posts/caching-playwright-in-github-actions/ |
|
if someone needs to set a cleaner version using npm: output |
|
does any of this work anymore??? I've tried and it just keeps saying cache not found nor does it ever create it..... |
|
Also wondering what @gselsidi asked. |
This worked for us after seeing some timeouts installing playwright today. Confirmed we're hitting the cache in our github actions. - name: Cache Playwright
uses: actions/cache@v3
id: playwright-cache
with:
path: ~/.cache/ms-playwright
key: ${{ runner.os }}-playwright-1.26.1
- name: Install Playwright
if: steps.playwright-cache.outputs.cache-hit != 'true'
run: pnpm dlx [email protected] install --with-depsyou could probably use the suggestion here to make it a little more robust and replace the hardcoded version # with whatever version you have in |
|
I got it to work @zkrzyzanowski thing is how can we cache all the ubuntu library stuff? What's the path for that? This still takes massive amounts of time:
|
|
Looks like all this caching stuff is pretty annoying and should probably use containers, I tried the default container and it's done super quickly. |
We had the ubuntu stuff download on the first run, then pulled from cache every run after. Either way it sounds like you found a solution based on your comment here |
|
Yes, same here using Playwright container in github and it's working like a charm 🎉 for all browsers including webkit: |
Thanks a lot :) |
|
Just my 2 cents for getting playwright version in a very simple and short manner: npx playwright -V | awk '{ print $2 }'Or you can use |
|
@avetisk |
|
Yarn edition with disabled colors because of the following phenomenon:
- name: Store Playwright's Version
run: |
PLAYWRIGHT_VERSION=$(YARN_ENABLE_COLORS=false yarn info @playwright/test | grep @playwright | sed 's/.*://')
echo "Playwright's Version: $PLAYWRIGHT_VERSION"
echo "PLAYWRIGHT_VERSION=$PLAYWRIGHT_VERSION" >> $GITHUB_ENV
- name: Cache Playwright Browsers for Playwright's Version
id: cache-playwright-browsers
uses: actions/cache@v3
with:
path: ~/.cache/ms-playwright
key: playwright-browsers-${{ env.PLAYWRIGHT_VERSION }}
- name: Setup Playwright Browsers
if: steps.cache-playwright-browsers.outputs.cache-hit != 'true'
run: yarn playwright install --with-deps |
|
Hmm it is not working.. Edit: solution was here: #7249 (comment) |
|
I confirmed this script works for my case: - name: Get Playwright version
run: echo "PLAYWRIGHT_VERSION=$(jq '.devDependencies["@playwright/test"]' package.json -r)" >> $GITHUB_ENV
- name: Cache Playwright Browsers
id: cache-playwright-browsers
uses: actions/cache@v4
with:
path: ~/.cache/ms-playwright
key: playwright-browsers-${{ runner.os }}-${{ env.PLAYWRIGHT_VERSION }}
- name: Install Playwright with dependencies
if: steps.cache-playwright-browsers.outputs.cache-hit != 'true'
run: bun playwright install --with-deps
- name: Install Playwright dependencies
if: steps.cache-playwright-browsers.outputs.cache-hit == 'true'
run: bun playwright install-deps
- name: Download test build
uses: actions/download-artifact@v4
with:
name: test-build
path: dist/
- name: Run Playwright tests
run: bun playwright test -c playwright.service.config.ts --workers=20
env:
DOTENV_KEY: ${{ secrets.DOTENV_KEY }}
PLAYWRIGHT_SERVICE_ACCESS_TOKEN: ${{ secrets.PLAYWRIGHT_SERVICE_ACCESS_TOKEN }}
PLAYWRIGHT_SERVICE_URL: ${{ secrets.PLAYWRIGHT_SERVICE_URL }}
PLAYWRIGHT_SERVICE_RUN_ID: ${{ github.run_id }}-${{ github.run_attempt }}-${{ github.sha }}
- uses: actions/upload-artifact@v4
if: always()
with:
name: playwright-report
path: playwright-report/ |
Citation needed ;) I don't want to sound entitled, but it would really be great if we had a first-party |
Can someone more clearly explain why both of these are needed? I think the documentation makes it seem like those commands should be equivalent... Shouldn't it be sufficient to just do I'm quite confused. Perhaps the docs could be updated to clarify exactly all the different pieces involved (playwright library itself, browsers, browser deps, playwright os deps, whatever), and which get installed (and where) with different commands. |
|
I tried to get the browsers caching working on Ubuntu but ran into errors regarding missing OS-level libs so living without caching for now. The docs state that downloading browsers is as fast as loading them from cache; it takes 1 minute to finish just the Update: After a couple of attempts I'm continuing with the Docker image provided by Playwright. It just does not make sense to manually do the caching. Also found this: https://radekmie.dev/blog/on-playwright-in-github-actions/ |
|
If someone needs to get the - name: Get installed Playwright version
run: echo "PLAYWRIGHT_VERSION=$(npm ll -p --depth=0 playwright | grep -o '@.*')" >> $GITHUB_ENV(Note that Then with:
path: "~/.cache/ms-playwright"
key: "${{ runner.os }}-playwright-${{ env.PLAYWRIGHT_VERSION }}" |
|
If, like me, you:
Here is the corresponding yml - name: Install node modules
run: npm install
env:
PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD: 1
# Figures out the version of playwright that's installed.
# 1. Because we don't know what version npm will resolve it to
# 2. Because we're in a workspace, we need to make sure we get the version
# for the root and not any children, hence the `grep`. If not using
# workspaces, this can be skipped.
# The result is stored in $PLAYWRIGHT_VERSION
# https://github.com/microsoft/playwright/issues/7249#issuecomment-2299065930
# https://stackoverflow.com/a/33426237/24573072
# https://github.com/actions/runner/issues/1636#issuecomment-1024531638
- name: Detect Playwright version (Windows)
if: runner.os == 'windows'
run: echo "PLAYWRIGHT_VERSION=$(npm ll -p --depth=0 playwright | grep -o '@.*')" | Out-File -FilePath $env:GITHUB_ENV -Append
- name: Detect Playwright version (Linux and Mac)
if: runner.os != 'windows'
run: echo "PLAYWRIGHT_VERSION=$(npm ll -p --depth=0 playwright | grep -o '@.*')" >> $GITHUB_ENV
- name: Put $HOME in env
if: runner.os == 'windows'
run: echo "HOME=$HOME" | Out-File -FilePath $env:GITHUB_ENV -Append
# Attempt to restore the correct Playwright browser binaries based on the
# currently installed version of Playwright (The browser binary versions
# may change with Playwright versions).
# Note: Playwright's cache directory is hard coded because that's what it
# says to do in the docs. There doesn't appear to be a command that prints
# it out for us.
- name: Cache playwright binaries
uses: actions/cache@v4
id: playwright-cache
with:
# see https://playwright.dev/docs/browsers#managing-browser-binaries
path: ${{ runner.os == 'Windows' && format('{0}{1}', env.HOME, '\AppData\Local\ms-playwright') || runner.os == 'Linux' && '~/.cache/ms-playwright' || '~/Library/Caches/ms-playwright' }}
key: ${{ runner.os }}-playwright-${{ env.PLAYWRIGHT_VERSION }}
# As a fallback, if the Playwright version has changed, try use the
# most recently cached version. There's a good chance that at least one
# of the browser binary versions haven't been updated, so Playwright can
# skip installing that in the next step.
# Note: When falling back to an old cache, `cache-hit` (used below)
# will be `false`. This allows us to restore the potentially out of
# date cache, but still let Playwright decide if it needs to download
# new binaries or not.
restore-keys: |
${{ runner.os }}-playwright-
# If the Playwright browser binaries weren't able to be restored, we tell
# paywright to install everything for us.
- name: Install Playwright with dependencies
if: steps.playwright-cache.outputs.cache-hit != 'true'
run: npx playwright install --with-deps
# If browser binaries are available we ask playwright to install os specific deps
# For example on linux it will install ffmpeg, xvfb, and many more libs
- name: Install Playwright's dependencies
if: steps.playwright-cache.outputs.cache-hit == 'true'
run: npx playwright install-deps |
|
Okay I think I maybe understand this a little bit better now? I think the command/flag names are a bit confusing, and maybe there should be a first-party But in the meantime, I've made this general-purpose composite action: name: Install Playwright
description: Install Playwright and dependencies with cache
# https://github.com/microsoft/playwright/issues/7249
inputs:
working-directory:
description: Where to install Playwright
default: ./
browsers:
description: Browsers to install
default: chromium webkit firefox
outputs:
version:
description: Installed version of Playwright
value: ${{ steps.version.outputs.version }}
cache-hit:
description: Whether cache for Playwright was found
value: ${{ steps.cache.outputs.cache-hit }}
runs:
using: composite
steps:
- name: Get Playwright version
uses: actions/github-script@v7
id: version
with:
script: |
// https://github.com/actions/toolkit/issues/1624
// const workingDirectory = core.getInput("working-directory");
const workingDirectory = "${{ inputs.working-directory }}";
console.debug("Specified working directory:", workingDirectory);
if (workingDirectory) process.chdir(workingDirectory);
console.debug("Actual working directory:", process.cwd());
let version = "";
try {
version = require("@playwright/test/package.json").version;
} catch (error) {
console.log(error.message);
}
console.debug("Version:", version);
if (version) {
core.exportVariable("PLAYWRIGHT_VERSION", version);
core.setOutput("version", version);
} else core.setFailed("Couldn't get Playwright version");
- name: Cache Playwright
id: cache
uses: actions/cache@v4
with:
path: ~/.cache/ms-playwright
key: playwright-${{ env.PLAYWRIGHT_VERSION }}
- name: Install Playwright and its dependencies
shell: bash
if: steps.cache.outputs.cache-hit != 'true'
working-directory: ${{ inputs.working-directory }}
run: npx playwright install ${{ inputs.browsers }} --with-deps
- name: Install just Playwright's dependencies
shell: bash
if: steps.cache.outputs.cache-hit == 'true'
working-directory: ${{ inputs.working-directory }}
run: npx playwright install-deps
Drop it in e.g. Then to use it: - name: Install packages
run: npm install
with:
working-directory: ./some-subfolder-if-applicable
- name: Install Playwright
id: install-playwright
uses: ./.github/actions/install-playwright
with:
working-directory: ./some-subfolder-if-applicable
browsers: chromium webkit
- run: echo Playwright version: "${{ steps.install-playwright.outputs.version }}"
- run: echo Playwright cached: "${{ steps.install-playwright.outputs.cache-hit }}" |
|
Is there maybe the possibility to also cache the dependencies and not only the browsers? 🤔 |
|
@vincerubinetti Thank you so much for your action!! I've just set it up in one of my repos and it works like butter! 🧈 |
I could be very wrong about this, but this is my understanding. Please correct me if I'm wrong.
Where "browsers" means the specific browser executables, and "dependencies" means any system dependencies the browsers need, e.g. visual studio c++ redistributables on Windows. Because "dependencies" are installed in different places across the system and perhaps quite large, they're difficult to cache. And then to interact with the browsers from tests and such, that's where you install the Playwright npm "packages", not to be confused with "dependencies" here. Again, it'd be really good if the docs explained things at a high level like this. |
and others
Hi,
I've setup github actions on Linux and it works fine.
I'm now running the same tests on MacOS and the binaries are not the same, so caching the installation is not working (for obvious reasons).
What is the recommended way to cache the playwright binaries for MacOS on github actions?
The text was updated successfully, but these errors were encountered: