Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🏗 Sync Chrome version used by Visual tests with Percy backend #32774

Merged
merged 2 commits into from
Feb 22, 2021
Merged

🏗 Sync Chrome version used by Visual tests with Percy backend #32774

merged 2 commits into from
Feb 22, 2021

Conversation

rsimha
Copy link
Contributor

@rsimha rsimha commented Feb 20, 2021

Visual tests have been flaky of late because the Percy backend was using Chrome v78. We therefore pinned our CI to that version while running visual tests. Now, Percy has moved to v87.

This PR pins our CI to the current stable channel version v88. This will be the way to go until Percy agrees to regularly update to Chrome stable channel.

@amp-owners-bot
Copy link

Hey @danielrozenberg! These files were changed:

build-system/tasks/visual-diff/index.js

build-system/tasks/visual-diff/index.js Outdated Show resolved Hide resolved
@rsimha
Copy link
Contributor Author

rsimha commented Feb 22, 2021

This version did not work: https://app.circleci.com/pipelines/github/ampproject/amphtml/2336/workflows/c7aa7989-fe8a-44da-8125-00ba1cae5f32/jobs/25482/parallel-runs/0/steps/0-108

@danielrozenberg Any other ideas? Do you know if there's a way to tell which sub-versions of Chrome 87 will work with puppeteer?

@danielrozenberg
Copy link
Member

This version did not work: https://app.circleci.com/pipelines/github/ampproject/amphtml/2336/workflows/c7aa7989-fe8a-44da-8125-00ba1cae5f32/jobs/25482/parallel-runs/0/steps/0-108

@danielrozenberg Any other ideas? Do you know if there's a way to tell which sub-versions of Chrome 87 will work with puppeteer?

No idea why, but r812852 is just nowhere to be found, it's like it's been purged off the CI build storage... r827102 (88.0.4324.0) is the next version after that and that number works for me. I think using that version would be the best option we have for now as the skew is tiny and there doesn't seem to be any changes that should affect visual diffs: https://support.google.com/chrome/a/answer/7679408#88

@rsimha
Copy link
Contributor Author

rsimha commented Feb 22, 2021

New snapshots all appear to be more correct than the current baseline. Let's merge this and pay attention to overall flakiness.

@rsimha rsimha merged commit 3d0c39b into ampproject:master Feb 22, 2021
@rsimha rsimha deleted the 2021-02-19-PercyChromeVersion branch February 22, 2021 21:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants