Upgrade Playwright and Apache Arrow dependencies #2387

jattasNI · 2024-09-05T20:51:28Z

Pull Request

🤨 Rationale

#1747 has a task to manually update Playwright and Apache Arrow since their versions should stay in sync across npm and nuget.

Was hoping this would help with #2169 because the new Playwright Webkit now has offscreen canvas support that allows these tests to pass. But there still seem to be issues on Windows (see thread in PR).

👩‍💻 Implementation

Upgrade both dependencies across both package managers to the latest release that is available.

🧪 Testing

Locally ran tests in Playwright Webkit and Safari.

Spot checked the wafer map in storybook with the experimental renderer enabled and didn't see any differences.

✅ Checklist

I have updated the project documentation to reflect my changes or determined no changes are needed.

jattasNI · 2024-09-06T17:26:53Z

change/@ni-jasmine-parameterized-6f3cf07b-2ad9-4dd6-ae3c-dcf4eb9af2d9.json

@@ -0,0 +1,7 @@
+{


Notes on test failures:
Playwright 1.46 contains Chrome 128, Playwright 1.47 contains Chrome 129 (but isn't available on Nuget yet)

Tests pass in headless Playwright Webkit and Firefox
Tests fail in headless Playwright Chromium 128
Tests pass in headed Playwright Chromium 128, headed macOS Chrome 128 or 129 if browser has focus
Tests fail in headed Playwright Chromium 128, headed macOS Chrome 129 (didn't try 128) if browser doesn't have focus

We saw some new test failures in SLE when we upgraded to Playwright 1.46. In that case it was that something changed behavior in Chrome when a test opened a new tab, and the fix was that tests shouldn't open new tabs. Bug 2823451: SystemLinkShared Renovate upgrade of Playwright is blocked due to file-lib tests which open new browser tabs

Test failures are nearly all in table tests, mostly with keyboard nav

FAILED Table action menu when action menu button is blurred, cell fires cell-action-menu-blur event and then moving cursor slightly to right causes no column width changes FAILED TableCell fires the cell-view-focus-in event when cell contents are focused, after the cell is already focused FAILED TableCell fires cell-view-focus-in/cell-focus-in/cell-blur events when cell contents are focused/blurred FAILED Table keyboard navigation with interactive columns and cell[0, 1] focused, when the interactive content in the cell is focused, and then a scroll happens, the interactive content is blurred FAILED Table keyboard navigation with interactive columns and cell[0, 1] focused, when the interactive content in the cell is focused, and then a data update happens, the matching cell will be focused afterwards FAILED Table keyboard navigation with interactive columns and cell[0, 1] focused, when the interactive content in the cell is focused, pressing the given key will focus the cell: F2 FAILED Table keyboard navigation with interactive columns and cell[0, 1] focused, when the interactive content in the cell is focused, pressing the given key will focus the cell: Escape FAILED Table keyboard navigation with interactive columns and cell[0, 1] focused, pressing the given key will focus the interactive cell content in the current cell: F2 FAILED Table keyboard navigation with interactive columns and cell[0, 1] focused, pressing the given key will focus the interactive cell content in the current cell: Enter FAILED Table keyboard navigation with interactive columns and cell[0, 1] focused, each Shift-Tab press will focus each of the remaining tabbable elements in the row (to the left), then will move focus past the table FAILED Table keyboard navigation with interactive columns and cell[0, 1] focused, each Tab press will focus each of the remaining tabbable elements in the row (to the right), then will move focus past the table FAILED Table keyboard navigation with interactive columns with the first row focused, clicking interactive cell content in another row will focus that content, and an Esc key press will refocus the cell FAILED Table keyboard navigation with non-interactive columns for a large dataset (virtualized) table with cell[0, 0] focused, with a cell focused, Ctrl-End focuses the cell (same column) in the last row, and Ctrl-Home focuses the cell (same column) in the 1st row FAILED Table keyboard navigation with non-interactive columns for a large dataset (virtualized) table with cell[0, 0] focused, with a row focused, End focuses the last row, and Home focuses the 1st row FAILED Table keyboard navigation with non-interactive columns for a large dataset (virtualized) table with cell[0, 0] focused, with a row focused, PgUp will focus a row one page (table viewport height of rows) up FAILED Table keyboard navigation with non-interactive columns for a large dataset (virtualized) table with cell[0, 0] focused, with a row focused, PgDown will focus a row one page (table viewport height of rows) down FAILED Table keyboard navigation with non-interactive columns for a large dataset (virtualized) table with cell[0, 0] focused, with a cell focused, if the table is scrolled away from that cell, then back to it, it is focused again FAILED Table keyboard navigation with non-interactive columns for a large dataset (virtualized) table with cell[0, 0] focused, with a row focused, if the table is scrolled away from that row, then back to it, it is focused again FAILED Table keyboard navigation with non-interactive columns for a table with a grouped column, when a group row is focused, for a collapsed group row, pressing the given key will expand the group: Enter FAILED Table keyboard navigation with non-interactive columns for a table with a grouped column, when a group row is focused, for a collapsed group row, pressing the given key will expand the group: RightArrow FAILED Table keyboard navigation with non-interactive columns for a table with a grouped column, when a group row is focused, for a collapsed group row, pressing the given key will expand the group: Space FAILED Table keyboard navigation with non-interactive columns for a table with a grouped column, when a group row is focused, for an expanded group row, pressing the given key will collapse the group: Space FAILED Table keyboard navigation with non-interactive columns for a table with a grouped column, when a group row is focused, for an expanded group row, pressing the given key will collapse the group: LeftArrow FAILED Table keyboard navigation with non-interactive columns for a table with a grouped column, when a group row is focused, for an expanded group row, pressing the given key will collapse the group: Enter FAILED Table keyboard navigation with non-interactive columns for a table with a grouped column, when a group row is focused, when multiple selection is enabled pressing RightArrow on an focused expanded group row will focus the selection checkbox FAILED Table keyboard navigation with non-interactive columns for a table with a grouped column, when a group row is focused, when multiple selection is enabled pressing LeftArrow on a focused group row selection checkbox will focus the entire group row FAILED Table keyboard navigation with non-interactive columns for a table with a grouped column, when a group row is focused, with no tabbable elements in the group row, pressing Tab will move focus past the table FAILED Table keyboard navigation with non-interactive columns for a table with a grouped column, when a group row is focused, with no tabbable elements in the group row, pressing Shift-Tab will move focus past the table FAILED Table keyboard navigation with non-interactive columns for a multiple-selection table with focus, when a row is focused, pressing DownArrow multiple times when the row selection checkbox is focused will focus the row selection checkbox on subsequent rows FAILED Table keyboard navigation with non-interactive columns for a multiple-selection table with focus, when a row is focused, pressing Space will select the row FAILED Table keyboard navigation with non-interactive columns for a multiple-selection table with focus, when a row is focused, pressing End when the row selection checkbox is focused will focus the last cell FAILED Table keyboard navigation with non-interactive columns for a multiple-selection table with focus, when a row is focused, pressing RightArrow will focus the row selection checkbox FAILED Table keyboard navigation with non-interactive columns for a multiple-selection table with focus, when a row is focused, pressing Home when a cell is focused will focus the row selection checkbox FAILED Table keyboard navigation with non-interactive columns for a table with hierarchy when an expanded row is focused, pressing RightArrow will move to the 1st cell of the row (and the row stays expanded) FAILED Table keyboard navigation with non-interactive columns for a table with hierarchy when a collapsed row is focused, pressing RightArrow will expand the row FAILED Table keyboard navigation with non-interactive columns for a table with hierarchy when an expanded row is focused, pressing LeftArrow will collapse the row FAILED Table keyboard navigation with non-interactive columns for a simple table (no hierarchy/grouping/selection) with focus when a cell is focused, its action menu is visible #SkipFirefox #SkipWebkit FAILED Table keyboard navigation with non-interactive columns for a simple table (no hierarchy/grouping/selection) with focus when a cell action menu is opened via mouse click, scrolling the table will close the action menu and focus the cell FAILED Table keyboard navigation with non-interactive columns for a simple table (no hierarchy/grouping/selection) with focus when a cell action menu is opened via keyboard, scrolling the table will close the action menu and focus the cell FAILED Table keyboard navigation with non-interactive columns for a simple table (no hierarchy/grouping/selection) with focus consecutive Enter key presses will sort the focused column (ascending, descending, no sort) FAILED Table keyboard navigation with non-interactive columns for a simple table (no hierarchy/grouping/selection) with focus DownArrow key press will focus the cell in the same column in the next row FAILED Table keyboard navigation with non-interactive columns for a simple table (no hierarchy/grouping/selection) with focus on initial table focus, the 1st column header is focused if sortable FAILED Table keyboard navigation with non-interactive columns for a simple table (no hierarchy/grouping/selection) with focus after a DownArrow key press (from 1st column header), cell[0, 0] is focused FAILED Table keyboard navigation with non-interactive columns for a simple table (no hierarchy/grouping/selection) with focus starting with cell[0, 0] focused, when the table data changes (same or greater row count), the focused cell is maintained (row at the same index) FAILED Table keyboard navigation with non-interactive columns for a simple table (no hierarchy/grouping/selection) with focus starting with cell[0, 0] focused, consecutive RightArrow presses focus each cell to the right in the same row, stopping at the last cell FAILED Table keyboard navigation with non-interactive columns for a simple table (no hierarchy/grouping/selection) with focus starting with cell[0, 0] focused, with no tabbable elements in the row, pressing Shift-Tab will move focus past the table FAILED Table keyboard navigation with non-interactive columns for a simple table (no hierarchy/grouping/selection) with focus starting with cell[0, 0] focused, from the last cell being focused, consecutive LeftArrow presses focus each cell to the left in the same row FAILED Table keyboard navigation with non-interactive columns for a simple table (no hierarchy/grouping/selection) with focus starting with cell[0, 0] focused, when the table data changes (fewer rows than previously focused row index), the focused cell resets to be for the 1st row FAILED Table keyboard navigation with non-interactive columns for a simple table (no hierarchy/grouping/selection) with focus starting with cell[0, 0] focused, pressing LeftArrow will focus the entire row FAILED Table keyboard navigation with non-interactive columns for a simple table (no hierarchy/grouping/selection) with focus starting with cell[0, 0] focused, with no tabbable elements in the row, pressing Tab will move focus past the table FAILED Table keyboard navigation with non-interactive columns for a simple table (no hierarchy/grouping/selection) with focus starting with cell[0, 0] focused, End will focus the last cell in the same row, and Home will focus the 1st cell in the same row FAILED Table keyboard navigation with non-interactive columns for a simple table (no hierarchy/grouping/selection) with focus starting with cell[0, 0] focused, consecutive DownArrow presses focus each cell to the bottom in subsequent rows FAILED Table keyboard navigation with non-interactive columns for a simple table (no hierarchy/grouping/selection) with focus starting with cell[0, 0] focused, clicking a cell in a new row, then a cell in the original row, will focus the clicked (original) row FAILED Table keyboard navigation with non-interactive columns for a simple table (no hierarchy/grouping/selection) with focus starting with cell[0, 0] focused, when the table data changes to zero rows, the focus resets to the header FAILED Table keyboard navigation with non-interactive columns for a simple table (no hierarchy/grouping/selection) with focus when a row is focused, all action menus in that row are visible (and otherwise hidden) #SkipFirefox #SkipWebkit FAILED Table keyboard navigation with non-interactive columns for a simple table (no hierarchy/grouping/selection) with focus when a row is focused, pressing Space has no effect when selectionMode is none (row not selected) FAILED Table keyboard navigation with non-interactive columns for a simple table (no hierarchy/grouping/selection) with focus UpArrow key press will focus the cell in the same column in the previous row FAILED Table keyboard navigation with non-interactive columns for a simple table (no hierarchy/grouping/selection) with focus consecutive RightArrow presses focus each header to the right, stopping at the last header FAILED Table keyboard navigation with non-interactive columns for a simple table (no hierarchy/grouping/selection) with focus from the last header being focused, consecutive LeftArrow presses focus each header to the left, stopping at the 1st header FAILED Table keyboard navigation with non-interactive columns for a simple table (no hierarchy/grouping/selection) with focus a row is tabindex 0 when focused, and tabindex -1 when clicked away from FAILED Table keyboard navigation with non-interactive columns for a simple table (no hierarchy/grouping/selection) with focus Shift-Enter key presses will sort the focused column (ascending, descending, no sort) and maintain other sorted columns FAILED Table keyboard navigation with non-interactive columns for a simple table with no columns sortable, on initial table focus, the 1st row is focused FAILED Table keyboard navigation with non-interactive columns for a single-selection table with focus when a cell is focused, pressing Shift-Space will select the row, and pressing it again will deselect the row FAILED Table keyboard navigation with non-interactive columns for a single-selection table with focus when a row is focused, pressing Space will select the row, and pressing it again will deselect the row FAILED MenuButton menu interaction with menu slotted directly in menu-button should close the menu when the element loses focus FAILED MenuButton menu interaction with menu passed through slot of additional element should close the menu when the element loses focus FAILED TableColumnMenuButton menu button interactions shared menu menu button column can share menu with action menu FAILED TableColumnMenuButton menu button interactions shared menu menu opens and closes when column1 menu button is clicked and then column2 menu button is clicked FAILED TableColumnMenuButton menu button interactions scrolling the table closes the menu button FAILED TableColumnMenuButton menu button interactions calling setData() closes the menu button FAILED TableColumnMenuButton keyboard navigation with cell[0, 0] focused, menu button in cells are reachable via Tab

Looks like at least some of these tests have always failed when running headed if the browser doesn't have focus. In that situation, calling element.focus() doesn't seem to have an effect. So the change with Chromium 128 seems to be that headless runs now behave like headed runs did.

One other note I forgot to write down but probably should since I'm backburnering this for a while:

I need to re-check that the test failures on the CI build match the list above. When I first investigated this I thought that was the case but might have been mistaken. In at least one of the CI builds some of the failures were more like this:

[test:sequential] Nimble radio control value accessor [test:sequential] when using a template-based form [test:sequential] ✗ sets correct initial selected value (5004ms) [test:sequential] Error: Timeout - Async function did not complete within 5000ms (set by jasmine.DEFAULT_TIMEOUT_INTERVAL) [test:sequential] at <Jasmine>

Thanks for testing, these failures are annoying!

I updated the PR to re-disable the wafer map tests related to #2169 that it had been enabling and updated the description to no longer resolve that issue. From a quick scan it looks like that covers most of the test failures you saw. @m-akinc @rajsite wanna try again and see if that covers all of them and also fixes the mysterious floating window?

Did a sync, full clean, build, and then tried npm run validate three times. During all the runs at some point test-webkit just starts timing out and stops making any progress:

[test-concurrent:nimble-components] [test-webkit ] × should not navigate to hidden items (5018ms) [test-concurrent:nimble-components] [test-webkit ] Error: Timeout - Async function did not complete within 5000ms (set by jasmine.DEFAULT_TIMEOUT_INTERVAL) in C:/dev/github/nimble/node_modules/karma-jasmine/node_modules/jasmine-core/lib/jasmine-core/jasmine.js (line 7612) [test-concurrent:nimble-components] [test-webkit ] <Jasmine>

I have to kill the WebKitWebProcess.exe, it restarts, and then tests start making progress again. But the test execution becomes extremely slow. Like 10 seconds per test even though the timing in jasmine says 0 ms per test.

The tests start going very slow timing out one after another every 5 seconds. Also mysterious window still shows up.

I'm still seeing a lot of weird stuff from the WebKit tests:

Blank window still shows up when running tests concurrently (but not when running just the WebKit tests)

After a while, the blank window disappears (while WebKit tests are still running), then tests start failing due to 5s timeout. The same thing seems to happen when running the WebKit tests by themselves, too, except the blank window was never visible, i.e. after a certain time (about 4 min), every test just starts timing out.

Tests are slooooowwww. I had improved this via a submission to Jasmine, but somehow it's slow again?

Maybe running WebKit tests on Windows is more trouble than it is worth.

I don't think we have a clear next step planned so I'll move back to draft for now

Think we should split out the arrow upgrade. Will do that after #2474 is merged. I'll close this PR for now.

This reverts commit 7db5187.

This reverts commit a998625.

# Pull Request ## 🤨 Rationale We have several tests that verify Angular router link behavior by clicking links and inspecting router state. These tests exhibit a couple problems: 1. currently they report warnings in the output log like: `[web-server]: 404: /_karma_webpack_/page1?param1=true` 2. In #2387 when we bring in a newer version of Chromium, these cause other tests to fail to execute and report timeouts The root cause of these issues is that the tests are actually trying to navigate the page when the link is clicked. ## 👩‍💻 Implementation ### "with-href" `nimbleRouterLink` tests In researching best practices for writing tests like this I learned that the `RouterTestingModule` we had been using has been deprecated and replaced with a more powerful `RouterTestingHarness`. [This blog and video](https://www.rainerhahnekamp.com/en/how-do-i-test-using-the-routertestingharness/) does a good job of explaining it, much better than [the angular docs](https://angular.dev/api/router/testing/RouterTestingHarness). The basic idea is that the harness sets up a parent component and router which host your component under test. When something tries to navigate the harness captures information about the navigation but doesn't actually navigate the page. The fixes in this PR are to use `RouterTestingHarness` instead of `RouterTestingModule`. This has these side effects: 1. The routes are configured with `provideRouter` instead of `withRoutes` 2. The navigated route started to be relative to the current route, so starting from `/start` and clicking a link to `page` resulted in a URL of `/start/page`. The simplest fix I found for this was to change the starting page to `/`. 3. Some setup code could be deleted and made sync/async instead of fakeAsync. In addition even if the router doesn't navigate the page we are still invoking click handlers on anchors which try to navigate the page. To address that I'm calling `preventDefault()` from those handlers. ### error `routerLink` tests These tests included a `RouterTestingModule` but didn't actually need it or `RouterTestingHarness` so I deleted those imports. ### package.json Added a couple dev scripts that I found useful while running tests locally. We didn't have an obvious way to debug angular tests or to run just the tests for one project and now we do. ## 🧪 Testing 1. Verified the 404 warnings are no longer printed to the console 5. Verified the tests don't navigate the page in the newer version of Chromium 6. Verified tests still fail with various changes like changing the URL or not clicking the link ## ✅ Checklist  - [ ] I have updated the project documentation to reflect my changes or determined no changes are needed. --------- Co-authored-by: Milan Raj <[email protected]>

packages/blazor-workspace/package.json

This reverts commit 03886af.

rajsite · 2024-10-10T18:35:31Z

FYI @munteannatan @DStavilaNI this PR updates the Apache arrow version to 17

# Pull Request ## 🤨 Rationale As part of the manual tasks process, updating the pinned apache arrow and comlink versions. Continuation of #2387, see [discussion](#2387 (comment)). ## 👩‍💻 Implementation See above. ## 🧪 Testing Rely on CI. ## ✅ Checklist - [x] I have updated the project documentation to reflect my changes or determined no changes are needed.

jattasNI added 4 commits September 5, 2024 15:28

Update Playwright to 1.46.0

ba86576

Upgrade Apache Arrow to 17.0.0

ba62623

Re-enable wafer map tests on webkit

7db5187

Change files

8006a17

jattasNI commented Sep 6, 2024

View reviewed changes

jattasNI added 6 commits September 6, 2024 16:15

Roll back playwright update

781641b

Revert "Re-enable wafer map tests on webkit"

a998625

This reverts commit 7db5187.

Merge branch 'main' into quarterly-deps-update

b6db013

Playwright 1.47.0

0991f56

Change files

cb72512

Reapply "Re-enable wafer map tests on webkit"

6a2db40

This reverts commit a998625.

rajsite mentioned this pull request Oct 1, 2024

Update dependency ubuntu to v24 #2420

Closed

1 task

jattasNI added 2 commits October 1, 2024 11:35

Merge branch 'main' into quarterly-deps-update

2ea5b61

Regenerate lock file after merge

262f9e8

jattasNI mentioned this pull request Oct 3, 2024

Migrate Angular router link tests to use RouterTestingHarness #2430

Merged

1 task

jattasNI added 2 commits October 9, 2024 17:04

Merge branch 'main' into quarterly-deps-update

f3ba1d1

Regenerate package-lock after merge

ca80ceb

jattasNI marked this pull request as ready for review October 9, 2024 22:37

jattasNI requested review from DStavilaNI, munteannatan, atmgrifter00 and msmithNI as code owners October 9, 2024 22:37

rajsite approved these changes Oct 9, 2024

View reviewed changes

packages/blazor-workspace/package.json Outdated Show resolved Hide resolved

msmithNI approved these changes Oct 10, 2024

View reviewed changes

jattasNI added 4 commits October 10, 2024 09:28

rimraf 🧟

e082587

ubuntu agent version

03886af

Merge branch 'main' into quarterly-deps-update

561bbf6

Revert "ubuntu agent version"

a857e6d

This reverts commit 03886af.

atmgrifter00 approved these changes Oct 10, 2024

View reviewed changes

jattasNI mentioned this pull request Oct 10, 2024

Upgrade GitHub Actions agent ubuntu version from 22 to 24 #2434

Open

Skip wafermap tests that fail on webkit

d7ae0d3

rajsite marked this pull request as draft October 16, 2024 22:35

rajsite closed this Nov 18, 2024

rajsite mentioned this pull request Nov 18, 2024

Update Apache Arrow to v18 #2475

Merged

1 task

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Upgrade Playwright and Apache Arrow dependencies #2387

Upgrade Playwright and Apache Arrow dependencies #2387

jattasNI commented Sep 5, 2024 •

edited

Loading

jattasNI Sep 6, 2024

jattasNI Sep 6, 2024 •

edited

Loading

jattasNI Sep 6, 2024

rajsite Sep 11, 2024

jattasNI Sep 11, 2024

jattasNI Oct 10, 2024 •

edited

Loading

rajsite Oct 10, 2024 •

edited

Loading

m-akinc Oct 10, 2024

rajsite Oct 16, 2024

rajsite Nov 18, 2024

rajsite commented Oct 10, 2024

Upgrade Playwright and Apache Arrow dependencies #2387

Upgrade Playwright and Apache Arrow dependencies #2387

Conversation

jattasNI commented Sep 5, 2024 • edited Loading

Pull Request

🤨 Rationale

👩‍💻 Implementation

🧪 Testing

✅ Checklist

jattasNI Sep 6, 2024

Choose a reason for hiding this comment

jattasNI Sep 6, 2024 • edited Loading

Choose a reason for hiding this comment

jattasNI Sep 6, 2024

Choose a reason for hiding this comment

rajsite Sep 11, 2024

Choose a reason for hiding this comment

jattasNI Sep 11, 2024

Choose a reason for hiding this comment

jattasNI Oct 10, 2024 • edited Loading

Choose a reason for hiding this comment

rajsite Oct 10, 2024 • edited Loading

Choose a reason for hiding this comment

m-akinc Oct 10, 2024

Choose a reason for hiding this comment

rajsite Oct 16, 2024

Choose a reason for hiding this comment

rajsite Nov 18, 2024

Choose a reason for hiding this comment

rajsite commented Oct 10, 2024

jattasNI commented Sep 5, 2024 •

edited

Loading

jattasNI Sep 6, 2024 •

edited

Loading

jattasNI Oct 10, 2024 •

edited

Loading

rajsite Oct 10, 2024 •

edited

Loading