E2E: Enable E2E v2 in CI #22840
ci.yml
on: pull_request
Test Matrix (Fast)
0s
Build
2m 17s
Test Matrix (Slow)
0s
Matrix: test-integration
Matrix: test-unit
Matrix: test-e2e
Matrix: Test - Legacy E2E
Annotations
19 errors, 17 warnings, and 6 notices
Test - Unit (ubuntu, Node v20)
The v0 series of google-github-actions/auth is no longer maintained. It will not receive updates, improvements, or security patches. Please upgrade to the latest supported versions:
https://github.com/google-github-actions/auth
|
Test - E2E (ubuntu - 4/4)
Process completed with exit code 1.
|
Test - Unit (ubuntu, Node v18)
The v0 series of google-github-actions/auth is no longer maintained. It will not receive updates, improvements, or security patches. Please upgrade to the latest supported versions:
https://github.com/google-github-actions/auth
|
Test - Unit (ubuntu, Node v18)
Process completed with exit code 1.
|
[e2e] › e2e/telemetry/at-mention.test.ts:10:17 › cody.at-mention › mention events fire correctly for @enterprise in a @public-repo:
vscode/e2e/telemetry/at-mention.test.ts#L1
1) [e2e] › e2e/telemetry/at-mention.test.ts:10:17 › cody.at-mention › mention events fire correctly for @enterprise in a @public-repo
|
[e2e] › e2e/telemetry/at-mention.test.ts:10:17 › cody.at-mention › mention events fire correctly for @enterprise in a @public-repo:
vscode/e2e/telemetry/at-mention.test.ts#L52
1) [e2e] › e2e/telemetry/at-mention.test.ts:10:17 › cody.at-mention › mention events fire correctly for @enterprise in a @public-repo
Error: expect(locator).toBeVisible()
Locator: frameLocator('.webview[name="28146e5c-2f50-47a8-9727-5e6264df9307"]').frameLocator('#active-frame').getByRole('textbox', { name: 'Chat message' })
Expected: visible
Received: <element(s) not found>
Call log:
- expect.toBeVisible with timeout 20000ms
- waiting for frameLocator('.webview[name="28146e5c-2f50-47a8-9727-5e6264df9307"]').frameLocator('#active-frame').getByRole('textbox', { name: 'Chat message' })
50 | //TODO: make a nice UIX class for this
51 | const chatInput = chat.content.getByRole('textbox', { name: 'Chat message' })
> 52 | await expect(chatInput).toBeVisible()
| ^
53 |
54 | const initTelemetry = telemetry.snap()
55 | // We don't want to have any at mention events triggered by default.
at /home/runner/work/cody/cody/vscode/e2e/telemetry/at-mention.test.ts:52:41
|
Test - E2E (Report)
Tests failed
|
Test - Unit (windows, Node v20)
The v0 series of google-github-actions/auth is no longer maintained. It will not receive updates, improvements, or security patches. Please upgrade to the latest supported versions:
https://github.com/google-github-actions/auth
|
chat-keyboard-shortcuts.test.ts:35:5 › re-opening chat adds selection:
vscode/test/e2e/chat-keyboard-shortcuts.test.ts#L1
1) chat-keyboard-shortcuts.test.ts:35:5 › re-opening chat adds selection ─────────────────────────
Test timeout of 30000ms exceeded.
|
chat-keyboard-shortcuts.test.ts:35:5 › re-opening chat adds selection:
vscode/test/e2e/common.ts#L179
1) chat-keyboard-shortcuts.test.ts:35:5 › re-opening chat adds selection ─────────────────────────
Error: locator.fill: Target page, context or browser has been closed
Call log:
- waiting for getByPlaceholder('Type the name of a command to run.')
- locator resolved to <input wrap="off" type="text" class="input" role="combobox" autocorrect="off" spellcheck="false" aria-label="input" custom-hover="true" autocapitalize="off" aria-haspopup="menu" aria-expanded="true" aria-autocomplete="list" aria-controls="quickInput_list" aria-activedescendant="list_id_4_0" aria-describedby="quickInput_message" placeholder="Type the name of a command to run."/>
- fill("buzz.ts")
- attempting fill action
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #1
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #2
- waiting 20ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #3
- waiting 100ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #4
- waiting 100ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #5
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #6
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #7
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #8
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #9
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #10
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #11
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #12
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #13
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #14
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #15
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #16
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #17
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #18
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #19
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #20
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
|
support-menu-items.test.ts:9:1 › shows support link for free users:
vscode/test/e2e/support-menu-items.test.ts#L1
1) support-menu-items.test.ts:9:1 › shows support link for free users ────────────────────────────
Test timeout of 30000ms exceeded.
|
support-menu-items.test.ts:9:1 › shows support link for free users:
vscode/test/e2e/support-menu-items.test.ts#L15
1) support-menu-items.test.ts:9:1 › shows support link for free users ────────────────────────────
Error: locator.click: Target page, context or browser has been closed
Call log:
- waiting for getByRole('button', { name: 'cody-logo-heavy, Cody Settings' })
13 |
14 | const statusBarButton = page.getByRole('button', { name: 'cody-logo-heavy, Cody Settings' })
> 15 | await statusBarButton.click()
| ^
16 |
17 | const input = page.getByPlaceholder('Choose an option')
18 | await input.fill('support')
at /home/runner/work/cody/cody/vscode/test/e2e/support-menu-items.test.ts:15:27
|
support-menu-items.test.ts:9:1 › shows support link for free users:
vscode/test/e2e/support-menu-items.test.ts#L1
1) support-menu-items.test.ts:9:1 › shows support link for free users ────────────────────────────
Retry #1 ───────────────────────────────────────────────────────────────────────────────────────
Test timeout of 30000ms exceeded.
|
support-menu-items.test.ts:9:1 › shows support link for free users:
vscode/test/e2e/support-menu-items.test.ts#L15
1) support-menu-items.test.ts:9:1 › shows support link for free users ────────────────────────────
Retry #1 ───────────────────────────────────────────────────────────────────────────────────────
Error: locator.click: Target page, context or browser has been closed
Call log:
- waiting for getByRole('button', { name: 'cody-logo-heavy, Cody Settings' })
13 |
14 | const statusBarButton = page.getByRole('button', { name: 'cody-logo-heavy, Cody Settings' })
> 15 | await statusBarButton.click()
| ^
16 |
17 | const input = page.getByPlaceholder('Choose an option')
18 | await input.fill('support')
at /home/runner/work/cody/cody/vscode/test/e2e/support-menu-items.test.ts:15:27
|
support-menu-items.test.ts:24:1 › shows support link for pro users:
vscode/test/e2e/support-menu-items.test.ts#L1
2) support-menu-items.test.ts:24:1 › shows support link for pro users ────────────────────────────
Test timeout of 30000ms exceeded.
|
support-menu-items.test.ts:24:1 › shows support link for pro users:
vscode/test/e2e/support-menu-items.test.ts#L32
2) support-menu-items.test.ts:24:1 › shows support link for pro users ────────────────────────────
Error: locator.click: Target page, context or browser has been closed
Call log:
- waiting for getByRole('button', { name: 'cody-logo-heavy, Cody Settings' })
30 |
31 | const statusBarButton = page.getByRole('button', { name: 'cody-logo-heavy, Cody Settings' })
> 32 | await statusBarButton.click()
| ^
33 |
34 | const input = page.getByPlaceholder('Choose an option')
35 | await input.fill('support')
at /home/runner/work/cody/cody/vscode/test/e2e/support-menu-items.test.ts:32:27
|
support-menu-items.test.ts:24:1 › shows support link for pro users:
vscode/test/e2e/support-menu-items.test.ts#L1
2) support-menu-items.test.ts:24:1 › shows support link for pro users ────────────────────────────
Retry #1 ───────────────────────────────────────────────────────────────────────────────────────
Test timeout of 30000ms exceeded.
|
support-menu-items.test.ts:24:1 › shows support link for pro users:
vscode/test/e2e/support-menu-items.test.ts#L32
2) support-menu-items.test.ts:24:1 › shows support link for pro users ────────────────────────────
Retry #1 ───────────────────────────────────────────────────────────────────────────────────────
Error: locator.click: Target page, context or browser has been closed
Call log:
- waiting for getByRole('button', { name: 'cody-logo-heavy, Cody Settings' })
30 |
31 | const statusBarButton = page.getByRole('button', { name: 'cody-logo-heavy, Cody Settings' })
> 32 | await statusBarButton.click()
| ^
33 |
34 | const input = page.getByPlaceholder('Choose an option')
35 | await input.fill('support')
at /home/runner/work/cody/cody/vscode/test/e2e/support-menu-items.test.ts:32:27
|
Test - Legacy E2E (ubuntu, 5/5)
Process completed with exit code 1.
|
Test - Unit (ubuntu, Node v20)
The following actions use a deprecated Node.js version and will be forced to run on node20: google-github-actions/auth@v0. For more info: https://github.blog/changelog/2024-03-07-github-actions-all-actions-will-run-on-node20-instead-of-node16-by-default/
|
Test - Unit (ubuntu, Node v18)
The following actions use a deprecated Node.js version and will be forced to run on node20: google-github-actions/auth@v0. For more info: https://github.blog/changelog/2024-03-07-github-actions-all-actions-will-run-on-node20-instead-of-node16-by-default/
|
Slow Test:
vscode/[e2e] › e2e/features/fixup-decorator.test.ts#L1
vscode/[e2e] › e2e/features/fixup-decorator.test.ts took 16.8s
|
Slow Test:
vscode/[e2e] › e2e/features/auth.test.ts#L1
vscode/[e2e] › e2e/features/auth.test.ts took 16.4s
|
Slow Test:
vscode/[e2e] › e2e/features/enterprise/cody-ignore.test.ts#L1
vscode/[e2e] › e2e/features/enterprise/cody-ignore.test.ts took 16.0s
|
Test - Unit (windows, Node v20)
The following actions use a deprecated Node.js version and will be forced to run on node20: google-github-actions/auth@v0. For more info: https://github.blog/changelog/2024-03-07-github-actions-all-actions-will-run-on-node20-instead-of-node16-by-default/
|
Slow Test:
vscode/chat-atFile.test.ts#L1
vscode/chat-atFile.test.ts took 1.1m
|
Slow Test:
vscode/attribution.test.ts#L1
vscode/attribution.test.ts took 18.5s
|
Slow Test:
vscode/chat-input.test.ts#L1
vscode/chat-input.test.ts took 47.0s
|
Slow Test:
vscode/chat-rateLimit.test.ts#L1
vscode/chat-rateLimit.test.ts took 23.6s
|
Slow Test:
vscode/initial-context.test.ts#L1
vscode/initial-context.test.ts took 40.9s
|
Slow Test:
vscode/command-edit.test.ts#L1
vscode/command-edit.test.ts took 39.4s
|
Slow Test:
vscode/enterprise-cody-ignore.test.ts#L1
vscode/enterprise-cody-ignore.test.ts took 18.6s
|
Slow Test:
vscode/command-custom.test.ts#L1
vscode/command-custom.test.ts took 57.0s
|
Slow Test:
vscode/command-core.test.ts#L1
vscode/command-core.test.ts took 34.1s
|
Slow Test:
vscode/code-actions.test.ts#L1
vscode/code-actions.test.ts took 30.9s
|
Slow Test:
vscode/uninstall.test.ts#L1
vscode/uninstall.test.ts took 24.4s
|
🎭 Playwright Run Summary
1 failed
[e2e] › e2e/telemetry/at-mention.test.ts:10:17 › cody.at-mention › mention events fire correctly for @enterprise in a @public-repo
17 passed (47.8s)
|
🎭 Playwright Run Summary
10 passed (2.0m)
|
🎭 Playwright Run Summary
1 flaky
chat-keyboard-shortcuts.test.ts:35:5 › re-opening chat adds selection ──────────────────────────
9 passed (2.4m)
|
🎭 Playwright Run Summary
1 skipped
10 passed (2.1m)
|
🎭 Playwright Run Summary
10 passed (2.3m)
|
🎭 Playwright Run Summary
2 failed
support-menu-items.test.ts:9:1 › shows support link for free users ─────────────────────────────
support-menu-items.test.ts:24:1 › shows support link for pro users ─────────────────────────────
2 skipped
2 passed (2.7m)
|
Artifacts
Produced during runtime
Name | Size | |
---|---|---|
e2e-blob-report-ubuntu-1
Expired
|
14.2 MB |
|
e2e-blob-report-ubuntu-2
Expired
|
14 MB |
|
e2e-blob-report-ubuntu-3
Expired
|
10.4 MB |
|
e2e-blob-report-ubuntu-4
Expired
|
12.1 MB |
|
e2e-html-report
|
51.2 MB |
|
playwright-recordings ubuntu
|
2.79 MB |
|
vscode_dist
Expired
|
41.7 MB |
|