Skip to content
This repository has been archived by the owner on Feb 22, 2023. It is now read-only.

Fix/e2e tests #1640

Merged
merged 14 commits into from
Aug 11, 2022
3 changes: 2 additions & 1 deletion bin/playwright.sh
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
#! /usr/bin/env sh
set -e

version() {
pnpm ls --depth=0 | grep -e playwright | awk '{print $2}';
Expand All @@ -11,6 +12,6 @@ export TEST_COMMAND=${TEST_COMMAND:-test:playwright:local}

echo Running Playwright v$PLAYWRIGHT_VERSION as $USER_ID with Playwright arguments $PLAYWRIGHT_ARGS

docker-compose -f docker-compose.playwright.yml up --build --force-recreate --abort-on-container-exit --exit-code-from playwright --remove-orphans
docker-compose -f docker-compose.playwright.yml up --build --force-recreate --exit-code-from playwright --remove-orphans
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So was it the docker-compose down that was causing this not to fail because the script continued to the dc down command after the failure of the test run? If so, an alternative fix (assuming the down is necessary) would be to add set -x to the top of the script so that any command failure causes the script to exit with the failure exit code.

Copy link
Contributor Author

@obulat obulat Aug 11, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, if you remove the dc down line, the tests start to fail again. But I'm not very clear as to what exactly happens. I'll try your suggestion.

Copy link
Contributor

@sarayourfriend sarayourfriend Aug 11, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If that is the case then I think it is what I said. The default behavior with shell scripts is not to bail out the entire script if a command fails unless the e option is set.

I had remembered it wrong, btw, it should be set -e. set -x prints the commands as they're run. e bails on the script if any error occurs. https://linuxcommand.org/lc3_man_pages/seth.html

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It worked! Thank you :)
How did you see that the Docker container was still running after tests in GitHub actions, by the way?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How did you see that the Docker container was still running after tests in GitHub actions, by the way?

What do you mean?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was going to suggest using docker-compose run --rm ... instead but glad to see it's working now! :)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't remember why but dc run did not work for me when I tried it when originally developing the dockerized Playwright tests. There was something strange about it where it would never end the containers, IIRC.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you mean?

The dc down line was added to solve the problem where a Playwright container wouldn't stop, right? I have never known that it didn't stop in the CI. How did you understand that that was, indeed, happening?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think @krysal added it (IIRC) upon noticing that the containers weren't stopping locally. I don't think it matters if the containers continue to run in CI, as far as I know.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think @krysal added it (IIRC) upon noticing that the containers weren't stopping locally.

That is correct. The machines running the GH actions are a big black box, so that would have been much more difficult to identify, though locally it was a coincidence 🙂


docker-compose -f docker-compose.playwright.yml down
2 changes: 2 additions & 0 deletions test/playwright/e2e/all-results-keyboard.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,8 @@ const locateFocusedResult = async (page: Page) => {
return page.locator(`[href="${url.pathname}"]`)
}

test.describe.configure({ mode: 'parallel' })

test.describe('all results grid keyboard accessibility test', () => {
test.beforeEach(async ({ page }) => {
await page.goto('/search?q=birds')
Expand Down
2 changes: 2 additions & 0 deletions test/playwright/e2e/attribution.spec.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
import { test, expect } from '@playwright/test'

test.describe.configure({ mode: 'parallel' })

test.beforeEach(async ({ context }) => {
await context.grantPermissions(['clipboard-read', 'clipboard-write'])
})
Expand Down
2 changes: 2 additions & 0 deletions test/playwright/e2e/homepage.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@ import {

import { supportedSearchTypes } from '~/constants/media'

test.describe.configure({ mode: 'parallel' })

test.beforeEach(async ({ context }) => {
await mockProviderApis(context)
})
Expand Down
2 changes: 2 additions & 0 deletions test/playwright/e2e/image-detail.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,8 @@ const showsErrorPage = async (page: Page) => {
)
}

test.describe.configure({ mode: 'parallel' })

test.beforeEach(async ({ context }) => {
await mockProviderApis(context)
})
Expand Down
2 changes: 2 additions & 0 deletions test/playwright/e2e/migration-banner.spec.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
import { test, expect } from '@playwright/test'

test.describe.configure({ mode: 'parallel' })

test.describe('migration banner', () => {
test('shows migration banner on homepage', async ({ page }) => {
await page.goto('/?referrer=creativecommons.org')
Expand Down
2 changes: 2 additions & 0 deletions test/playwright/e2e/mobile-menu.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,8 @@ const mobileFixture = {
}
test.use(mobileFixture)

test.describe.configure({ mode: 'parallel' })

test('Can open filters menu on mobile at least twice', async ({ page }) => {
await page.goto('/search/?q=cat')

Expand Down
2 changes: 2 additions & 0 deletions test/playwright/e2e/search-navigation.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@ import {
} from '~~/test/playwright/utils/navigation'
import { mockProviderApis } from '~~/test/playwright/utils/route'

test.describe.configure({ mode: 'parallel' })

test.beforeEach(async ({ context }) => {
await mockProviderApis(context)
})
Expand Down
2 changes: 2 additions & 0 deletions test/playwright/e2e/search-query-client.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,8 @@ import { AUDIO, IMAGE } from '~/constants/media'
* All of these tests test search page on the client
*/

test.describe.configure({ mode: 'parallel' })

test.beforeEach(async ({ context }) => {
await mockProviderApis(context)
})
Expand Down
2 changes: 2 additions & 0 deletions test/playwright/e2e/search-query-server.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,8 @@ import { AUDIO, IMAGE } from '~/constants/media'
* All of these tests test server-generated search page, not the one generated on the client
*/

test.describe.configure({ mode: 'parallel' })

test.beforeEach(async ({ context }) => {
await mockProviderApis(context)
})
Expand Down
2 changes: 2 additions & 0 deletions test/playwright/e2e/search-types.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,8 @@ import { mockProviderApis } from '~~/test/playwright/utils/route'
* Results include search meta information, media grid and Meta search form, can load more media if there are more media items.
*/

test.describe.configure({ mode: 'parallel' })

test.beforeEach(async ({ context }) => {
await mockProviderApis(context)
})
Expand Down
2 changes: 2 additions & 0 deletions test/playwright/e2e/search.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,8 @@ import {
searchFromHeader,
} from '~~/test/playwright/utils/navigation'

test.describe.configure({ mode: 'parallel' })

test.beforeEach(async ({ context }) => {
await mockProviderApis(context)
})
Expand Down
2 changes: 2 additions & 0 deletions test/playwright/e2e/translation-banner.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@ import { test, expect } from '@playwright/test'

const russianSearchPath = '/ru/search?q=dog'

test.describe.configure({ mode: 'parallel' })

test.describe('translation banner', () => {
test('Can see the translation banner and go to the correct link', async ({
page,
Expand Down
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
80 changes: 80 additions & 0 deletions test/tapes/response-494.json5
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
{
meta: {
createdAt: '2022-08-11T09:32:39.277Z',
host: 'https://api.openverse.engineering',
resHumanReadable: true,
},
req: {
headers: {
connection: 'close',
},
url: '/v1/auth_tokens/token/',
method: 'POST',
body: 'Y2xpZW50X2lkPVNCbzM1dU0wNDlvQ05ZdGtOVXdXYTZ3clFpdFF1dHNMMkZhWGo2VlkmY2xpZW50X3NlY3JldD0zVG80VzJGRXhPQnc5ZUdtbDc0Uzdib2M2c2ZUM0hFZ2xBY3VqUWdMbDBuQ0NZdUhYMThwcDRoQnEwc2ZNM2FzNkpIeDlkZG5odG5lenlYaGRYb0ZDN0JvQlZnakRheWM1OU8yUm9kclI0UkN5NnlBekRtcG91YlNOaktRaVhhbyZncmFudF90eXBlPWNsaWVudF9jcmVkZW50aWFscw==',
},
res: {
status: 401,
headers: {
date: [
'Thu, 11 Aug 2022 09:32:40 GMT',
],
'content-type': [
'application/json',
],
'content-length': [
'27',
],
connection: [
'close',
],
'cache-control': [
'no-store',
],
pragma: [
'no-cache',
],
'www-authenticate': [
'Bearer error="invalid_client"',
],
vary: [
'Authorization, Origin',
],
'x-frame-options': [
'DENY',
],
'x-content-type-options': [
'nosniff',
],
'referrer-policy': [
'same-origin',
],
'cross-origin-opener-policy': [
'same-origin',
],
'x-request-id': [
'e8610770b47d46b99e41b38b54600063',
],
'cf-cache-status': [
'DYNAMIC',
],
'expect-ct': [
'max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"',
],
'strict-transport-security': [
'max-age=15552000; includeSubDomains; preload',
],
server: [
'cloudflare',
],
'cf-ray': [
'738ff71b1876c223-VIE',
],
'alt-svc': [
'h3=":443"; ma=86400, h3-29=":443"; ma=86400',
],
},
body: {
error: 'invalid_client',
},
},
}