Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: allow multi-browser configuration #6975

Merged
merged 53 commits into from
Dec 19, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
53 commits
Select commit Hold shift + click to select a range
cb8f9c8
feat: allow multi-browser configuration
sheremet-va Nov 27, 2024
bcd8a37
refactor: code code code
sheremet-va Nov 28, 2024
417cf8f
chore: cleanup
sheremet-va Nov 28, 2024
ac15988
chore: lint
sheremet-va Nov 28, 2024
d414864
fix: inherit the name to avoid name conflict
sheremet-va Nov 28, 2024
2f0c084
docs: add configuring playwright/webdriverio section
sheremet-va Nov 28, 2024
7eb3001
feat: merge custom config
sheremet-va Nov 28, 2024
04048a2
docs: add `actionTimeout` to docs
sheremet-va Dec 1, 2024
b1087f8
refactor: rename capabilities to configs
sheremet-va Dec 4, 2024
c737b3d
test: cover incorrect browser configs
sheremet-va Dec 4, 2024
98c6d09
fix: export BrowserConfig, providers still only get BrowserProviderCo…
sheremet-va Dec 4, 2024
6d04b97
fix: make sure configs are correctly inherited
sheremet-va Dec 4, 2024
8102f1a
test: add a fail for a duplcate name
sheremet-va Dec 4, 2024
5da2a78
test: remove .only
sheremet-va Dec 4, 2024
06e690d
chore: fix type issue
sheremet-va Dec 4, 2024
e935ece
chore: fix typecheck
sheremet-va Dec 9, 2024
ae35867
chore: test
sheremet-va Dec 9, 2024
4d98069
chore: test
sheremet-va Dec 9, 2024
3cc6dc5
chore: remove ts directive
sheremet-va Dec 9, 2024
99ef86e
Merge branch 'main' of github.com:vitest-dev/vitest into feat/multi-b…
sheremet-va Dec 10, 2024
5006f1b
fix: expose browser sessions
sheremet-va Dec 10, 2024
4844839
docs: fix version
sheremet-va Dec 10, 2024
4fdaa45
feat: add a confirmation flag if running in headed mode
sheremet-va Dec 10, 2024
72cd39c
fix: pass project id so the preview provider can work without queries
sheremet-va Dec 10, 2024
eae4fd4
fix: projectName is not null
sheremet-va Dec 10, 2024
03bdf64
docs: add `browse.configs` docs
sheremet-va Dec 10, 2024
b7b07af
feat: allow per-cofig headless mode
sheremet-va Dec 10, 2024
9acafa5
docs: add multi project guide
sheremet-va Dec 10, 2024
fe8b84e
docs: fix broken links
sheremet-va Dec 10, 2024
163097e
test: fix project name
sheremet-va Dec 10, 2024
59b4a40
test: fix project name
sheremet-va Dec 10, 2024
53e509c
docs: update browser links
sheremet-va Dec 10, 2024
3f16f8d
refactor: rename browser.configs to browser.instances
sheremet-va Dec 11, 2024
978e369
chore: cleanup
sheremet-va Dec 11, 2024
da3c0c3
test: rewrite browser tests to use several browsers
sheremet-va Dec 11, 2024
9ba935d
fix: don't return ctx.updateSnapshot
sheremet-va Dec 11, 2024
99e3ae2
chore: add node-20 to ci title
sheremet-va Dec 12, 2024
71f91d6
fix: use local browser server and allow --project filter
sheremet-va Dec 16, 2024
b28dbeb
fix: filter with a regexp
sheremet-va Dec 16, 2024
0ca97b8
feat: add connect timeout for the websocket connection
sheremet-va Dec 17, 2024
57a0af1
fix: fail tester if session is invalid
sheremet-va Dec 17, 2024
0d62e6d
fix: check for body height before saving a screenshot
sheremet-va Dec 17, 2024
1e41e34
chore: add more debug statements to the orchestrator
sheremet-va Dec 17, 2024
603e323
fix: make sure the cloned config doesn't reference the root config
sheremet-va Dec 17, 2024
6f6023f
Merge branch 'main' of github.com:vitest-dev/vitest into feat/multi-b…
sheremet-va Dec 17, 2024
35b7f3d
chore: ctx -> vitest
sheremet-va Dec 17, 2024
00f76df
chore: cleanup
sheremet-va Dec 17, 2024
8de42d2
test: add webdriverio test
sheremet-va Dec 17, 2024
0882a81
fix: populate configs if "browser" is used, but "instances" is not
sheremet-va Dec 17, 2024
fd781df
chore: fix browser snapshot
sheremet-va Dec 17, 2024
a76eaa0
test: fix tests
sheremet-va Dec 19, 2024
4ea5ff6
test: fix bail test
sheremet-va Dec 19, 2024
ee42ccb
fix: don't enable browser if the config is set
sheremet-va Dec 19, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 2 additions & 13 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -124,7 +124,7 @@ jobs:

test-browser:
needs: changed
name: 'Browser: ${{ matrix.browser[0] }}, ${{ matrix.os }}'
name: 'Browsers: node-20, ${{ matrix.os }}'
if: needs.changed.outputs.should_skip != 'true'

runs-on: ${{ matrix.os }}
Expand All @@ -133,10 +133,6 @@ jobs:
os:
- macos-latest
- windows-latest
browser:
- [chromium, chrome]
- [firefox, firefox]
- [webkit]
fail-fast: false

timeout-minutes: 30
Expand All @@ -149,26 +145,19 @@ jobs:
node-version: 20

- uses: browser-actions/setup-chrome@v1
if: ${{ matrix.browser[0] == 'chromium' }}
- uses: browser-actions/setup-firefox@v1
if: ${{ matrix.browser[0] == 'firefox' }}

- name: Install
run: pnpm i

- name: Install Playwright Dependencies
run: pnpm exec playwright install ${{ matrix.browser[0] }} --with-deps --only-shell
run: pnpm exec playwright install --with-deps --only-shell

- name: Build
run: pnpm run build

- name: Test Browser (playwright)
run: pnpm run test:browser:playwright
env:
BROWSER: ${{ matrix.browser[0] }}

- name: Test Browser (webdriverio)
run: pnpm run test:browser:webdriverio
if: ${{ matrix.browser[1] }}
env:
BROWSER: ${{ matrix.browser[1] }}
33 changes: 33 additions & 0 deletions docs/.vitepress/config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,7 @@ export default ({ mode }: { mode: string }) => {
groupIconVitePlugin({
customIcon: {
'CLI': 'vscode-icons:file-type-shell',
'vitest.shims': 'vscode-icons:file-type-vitest',
'vitest.workspace': 'vscode-icons:file-type-vitest',
'vitest.config': 'vscode-icons:file-type-vitest',
'.spec.ts': 'vscode-icons:file-type-testts',
Expand Down Expand Up @@ -214,6 +215,27 @@ export default ({ mode }: { mode: string }) => {
},
],
},
{
text: 'Configuration',
collapsed: false,
items: [
{
text: 'Browser Config Reference',
link: '/guide/browser/config',
docFooterText: 'Browser Config Reference | Browser Mode',
},
{
text: 'Configuring Playwright',
link: '/guide/browser/playwright',
docFooterText: 'Configuring Playwright | Browser Mode',
},
{
text: 'Configuring WebdriverIO',
link: '/guide/browser/webdriverio',
docFooterText: 'Configuring WebdriverIO | Browser Mode',
},
],
},
{
text: 'API',
collapsed: false,
Expand Down Expand Up @@ -245,6 +267,17 @@ export default ({ mode }: { mode: string }) => {
},
],
},
{
text: 'Guides',
collapsed: false,
items: [
{
text: 'Multiple Setups',
link: '/guide/browser/multiple-setups',
docFooterText: 'Multiple Setups | Browser Mode',
},
],
},
{
items: [
...footer(),
Expand Down
2 changes: 1 addition & 1 deletion docs/.vitepress/scripts/cli-generator.ts
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,7 @@ const options = resolveOptions(cliOptionsConfig)
const template = options.map((option) => {
const title = option.title
const cli = option.cli
const config = skipConfig.has(title) ? '' : `[${title}](/config/#${title.toLowerCase().replace(/\./g, '-')})`
const config = skipConfig.has(title) ? '' : `[${title}](${title.includes('browser.') ? '/guide/browser/config' : '/config/'}#${title.toLowerCase().replace(/\./g, '-')})`
return `### ${title}\n\n- **CLI:** ${cli}\n${config ? `- **Config:** ${config}\n` : ''}\n${option.description}\n`
}).join('\n')

Expand Down
9 changes: 8 additions & 1 deletion docs/advanced/api/vitest.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,6 @@ Vitest 3 is one step closer to stabilising the public API. To achieve that, we d
- `changeNamePattern`
- `changeFilenamePattern`
- `rerunFailed`
- `updateSnapshot`
- `_createRootProject` (renamed to `_ensureRootProject`, but still private)
- `filterTestsBySource` (this was moved to the new internal `vitest.specifications` instance)
- `runFiles` (use [`runTestSpecifications`](#runtestspecifications) instead)
Expand Down Expand Up @@ -326,6 +325,14 @@ function runTestSpecifications(

This method emits `reporter.onWatcherRerun` and `onTestsRerun` events, then it runs tests with [`runTestSpecifications`](#runtestspecifications). If there were no errors in the main process, it will emit `reporter.onWatcherStart` event.

## updateSnapshot

```ts
function updateSnapshot(files?: string[]): Promise<TestRunResult>
```

Update snapshots in specified files. If no files are provided, it will update files with failed tests and obsolete snapshots.

## collectTests

```ts
Expand Down
212 changes: 4 additions & 208 deletions docs/config/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -1739,221 +1739,17 @@ Open Vitest UI (WIP)

Listen to port and serve API. When set to true, the default port is 51204

### browser {#browser}
### browser <Badge type="warning">experimental</Badge> {#browser}

- **Type:** `{ enabled?, name?, provider?, headless?, api? }`
- **Default:** `{ enabled: false, headless: process.env.CI, api: 63315 }`
- **CLI:** `--browser`, `--browser=<name>`, `--browser.name=chrome --browser.headless`
- **Default:** `{ enabled: false }`
- **CLI:** `--browser=<name>`, `--browser.name=chrome --browser.headless`

Run Vitest tests in a browser. We use [WebdriverIO](https://webdriver.io/) for running tests by default, but it can be configured with [browser.provider](#browser-provider) option.

::: tip NOTE
Read more about testing in a real browser in the [guide page](/guide/browser/).
:::
Configuration for running browser tests. Please, refer to the ["Browser Config Reference"](/guide/browser/config) article.

::: warning
This is an experimental feature. Breaking changes might not follow SemVer, please pin Vitest's version when using it.
:::

#### browser.enabled

- **Type:** `boolean`
- **Default:** `false`
- **CLI:** `--browser`, `--browser.enabled=false`

Run all tests inside a browser by default.

#### browser&#46;name

- **Type:** `string`
- **CLI:** `--browser=safari`

Run all tests in a specific browser. Possible options in different providers:

- `webdriverio`: `firefox`, `chrome`, `edge`, `safari`
- `playwright`: `firefox`, `webkit`, `chromium`
- custom: any string that will be passed to the provider

#### browser.headless

- **Type:** `boolean`
- **Default:** `process.env.CI`
- **CLI:** `--browser.headless`, `--browser.headless=false`

Run the browser in a `headless` mode. If you are running Vitest in CI, it will be enabled by default.

#### browser.isolate

- **Type:** `boolean`
- **Default:** `true`
- **CLI:** `--browser.isolate`, `--browser.isolate=false`

Run every test in a separate iframe.

#### browser.testerHtmlPath <Version>2.1.4</Version> {#browser-testerhtmlpath}

- **Type:** `string`
- **Default:** `@vitest/browser/tester.html`

A path to the HTML entry point. Can be relative to the root of the project. This file will be processed with [`transformIndexHtml`](https://vite.dev/guide/api-plugin#transformindexhtml) hook.

#### browser.api

- **Type:** `number | { port?, strictPort?, host? }`
- **Default:** `63315`
- **CLI:** `--browser.api=63315`, `--browser.api.port=1234, --browser.api.host=example.com`

Configure options for Vite server that serves code in the browser. Does not affect [`test.api`](#api) option. By default, Vitest assigns port `63315` to avoid conflicts with the development server, allowing you to run both in parallel.

#### browser.provider

- **Type:** `'webdriverio' | 'playwright' | 'preview' | string`
- **Default:** `'preview'`
- **CLI:** `--browser.provider=playwright`

Path to a provider that will be used when running browser tests. Vitest provides three providers which are `preview` (default), `webdriverio` and `playwright`. Custom providers should be exported using `default` export and have this shape:

```ts
export interface BrowserProvider {
name: string
getSupportedBrowsers: () => readonly string[]
initialize: (ctx: Vitest, options: { browser: string; options?: BrowserProviderOptions }) => Awaitable<void>
openPage: (url: string) => Awaitable<void>
close: () => Awaitable<void>
}
```

::: warning
This is an advanced API for library authors. If you just need to run tests in a browser, use the [browser](#browser) option.
:::

#### browser.providerOptions {#browser-provideroptions}

- **Type:** `BrowserProviderOptions`

Options that will be passed down to provider when calling `provider.initialize`.

```ts
import { defineConfig } from 'vitest/config'

export default defineConfig({
test: {
browser: {
providerOptions: {
launch: {
devtools: true,
},
},
},
},
})
```

::: tip
To have a better type safety when using built-in providers, you should reference one of these types (for provider that you are using) in your [config file](/config/):

```ts
/// <reference types="@vitest/browser/providers/playwright" />
/// <reference types="@vitest/browser/providers/webdriverio" />
```
:::

#### browser.ui {#browser-ui}

- **Type:** `boolean`
- **Default:** `!isCI`
- **CLI:** `--browser.ui=false`

Should Vitest UI be injected into the page. By default, injects UI iframe during development.

#### browser.viewport {#browser-viewport}

- **Type:** `{ width, height }`
- **Default:** `414x896`

Default iframe's viewport.

#### browser.locators {#browser-locators}

Options for built-in [browser locators](/guide/browser/locators).

##### browser.locators.testIdAttribute

- **Type:** `string`
- **Default:** `data-testid`

Attribute used to find elements with `getByTestId` locator.

#### browser.screenshotDirectory {#browser-screenshotdirectory}

- **Type:** `string`
- **Default:** `__snapshots__` in the test file directory

Path to the snapshots directory relative to the `root`.

#### browser.screenshotFailures {#browser-screenshotfailures}

- **Type:** `boolean`
- **Default:** `!browser.ui`

Should Vitest take screenshots if the test fails.

#### browser.orchestratorScripts {#browser-orchestratorscripts}

- **Type:** `BrowserScript[]`
- **Default:** `[]`

Custom scripts that should be injected into the orchestrator HTML before test iframes are initiated. This HTML document only sets up iframes and doesn't actually import your code.

The script `src` and `content` will be processed by Vite plugins. Script should be provided in the following shape:

```ts
export interface BrowserScript {
/**
* If "content" is provided and type is "module", this will be its identifier.
*
* If you are using TypeScript, you can add `.ts` extension here for example.
* @default `injected-${index}.js`
*/
id?: string
/**
* JavaScript content to be injected. This string is processed by Vite plugins if type is "module".
*
* You can use `id` to give Vite a hint about the file extension.
*/
content?: string
/**
* Path to the script. This value is resolved by Vite so it can be a node module or a file path.
*/
src?: string
/**
* If the script should be loaded asynchronously.
*/
async?: boolean
/**
* Script type.
* @default 'module'
*/
type?: string
}
```

#### browser.testerScripts {#browser-testerscripts}

- **Type:** `BrowserScript[]`
- **Default:** `[]`

Custom scripts that should be injected into the tester HTML before the tests environment is initiated. This is useful to inject polyfills required for Vitest browser implementation. It is recommended to use [`setupFiles`](#setupfiles) in almost all cases instead of this.

The script `src` and `content` will be processed by Vite plugins.

#### browser.commands {#browser-commands}

- **Type:** `Record<string, BrowserCommand>`
- **Default:** `{ readFile, writeFile, ... }`

Custom [commands](/guide/browser/commands) that can be imported during browser tests from `@vitest/browser/commands`.

### clearMocks

- **Type:** `boolean`
Expand Down
2 changes: 1 addition & 1 deletion docs/guide/browser/commands.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ CDP session works only with `playwright` provider and only when using `chromium`

## Custom Commands

You can also add your own commands via [`browser.commands`](/config/#browser-commands) config option. If you develop a library, you can provide them via a `config` hook inside a plugin:
You can also add your own commands via [`browser.commands`](/guide/browser/config#browser-commands) config option. If you develop a library, you can provide them via a `config` hook inside a plugin:

```ts
import type { Plugin } from 'vitest/config'
Expand Down
Loading
Loading