Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: allow multi-browser configuration #6975

Merged
merged 53 commits into from
Dec 19, 2024

Conversation

sheremet-va
Copy link
Member

@sheremet-va sheremet-va commented Nov 27, 2024

Description

Closes #5963
Closes #7017

The general idea is to inject new projects if instances are defined that inherit the same config and vite server

image

With this PR, browser.name and browser.providerOptions are deprecated. Use instances instead:

export default defineConfig({
  test: {
    browser: {
      provider: 'playwright',
      configs: [
        {
          browser: 'chromium',
          launch: { devtools: true },
        },
        {
          browser: 'firefox',
          setupFiles: ['./setup.firefox.ts'],
          provide: {
            secret: 'my-secret',
          },
        },
      ],
    }
  }
})

The main performance optimization here instead of using different projects is that we can reuse the same server and keep the cache.

Please don't delete this checklist! Before submitting the PR, please make sure you do the following:

  • It's really useful if your PR references an issue where it is discussed ahead of time. If the feature is substantial or introduces breaking changes without a discussion, PR might be closed.
  • Ideally, include a test that fails without this PR but passes with it.
  • Please, don't make changes to pnpm-lock.yaml unless you introduce a new test example.

Tests

  • Run the tests with pnpm test:ci.

Documentation

  • If you introduce new functionality, document it. You can run documentation with pnpm run docs command.

Changesets

  • Changes in changelog are generated from PR name. Please, make sure that it explains your changes in an understandable manner. Please, prefix changeset messages with feat:, fix:, perf:, docs:, or chore:.

Copy link

netlify bot commented Nov 27, 2024

Deploy Preview for vitest-dev ready!

Name Link
🔨 Latest commit 603e323
🔍 Latest deploy log https://app.netlify.com/sites/vitest-dev/deploys/676196bfb19888000840b793
😎 Deploy Preview https://deploy-preview-6975--vitest-dev.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

@sheremet-va sheremet-va added this to the 2.2.0 milestone Dec 1, 2024
@sheremet-va sheremet-va force-pushed the feat/multi-browser-config branch 2 times, most recently from 8e54cff to 227094e Compare December 4, 2024 17:04
@sheremet-va sheremet-va modified the milestones: 2.2.0, 3.0.0 Dec 5, 2024
@sheremet-va sheremet-va force-pushed the feat/multi-browser-config branch from 30ba4b6 to 3cc6dc5 Compare December 9, 2024 13:42
@JessicaSachs
Copy link
Contributor

JessicaSachs commented Dec 10, 2024

@sheremet-va asked for naming help. After going back and forth in Discord, I farmed the work out to some of our future Robot overlords. I think the overlords did an okay job.

instances is its choice. I don't quite know what mine is. I definitely dislike test.browser.browsers though.


🤖 Browser Configuration Naming Analysis

Quick Reference: All Options

// Current
test: {
  browser: {
    provider: 'playwright',
    configs: [{ browser: 'chromium' }, { browser: 'firefox' }]
  }
}

// Option 1: browsers
test: {
  browser: {
    provider: 'playwright',
    browsers: [{ browser: 'chromium' }, { browser: 'firefox' }]
  }
}

// Option 2: capabilities (conflicts with WebDriverIO)
test: {
  browser: {
    provider: 'playwright',
    capabilities: [{ browser: 'chromium' }, { browser: 'firefox' }]
  }
}

// Option 3: environments
test: {
  browser: {
    provider: 'playwright',
    environments: [{ browser: 'chromium' }, { browser: 'firefox' }]
  }
}

// Option 4: targets
test: {
  browser: {
    provider: 'playwright',
    targets: [{ browser: 'chromium' }, { browser: 'firefox' }]
  }
}

// Option 5: profiles
test: {
  browser: {
    provider: 'playwright',
    profiles: [{ browser: 'chromium' }, { browser: 'firefox' }]
  }
}

// Option 6: instances (recommended)
test: {
  browser: {
    provider: 'playwright',
    instances: [{ browser: 'chromium' }, { browser: 'firefox' }]
  }
}

Context

Vitest needs to name the configuration array for multiple browser setups in test.browser. The configuration allows running tests across different browsers while sharing a single Vite server instance for improved performance.

Requirements

  1. Must be semantically accurate
  2. Should avoid naming conflicts with existing APIs
  3. Must be clear in documentation
  4. Should avoid redundant terminology (e.g., browser.capabilities.capabilities)
  5. Should align with architectural concepts

Options Analysis

Option Pros Cons
configs (current) • Generic enough for all configuration options
• Common terminology in config files
• Ambiguous with parent browser config
• Too generic for specific purpose
• Harder to reference in documentation
browsers • Direct and clear
• Matches mental model
• Intuitive for users
• Creates repetitive test.browser.browsers path
• Could be confused with browser selection vs configuration
capabilities • Common terminology in browser testing
• Well understood in the industry
• Conflicts with WebDriverIO's API
• Would create browser.capabilities.capabilities
• Too broad for specific purpose
environments • Encompasses full test environment concept
• Distinct from other terminology
• Clear in documentation
• May be too broad (environment includes more than browser)
• Could be confused with test environments (dev/staging/prod)
targets • Clear purpose (targeting specific browsers)
• Distinct from other terminology
• Natural in documentation
• Could be confused with build targets
• Slightly less accurate (configurations do more than target browsers)
profiles • Suggests complete configuration sets
• Distinct from other terminology
• Common in development tools
• May be confused with user profiles
• Less specific to browser testing context
instances (recommended) • Matches diagram terminology ("test project instance")
• Avoids repetition with browser
• Accurately describes browser instances with specific configs
• Natural in documentation
• Clear relationship to runtime behavior
• Could be confused with class instances in other contexts
• Slightly technical term

Recommendation: instances

Sample documentation text:

"Configure multiple browser instances by providing an array of configurations in test.browser.instances. Each instance can specify its own browser type, setup files, and launch options while sharing a single Vite server for improved performance.

export default defineConfig({
  test: {
    browser: {
      provider: 'playwright',
      instances: [
        {
          browser: 'chromium',
          launch: { devtools: true },
        },
        {
          browser: 'firefox',
          setupFiles: ['./setup.firefox.ts'],
          provide: {
            secret: 'my-secret',
          },
        },
      ],
    }
  }
})

Each instance inherits from the global browser configuration while allowing browser-specific overrides."

This terminology:

  1. Avoids repetition in the config path
  2. Matches existing architectural concepts
  3. Accurately describes the technical implementation
  4. Reads naturally in documentation
  5. Maps clearly to the runtime behavior where each configuration spawns a browser instance
  6. Maintains clarity when referenced in other contexts (e.g., "browser instance configuration", "configure test instances")

@sheremet-va sheremet-va marked this pull request as ready for review December 11, 2024 14:30
Copy link
Contributor

@hi-ogawa hi-ogawa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool! I think I understood the idea and it makes sense to me 👍
I didn't check docs and there are some TODOs, but I guess those can be checked later.
Feel free to proceed if you want.

packages/browser/src/node/serverTester.ts Outdated Show resolved Hide resolved
@sheremet-va sheremet-va merged commit 78b62ff into vitest-dev:main Dec 19, 2024
13 checks passed
@sheremet-va sheremet-va deleted the feat/multi-browser-config branch December 19, 2024 13:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants