e2e tests from a monorepo with multiple apps deployed - how to get the right deployment URL?

[jin-current]

Question

Hi all, more of a question than a bug - we currently have a monorepo using Turborepo with multiple NextJs.App.
We are using this https://github.com/patrickedqvist/wait-for-vercel-preview to get the Vercel deployment URL which we would use to write an e2e test using Playwright.

However, not that we have the second app also deployed from the same repo, we noticed we may get the "wrong" URL, and therefore causing tests to fail.

Any ideas/best practice to workaround this?

[Jayraj-R]

Hey, did you find your answer? We are also trying to setup playwright testing for our turborepo next apps, can you help me with that ?

[jin-current]

No, unfortunately, I have not. But if you have only 1 deployment within the turbo repo, you can do some thing like this:

runs-on: ubuntu-latest
    timeout-minutes: 5
    outputs:
      preview_url: ${{ steps.waitForVercelPreviewDeployment.outputs.url }}
    steps:
      - name: Wait for Vercel preview deployment to be ready
        uses: patrickedqvist/[email protected]
        id: waitForVercelPreviewDeployment
        with:
          token: ${{ secrets.GITHUB_TOKEN }}
          max_timeout: 600
  test_e2e:
    if: github.event.deployment_status.state == 'success' # Only run when the deployment was successful.
    needs: test_setup
    name: Playwright tests
    timeout-minutes: 5
    runs-on: ubuntu-latest
    steps:
      - name: Prepare testing env
        uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 18
      - run: yarn install --frozen-lockfile
      - run: npx playwright install --with-deps
      - name: Run tests
        run: yarn test:e2e
        env:
          PLAYWRIGHT_TEST_BASE_URL: ${{ needs.test_setup.outputs.preview_url }}

[wiledal]

The way I solved this was to look at the url and determine which app it is. Since the name of the app will be part of the preview url. Using the GitHub Actions contains expression in the YML.

name: Dashboard Playwright Tests
on:
  deployment_status:

jobs:
  e2e-tests:
    if: github.event_name == 'deployment_status' && github.event.deployment_status.state == 'success' && contains(github.event.deployment_status.environment_url, 'dashboard')

    timeout-minutes: 60
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v4
    - uses: pnpm/action-setup@v2 
    - uses: actions/setup-node@v4
      with:
        node-version: 18
        cache: 'pnpm'

    - name: Install dependencies
      run: pnpm install

    - name: Install Playwright Browsers
      working-directory: apps/dashboard
      run: pnpm playwright install --with-deps chromium

    - name: Run Playwright tests
      run: pnpm e2e:test --workers 2
      working-directory: apps/dashboard
      env:
        PLAYWRIGHT_TEST_BASE_URL: ${{ github.event.deployment_status.environment_url }}
        PLAYWRIGHT_TEST_USER_PASSWORD: ${{ secrets.PLAYWRIGHT_TEST_USER_PASSWORD }}

    - uses: actions/upload-artifact@v3
      if: always()
      with:
        name: dashboard-playwright-report
        path: apps/dashboard/playwright-report/
        retention-days: 30

A problem I'm currently having is that if an app gets deployed after the one we want to test, it will signal the deployed event again, which will cause the last updated status to show "skipped", even though the tests are in fact running.

[lucasfelixc]

Hi guys, did you find any way to resolve this? I tried the same configuration as @wiledal but I had the same problem, workflows are being ignored.

[wiledal]

Are you sure the workflow is being ignored, or are you just seeing the state from an ignored application?

The Check Run always ends up as "Skipped" in some scenarios, depending on which application triggers the workflow last. If you are lucky that your tested application is deployed last, it will show up in the Check Runs together with the rest. I tried getting around this by creating my own Check Run, but did not find a good solution.

What I ended up doing is make a live comment that updates with the status instead. It's not ideal, since the Playwright Check Run will essentially end up useless and ignored, but at least I have the latest update on my tests always visible and accessible.

---

Edit: "Check Run", not "Run Check"