Tests fail while running a lot of them together

[chaitanya-uc]

When running a folder with a lot of test cases the runner fails with the following error:

/playwright/node_modules/@reportportal/agent-js-playwright/build/reporter.js:452
                        rootSuiteName = parentSuiteObj.rootSuite;
                                                       ^

TypeError: Cannot read properties of undefined (reading 'rootSuite')
    at RPReporter.<anonymous> (/playwright/node_modules/@reportportal/agent-js-playwright/src/reporter.ts:475:42)
    at step (/playwright/node_modules/@reportportal/agent-js-playwright/build/reporter.js:60:23)
    at Object.next (/playwright/node_modules/@reportportal/agent-js-playwright/build/reporter.js:41:53)
    at /playwright/node_modules/@reportportal/agent-js-playwright/build/reporter.js:35:71
    at new Promise (<anonymous>)
    at __awaiter (/playwright/node_modules/@reportportal/agent-js-playwright/build/reporter.js:31:12)
    at RPReporter.onTestEnd (/playwright/node_modules/@reportportal/agent-js-playwright/build/reporter.js:413:16)
    at /playwright/node_modules/@playwright/test/lib/reporters/multiplexer.js:69:131
    at wrap (/playwright/node_modules/@playwright/test/lib/reporters/multiplexer.js:111:5)
    at Multiplexer.onTestEnd (/playwright/node_modules/@playwright/test/lib/reporters/multiplexer.js:66:45)
    at Dispatcher._reportTestEnd (/playwright/node_modules/@playwright/test/lib/dispatcher.js:556:186)
    at Worker.onTestEnd (/playwright/node_modules/@playwright/test/lib/dispatcher.js:234:12)
    at Worker.emit (node:events:526:28)
    at ChildProcess.<anonymous> (/playwright/node_modules/@playwright/test/lib/dispatcher.js:610:12)
    at ChildProcess.emit (node:events:526:28)

When running the tests individually, no such error occurs.

My playwright.config.ts:

const { devices } = require('@playwright/test');
import { PlaywrightTestConfig } from '@playwright/test';

// /** @type {import('@playwright/test').PlaywrightTestConfig} */
const RPconfig = {
    'token': 'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx',
    'endpoint': 'https://<report-portal>/api/v1',
    'project': 'projname',
    'launch': 'Playwright Runner Test',
    'description': 'Tests with Playwright Runner',
  };


const config: PlaywrightTestConfig = {
    reporter: [['@reportportal/agent-js-playwright', RPconfig]],
    forbidOnly: !!process.env.CI,
    retries: process.env.CI ? 2 : 0,
    use: {
        trace: 'on-first-retry',
    },
    workers: 2,
    projects: [
        {
        name: 'chromium',
        use: { ...devices['Desktop Chrome'] },
        }
    ],
    timeout: 30 * 10000,
};

module.exports = config;

package.json:

{
  "name": "playwright",
  "version": "1.0.0",
  "description": "Dependencies and Report Portal integration information to run Playwright Test scripts",
  "main": "index.js",
  "scripts": {
    "test": "npx playwright test --config=playwright.config.ts"
  },
  "license": "ISC",
  "dependencies": {
     "@playwright/test": "^1.20.1",
    "concurrently": "^6.3.0",
    "playwright-chromium": "^1.18.0",
    "@reportportal/agent-js-playwright": "^5.0.2"

  }
}

Any help will be appreciated!
Cheers!

[AmsterGet]

Hello @chaitanya-uc !
Have you tried latest released 5.0.3 version?

[chaitanya-uc]

Hey @AmsterGet The latest 5.0.3 version has been installed on my system.

[renkyoji]

Hello @chaitanya-uc ,
Could you share your folders structure for tests and the structure of the test example inside the file?

[chaitanya-uc]

@oranje322 Hey!
My folder structure is as follows:

/playwright
---playwright.config.ts
---package.json
---node_modules/
---tests/
------test_cases_1/
---------test1.spec.js
---------test2.spec.js
------test_cases_2/
---------test1.spec.js
---------test2.spec.js

Each of the test_cases_1 folders has sub-folders with test cases for different scenarios. In total I have 100 individual .spec.js files under the test_cases_n folders.

Sorry I didn't get what you meant by test example inside the file.

My tests basically opens up a chromium browser and fills some data in my CRM and validates the fields which gets updated.

[renkyoji]

I meant test.describe structure inside test file.
How many nested describe block you have in tests with errors?

When running the tests individually, so such error occurs.

Do you mean "no such error occurs?"

[chaitanya-uc]

Do you mean "no such error occurs?"

Sorry it's supposed to be no. Fixed it.

How many nested describe block you have in tests with errors?

I have only 1 describe block per file.

[renkyoji]

The problem is caused by using the same file names.
Will be fixed in the next version.

[renkyoji]

The fix was released in version 5.0.4 #49