Parallel testing fixture bleed

[kflore16]

I have 3 files with a suite (fixture) in each file with 10 tests per file, the code snippet describes how they're written

fixture.disablePageCaching'TestA bleed 10 tests 1-10'//  The two other fixtures are TestB and TestC, 11-20 and 21-30 respectively
  .page`https://devexpress.github.io/testcafe/example/`.beforeEach( async t =>{
    
    await t.maximizeWindow()
  })
const Truth = true
const TestName = 'Test A'

//The Test numbering is in order with the fixture name
test.meta({
  smoke: 'yes',
  regression: 'yes',
  adhoc: 'yes',
  description: 'Test Descript for Test #1',
  attributes: [{ key: 'Type', value: 'Navigation' }],
})('Test 1 for ${TestName}', async t => {
  await t.expect(Truth).ok()
})`

Testcafe version 1.15.2
reporter-agent-js version 5.0.0
Report portal version API: 5.4.0, Index: 5.0.10

The image shows that the tests are bleeding into the next fixture, all ran in chrome:headless with a -c 3 flag
Fixture A
Fixture B
and similarly in Fixture C

``` PS C:\ npx testcafe -c 3 chrome:headless ./tests/DummyTest/*.test.js --skip-js-errors The "src" and "browsers" options from the configuration file will be ignored. Running tests in: - Chrome 94.0.4606.61 / Windows 10

	Start launch sk56uj1cku66jswc
	Start test item sk56uj1cku66jswh
	Success start item sk56uj1cku66jswh
	Start test item sk56uj1cku66jswl
	Start test item sk56uj1cku66jswv
	Start test item sk56uj1cku66jsww
	Success start item sk56uj1cku66jsww
	Success start item sk56uj1cku66jswl
	Success start item sk56uj1cku66jswv
	 √ TestA bleed 10 tests 1-10 - Test 1 for Test A
	Finish test item sk56uj1cku66jswl
	 √ TestA bleed 10 tests 1-10 - Test 2 for Test A
	Finish test item sk56uj1cku66jswv
	 √ TestA bleed 10 tests 1-10 - Test 3 for Test A
	Finish test item sk56uj1cku66jsww
	Start test item sk56uj1cku66ju61
	Start test item sk56uj1cku66ju64
	Start test item sk56uj1cku66ju67
	Success start item sk56uj1cku66ju64
	Success finish item sk56uj1cku66jswv
	Success finish item sk56uj1cku66jsww
	Success start item sk56uj1cku66ju61
	Success finish item sk56uj1cku66jswl
	Success start item sk56uj1cku66ju67
	 √ TestA bleed 10 tests 1-10 - Test 4 for Test A
	Finish test item sk56uj1cku66ju61
	 √ TestA bleed 10 tests 1-10 - Test 5 for Test A
	Finish test item sk56uj1cku66ju64
	Start test item sk56uj1cku66juyq
	 √ TestA bleed 10 tests 1-10 - Test 6 for Test A
	Finish test item sk56uj1cku66ju67
	Start test item sk56uj1cku66juyy
	Start test item sk56uj1cku66juz5
	Success finish item sk56uj1cku66ju67
	Success finish item sk56uj1cku66ju61
	Success finish item sk56uj1cku66ju64
	Success start item sk56uj1cku66juz5
	Success start item sk56uj1cku66juyy
	Success start item sk56uj1cku66juyq
	 √ TestA bleed 10 tests 1-10 - Test 7 for Test A
	Finish test item sk56uj1cku66juyq
	Start test item sk56uj1cku66jvr0
	 √ TestA bleed 10 tests 1-10 - Test 8 for Test A
	Finish test item sk56uj1cku66juyy
	 √ TestA bleed 10 tests 1-10 - Test 9 for Test A
	Finish test item sk56uj1cku66juz5
	Success finish item sk56uj1cku66juyq
	Start test item sk56uj1cku66jvrx
	Start test item sk56uj1cku66jvs3
	Success start item sk56uj1cku66jvr0
	Success finish item sk56uj1cku66juz5
	Success start item sk56uj1cku66jvrx
	Success finish item sk56uj1cku66juyy
	Success start item sk56uj1cku66jvs3
	 √ TestA bleed 10 tests 1-10 - Test 10 for Test A
	Finish test item sk56uj1cku66jvr0
	Start test item sk56uj1cku66jwix
	 √ TestB bleed 10 tests 11-20 - Test 11 for Test B
	Finish test item sk56uj1cku66jvrx
	 √ TestB bleed 10 tests 11-20 - Test 12 for Test B
	Finish test item sk56uj1cku66jvs3
	Success finish item sk56uj1cku66jvr0
	Success start item sk56uj1cku66jwix
	Start test item sk56uj1cku66jwj6
	Start test item sk56uj1cku66jwjr
	Start test item sk56uj1cku66jwk5
	Success finish item sk56uj1cku66jvrx
	Success start item sk56uj1cku66jwk5
	Success finish item sk56uj1cku66jvs3
	Success start item sk56uj1cku66jwj6
	Success start item sk56uj1cku66jwjr
	 √ TestB bleed 10 tests 11-20 - Test 13 for Test B
	Finish test item sk56uj1cku66jwj6
	Start test item sk56uj1cku66jxbb
	 √ TestB bleed 10 tests 11-20 - Test 14 for Test B
	Finish test item sk56uj1cku66jwjr
	Start test item sk56uj1cku66jxca
	Success start item sk56uj1cku66jxbb
	 √ TestB bleed 10 tests 11-20 - Test 15 for Test B
	Finish test item sk56uj1cku66jwk5
	Success finish item sk56uj1cku66jwj6
	Start test item sk56uj1cku66jxcu
	Success start item sk56uj1cku66jxca
	Success finish item sk56uj1cku66jwjr
	Success finish item sk56uj1cku66jwk5
	Success start item sk56uj1cku66jxcu
	 √ TestB bleed 10 tests 11-20 - Test 16 for Test B
	Finish test item sk56uj1cku66jxbb
	Start test item sk56uj1cku66jy3q
	 √ TestB bleed 10 tests 11-20 - Test 17 for Test B
	Finish test item sk56uj1cku66jxca
	Success finish item sk56uj1cku66jxbb
	Start test item sk56uj1cku66jy4m
	Success start item sk56uj1cku66jy3q
	 √ TestB bleed 10 tests 11-20 - Test 18 for Test B
	Finish test item sk56uj1cku66jxcu
	Start test item sk56uj1cku66jy5k
	Success finish item sk56uj1cku66jxca
	Success start item sk56uj1cku66jy4m
	Success finish item sk56uj1cku66jxcu
	Success start item sk56uj1cku66jy5k
	 √ TestB bleed 10 tests 11-20 - Test 19 for Test B
	Finish test item sk56uj1cku66jy3q
	Start test item sk56uj1cku66jyvw
	 √ TestB bleed 10 tests 11-20 - Test 20 for Test B
	Finish test item sk56uj1cku66jy4m
	Start test item sk56uj1cku66jywi
	 √ TestC bleed 10 tests 21-30 - Test 21 for Test C
	Finish test item sk56uj1cku66jy5k
	Success finish item sk56uj1cku66jy3q
	Success start item sk56uj1cku66jyvw
	Success start item sk56uj1cku66jywi
	Start test item sk56uj1cku66jywq
	Start test item sk56uj1cku66jyx4
	Success finish item sk56uj1cku66jy5k
	Success finish item sk56uj1cku66jy4m
	Success start item sk56uj1cku66jyx4
	Success start item sk56uj1cku66jywq
	 √ TestC bleed 10 tests 21-30 - Test 22 for Test C
	Finish test item sk56uj1cku66jyvw
	Start test item sk56uj1cku66jzo1
	 √ TestC bleed 10 tests 21-30 - Test 23 for Test C
	Finish test item sk56uj1cku66jywq
	Success finish item sk56uj1cku66jyvw
	Start test item sk56uj1cku66jzop
	Success start item sk56uj1cku66jzo1
	 √ TestC bleed 10 tests 21-30 - Test 24 for Test C
	Finish test item sk56uj1cku66jyx4
	Success finish item sk56uj1cku66jywq
	Success start item sk56uj1cku66jzop
	Start test item sk56uj1cku66jzpi
	Success finish item sk56uj1cku66jyx4
	Success start item sk56uj1cku66jzpi
	 √ TestC bleed 10 tests 21-30 - Test 25 for Test C
	Finish test item sk56uj1cku66jzo1
	 √ TestC bleed 10 tests 21-30 - Test 26 for Test C
	Finish test item sk56uj1cku66jzop
	Start test item sk56uj1cku66k0gh
	Start test item sk56uj1cku66k0gm
	Success finish item sk56uj1cku66jzo1
	Success start item sk56uj1cku66k0gh
	Success finish item sk56uj1cku66jzop
	 √ TestC bleed 10 tests 21-30 - Test 27 for Test C
	Finish test item sk56uj1cku66jzpi
	Success start item sk56uj1cku66k0gm
	Start test item sk56uj1cku66k0ho
	Success finish item sk56uj1cku66jzpi
	Success start item sk56uj1cku66k0ho
	 √ TestC bleed 10 tests 21-30 - Test 28 for Test C
	Finish test item sk56uj1cku66k0gh
	 √ TestC bleed 10 tests 21-30 - Test 29 for Test C
	Finish test item sk56uj1cku66k0gm
	Success finish item sk56uj1cku66k0gh
	Success finish item sk56uj1cku66k0gm
	 √ TestC bleed 10 tests 21-30 - Test 30 for Test C
	Finish test item sk56uj1cku66k0ho


	 30 passed (11s)
	Finish test item sk56uj1cku66jswh
	Finish test item sk56uj1cku66jwix
	Success finish item sk56uj1cku66k0ho
	Finish test item sk56uj1cku66jywi
	Success finish item sk56uj1cku66jwix
	Success finish item sk56uj1cku66jswh
	Success finish item sk56uj1cku66jywi
	Finish launch sk56uj1cku66jswc
	Success finish launch sk56uj1cku66jswc```
	</summary>
</details>

[kflore16]

A similar behavior is observed with ReportingApi. using concurrency, The logs for an earlier test is delayed by X tests for X amount of browser instances.

[kflore16]

TL:DR Tests are being completed in the wrong fixture when using concurrent runners. (Test A 1-10, Test B 11-20, and Test C 21-30 -> Test A completes till 1-12, Test B completes with 13-22, Test C completes 23-30 with 3 concurrent test runners)
If I were to fix this myself, where would I start? @AmsterGet

[AmsterGet]

Hello @kflore16 !
As I understand it correctly, events from tests fired using the ReportingAPI are isolated in the process in which the tests are executed, so in this case we need a mechanism to synchronize data between processes during test execution.
We already plan to implement this feature, but not in the near future.
Firstly, I would try to solve this using testcafe and its reporting system, in case of failure, I think a more custom solution will be needed here.