Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update to js-reporters 2 #249

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open

Conversation

Krinkle
Copy link
Contributor

@Krinkle Krinkle commented Jun 6, 2021

  • Fixes Mocha 8 compatibility.
  • Fixes broken builds that use QUnit.todo.

Fixes #248.
Fixes #247.

Krinkle added 3 commits June 6, 2021 22:26
Important changes:

* Mocha 8 and later are now supported. Previously, it's pending tests
  were incorrectly reported as undefined or failing.

* `QUnit.todo()` is now supported. Previously, these intentional
  failures were reported by browserstack-runner as failing tests.

  With js-reported 2.1 the runEnd report counts them as `todo` instead
  of as `failed`.

The API that browserstack-runner depends on has not changed since 1.1.0.
<https://github.com/js-reporters/js-reporters/blob/v2.1.0/CHANGELOG.md>.

Specifically, browserstack-runner captures the following data in
lib/_patch/reporter.js, and proccesses it in lib/server.js under
the "_progress" and "_report" handlers:

* The `testEnd` event, reading:
  - `name` string,
  - `suiteName` string,
  - `status` string,
  - `errors` array.

* The `runEnd` event, reading:
  - `status` string,
  - `testCounts` object with ints `total`, `passed`, `failed`, `skipped`,
  - `runtime` int.

These still exist the same way in the latest release:
<https://github.com/js-reporters/js-reporters/blob/v2.1.0/spec/cri-draft.adoc#runend>

Fixes browserstack#248.
Ref browserstack#247.
The update to js-reporters 2.1.0 in the previous commit has fixed
the most important problem, which is the build status reported
by the `runEnd` event, handled via "_report" submission.

However, the todo tests were still printed as errors in the output
which is confusing.

Before:

```
[Windows 8, Chrome 91.0] Error: "test.todo.each [0]" failed
  Expected: true
  Actual: false
[Windows 8, Chrome 91.0] Passed: 332 tests, 321 passed, 0 failed, 7 skipped; ran for 1255ms
```

After:

```
[Windows 10, Firefox 88.0] Passed: 332 tests, 321 passed, 0 failed, 7 skipped; ran for 1910ms
```

Fixes browserstack#247.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Test fails incorrectly with Mocha 8 and later Using QUnit.todo() causes build to fail instead of succeed
1 participant