-
-
Notifications
You must be signed in to change notification settings - Fork 305
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
What's the best way to report pass / fail to sauce labs? #148
Comments
@ericelliott Maybe see https://github.com/defunctzombie/zuul which is a test runner capable of running tape tests in saucelabs and marking them as success/error accordingly, see https://github.com/algolia/faux-jax tests and badge :) |
I'm still interested in knowing if there is a canonical way to just get the results from |
@ericelliott did you manage to find a way to do this without pulling out the REST hammer? |
Not yet... had to turn my attention to other things for a while. |
I'm running into the same issue trying to integrate my tape tests into BrowserStack. The results object looks like exactly what I need. The results object is currently attached to the main test harness, but it doesn't look like the main test harness is ever exposed to where I can get at it. I would like to be able to do something like this var tapeResults = require('tape')()._results
tapeResults.on('done', function() {
reportToBrowserStack(tapeResults.pass, tapeResults.fail)
} I'm motivated enough to get good automated browser testing that I'm prepared to fork tape to expose the results object.
|
I suppose to be thorough, other changes would need to be made in order to collect the results from any test harnesses that were created manually. I'll worry about that later I guess? |
This is my code to report tape output to BrowserStack using the code in this pull request: var tapeResults = require('tape-with-automated-reporting').results()
var start = new Date()
tapeResults.on('done', function() {
var finish = new Date()
var browserStackResults = {
runtime: finish.getTime() - start.getTime(),
total: tapeResults.count,
passed: tapeResults.pass,
failed: tapeResults.fail,
tracebacks: []
}
BrowserStack.post('/_report', browserStackResults, function(){})
}) I'll publish it in its own module for BrowserStack users when that pull request is merged. I assume that something similar will do the trick for SauceLabs. |
@TehShrike Do you think we could also get events on passed/failed tests so that we can have a feedback loop of the test? |
For Saucelabs? It doesn't look like the results object emits many events at the moment. |
|
Anybody using sauce-tap-runner? How's that working for you? |
As an aside, I should have linked to this module in this thread after I published it: browserstack-tape-roperter. It looks like using something similar with the sauce-results module might be reasonable. |
Allows reporting failed/succeeded tests to SauceLabs/BrowserStack. Fixes tape-testing#148
Orchestrating sauce labs runs with
selenium-sauce
, what's the best way to report pass / fail to Sauce Labs?The text was updated successfully, but these errors were encountered: