Tap reporting of or1k-tests #92
Conversation
|
This Pr requires the rebuild of https://hub.docker.com/r/librecores/librecores-ci-openrisc @oleg-nenashev so that it picks up new changes in openrisc/or1k-tests#5 |
Nancy-Chauhan
force-pushed
the
Tap-Reporting
branch
from
505aae4
to
b9f8943
Compare
|
@oleg-nenashev @stffrdhrn This seems to work, Is it fine? https://ci.librecores.org/job/Projects/job/OpenRISC/job/mor1kx/view/change-requests/job/PR-92/
|
|
Why does the result show 344 tests? It seems like 43 x 8, what are the 8 different reports? The first instance you had there were just 43. |
|
@stffrdhrn @oleg-nenashev these are giving results for all parallel jobs , since or1k-tets run for every job so it produces report.tap ( from runtest.sh) for ever job . Earlier I checked with only 1 job on my local Jenkins instance |
… name for every parallel job
|
Some suggestions, the file name report.tap is saved to the current directory and the name is not very descriptive. It would be nice to be able to override the name/location. Also it would be good to be able to disable output of this all together when it is not needed. |
|
Hi @stffrdhrn I am thinking to introduce different parameter for labeling, can you suggest me what labels/description I should go with |
|
Please provide suggestions on what you intend to communicate. I am not clear which labelling you mean. |
|
@stffrdhrn I meant that what names I could use for different tap file generated here : https://ci.librecores.org/job/Projects/job/OpenRISC/job/mor1kx/view/change-requests/job/PR-92/2/tapResults/ , so that it is more clear and doesnt look that there are 8 same reports are produced |
|
The script has variables to define what to test. The variables are passes in by Jenkins. I suggest use those to define the report .tap name. Just build up the file name in Jenkins and pass it in. |
|
|
||
| // TAP Plugin is used which adds the support to TAP test result files to Jenkins which can be seen at https://ci.librecores.org/job/Projects/job/OpenRISC/ | ||
| void tapReporting() { | ||
| step([$class: "TapPublisher", testResults: "report.tap"]) |
There was a problem hiding this comment.
The report.tap name is the same for every run. It's hard to tell which report is for which run. Just like you use env vars in dockerrun() we should create a report name that is unique for each test. One thought, is the stage name available in an env var? We could make stage name more meaningful too.
There was a problem hiding this comment.
Hi @stffrdhrn I am already on it report-cappuccino--feature-immu-none.tap
does this sounds reasonable. Just $PIPELINE $EXTRA_CORE_ARGS paramaters are passed with some modifications
There was a problem hiding this comment.
Looks ok. Sorry the comment was stuck in my browser history, I posted on accident this morning.
Create a Tap Reporting on Jenkins Instance ( https://ci.librecores.org/job/Projects/job/OpenRISC/ ) with the PR (openrisc/or1k-tests#5), which creates a new report.tap with test results written in TAP format.
CC @oleg-nenashev @stffrdhrn