-
Notifications
You must be signed in to change notification settings - Fork 3.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add suite start/finish events to listener #1118
Conversation
@@ -358,6 +358,7 @@ public Description getDescription() { | |||
public void run(final RunNotifier notifier) { | |||
EachTestNotifier testNotifier = new EachTestNotifier(notifier, | |||
getDescription()); | |||
testNotifier.fireTestSuiteStarted(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Note that ParentRunner
is the base class of Parameterized
. I'm not sure if people would find the events sent from Parameterized
confusing
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok, I see
suite started my.company.MyParameterizedTest
suite started [0]
test started
test
test finished
suite finished
suite started [1]
test started
test
test finished
suite finished
suite finished
Any ideas how to fix this?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I guess it should looks like
suite started my.company.MyParameterizedTest
test started
test
test finished
test started
test
test finished
suite finished
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if junit creates suite for each parameter maybe it's ok?
Ok, I see
suite started my.company.MyParameterizedTest
suite started [0]
test started
test
test finished
suite finished
suite started [1]
test started
test
test finished
suite finished
suite finished
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It might be easier to determine if this is a problem (and if so, the best way to address it) if you could describe the problem you are trying to solve with this change and/or how you plan to use the API
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am a developer of Allure Framework. We are using RunListener
to collect information about tests. This change can help me to get more information about test suites - as example get the right start/stop times for it.
BTW the problem with parameterized runner above -is not a problem for me, but I would like to choose the right way
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Well, since we can in theory report errors for the nested suites in Parameterized
this is okay with me. I want to wait for the other maintainer to see if they have concerns.
I am curious to find the other maintainers plan on using RunListener
for JUnit Lambda, and if so whether this change would work.
ref #444 |
@junit-team/junit-committers any objections to this pull? |
@@ -45,4 +45,12 @@ public void fireTestStarted() { | |||
public void fireTestIgnored() { | |||
notifier.fireTestIgnored(description); | |||
} | |||
|
|||
public void fireTestSuiteStarted() { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could you please add Javadoc that includes @since 4.13
to all new methods?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ae3e3a6
to
2cc8260
Compare
@@ -45,4 +45,22 @@ public void fireTestStarted() { | |||
public void fireTestIgnored() { | |||
notifier.fireTestIgnored(description); | |||
} | |||
|
|||
/** | |||
* Notify {@link #notifier} that a test suite is about to start. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM other than very minor issues with the Javadoc (which I can fix when I merge if you prefer). Sorry for the delay. Just need one of the other maintainers to add their LGTM and I'll merge this. |
@kcooney done |
This is a good thing in principle, and the implementation looks sound. I'd like to add some very clear documentation, however. Not every custom runner that reports a parent node and leaf nodes from getDescription() inherits from ParentRunner, so RunListeners must gracefully handle the situation where a suite node shows up in getDescription, but these new methods don't end up getting called. Likewise, we should probably add to the documentation of RunNotifier that the new methods are strongly encouraged and helpful, but not required; otherwise we risk essentially "breaking" existing Runners. |
@dsaff if the only concerns are the Javadoc, it might be easier to use the |
@dsaff It's been a month now. I would like to merge this (after applying the minor fixes). How about you improve the Javadoc after this is merged? |
Fair enough. |
@marcphilipp I can take care of it this weekend |
I created #1311 to try to resolve David's concerns about the Javadoc. |
testSuiteStarted() and testSuiteFinished() to make it clear that not all runners call these methods, but runners that call the started methods should also call the finished methods. Closes junit-team#1118
Merged (finallty!) @baev would you please update the release notes at https://github.com/junit-team/junit4/wiki/4.13-Release-Notes ? |
Done https://github.com/junit-team/junit4/wiki/4.13-Release-Notes#run-listener |
Thanks! |
Hi guys!
This pull request adds ability to handle suite start/finish events via
RunListener
.Maybe better to use
testStarted
andtestFinished
events instead new ones but I am afraid to break backwards compatibility. Tests will be added later.What are you guys think?