-
Notifications
You must be signed in to change notification settings - Fork 3.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add Ordering, Orderable and @OrderWith #1130
Merged
Merged
Changes from all commits
Commits
Show all changes
29 commits
Select commit
Hold shift + click to select a range
44b64df
Add Ordering, Orderable and @OrderWith.
kcooney e68c015
Remove unreachable code
kcooney 4770fff
Handle @RunWith in RunnerBuilder
kcooney a19a3ab
Change Sorter to no longer extend Ordering.
kcooney 82e019f
Revert "Change Sorter to no longer extend Ordering."
kcooney f19f035
Replace Ordering.order(List) with Ordering.orderDescription(Description)
kcooney 8390bc7
Rename GenericOrdering to GeneralOrdering
kcooney 6e38776
Merge branch 'master' into ordering
kcooney 1ba37d2
Add Ordering.Context so Orderings can use the Description to get
kcooney 62ca6a0
Rename parameters in applyOrdering() and Sorter.apply() from "runner"…
kcooney 753842d
Check ordering correctness in Ordering.
kcooney ea71fa4
Pass Ordering.Context in constructor when reflectively creating insta…
kcooney 121744f
Remove unnecessary call to unmodifableCollection()
kcooney 9d71b2f
Remove use of ReflectiveOperationException
kcooney 5a7186b
Merge branch 'master' into ordering
kcooney 2f9d21a
Fix javadoc for orderWith()
kcooney c6de86d
Minor formatting fixes
kcooney 5a3f954
Add Ordering.Factory.
kcooney f2f6131
Merge branch 'master' into ordering
kcooney 4ce52c1
Add missing @since Javadoc for Ordering methods
kcooney d8a1ee6
Merge branch 'master' into ordering
kcooney bfbad94
Minor formatting fix.
kcooney 9fb4772
Merge branch 'master' into ordering
kcooney 78ee8c6
Rename ComparsionBasedOrdering to ComparatorBasedOrdering; fix Javadoc
kcooney 9d1e2aa
Rename GeneralOrdering to Orderer and make it no longer implement Ord…
kcooney 550654a
Add OrderingRequest, to ensure orderWith() orders once
kcooney b2ce86a
Introduce MemoizingRequest
kcooney ca3e040
Sort tests in AllManipulationTests
kcooney 3e6d464
Removed Comparators.reverse(); it's broken and the JDK provides a bet…
kcooney File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
30 changes: 30 additions & 0 deletions
30
src/main/java/org/junit/internal/requests/MemoizingRequest.java
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,30 @@ | ||
package org.junit.internal.requests; | ||
|
||
import java.util.concurrent.locks.Lock; | ||
import java.util.concurrent.locks.ReentrantLock; | ||
|
||
import org.junit.runner.Request; | ||
import org.junit.runner.Runner; | ||
|
||
abstract class MemoizingRequest extends Request { | ||
private final Lock runnerLock = new ReentrantLock(); | ||
private volatile Runner runner; | ||
|
||
@Override | ||
public final Runner getRunner() { | ||
if (runner == null) { | ||
runnerLock.lock(); | ||
try { | ||
if (runner == null) { | ||
runner = createRunner(); | ||
} | ||
} finally { | ||
runnerLock.unlock(); | ||
} | ||
} | ||
return runner; | ||
} | ||
|
||
/** Creates the {@link Runner} to return from {@link #getRunner()}. Called at most once. */ | ||
protected abstract Runner createRunner(); | ||
} |
29 changes: 29 additions & 0 deletions
29
src/main/java/org/junit/internal/requests/OrderingRequest.java
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,29 @@ | ||
package org.junit.internal.requests; | ||
|
||
import org.junit.internal.runners.ErrorReportingRunner; | ||
import org.junit.runner.Request; | ||
import org.junit.runner.Runner; | ||
import org.junit.runner.manipulation.InvalidOrderingException; | ||
import org.junit.runner.manipulation.Ordering; | ||
|
||
/** @since 4.13 */ | ||
public class OrderingRequest extends MemoizingRequest { | ||
private final Request request; | ||
private final Ordering ordering; | ||
|
||
public OrderingRequest(Request request, Ordering ordering) { | ||
this.request = request; | ||
this.ordering = ordering; | ||
} | ||
|
||
@Override | ||
protected Runner createRunner() { | ||
Runner runner = request.getRunner(); | ||
try { | ||
ordering.apply(runner); | ||
} catch (InvalidOrderingException e) { | ||
return new ErrorReportingRunner(ordering.getClass(), e); | ||
} | ||
return runner; | ||
} | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,26 @@ | ||
package org.junit.runner; | ||
|
||
import java.lang.annotation.ElementType; | ||
import java.lang.annotation.Inherited; | ||
import java.lang.annotation.Retention; | ||
import java.lang.annotation.RetentionPolicy; | ||
import java.lang.annotation.Target; | ||
|
||
import org.junit.runner.manipulation.Ordering; | ||
|
||
/** | ||
* When a test class is annotated with <code>@OrderWith</code> or extends a class annotated | ||
* with <code>@OrderWith</code>, JUnit will order the tests in the test class (and child | ||
* test classes, if any) using the ordering defined by the {@link Ordering} class. | ||
* | ||
* @since 4.13 | ||
*/ | ||
@Retention(RetentionPolicy.RUNTIME) | ||
@Target(ElementType.TYPE) | ||
@Inherited | ||
public @interface OrderWith { | ||
/** | ||
* Gets a class that extends {@link Ordering}. The class must have a public no-arg constructor. | ||
*/ | ||
Class<? extends Ordering.Factory> value(); | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
27 changes: 27 additions & 0 deletions
27
src/main/java/org/junit/runner/manipulation/Alphanumeric.java
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,27 @@ | ||
package org.junit.runner.manipulation; | ||
|
||
import java.util.Comparator; | ||
|
||
import org.junit.runner.Description; | ||
|
||
/** | ||
* A sorter that orders tests alphanumerically by test name. | ||
* | ||
* @since 4.13 | ||
*/ | ||
public final class Alphanumeric extends Sorter implements Ordering.Factory { | ||
|
||
public Alphanumeric() { | ||
super(COMPARATOR); | ||
} | ||
|
||
public Ordering create(Context context) { | ||
return this; | ||
} | ||
|
||
private static final Comparator<Description> COMPARATOR = new Comparator<Description>() { | ||
public int compare(Description o1, Description o2) { | ||
return o1.getDisplayName().compareTo(o2.getDisplayName()); | ||
} | ||
}; | ||
} |
21 changes: 21 additions & 0 deletions
21
src/main/java/org/junit/runner/manipulation/InvalidOrderingException.java
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,21 @@ | ||
package org.junit.runner.manipulation; | ||
|
||
/** | ||
* Thrown when an ordering does something invalid (like remove or add children) | ||
* | ||
* @since 4.13 | ||
*/ | ||
public class InvalidOrderingException extends Exception { | ||
private static final long serialVersionUID = 1L; | ||
|
||
public InvalidOrderingException() { | ||
} | ||
|
||
public InvalidOrderingException(String message) { | ||
super(message); | ||
} | ||
|
||
public InvalidOrderingException(String message, Throwable cause) { | ||
super(message, cause); | ||
} | ||
} |
21 changes: 21 additions & 0 deletions
21
src/main/java/org/junit/runner/manipulation/Orderable.java
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,21 @@ | ||
package org.junit.runner.manipulation; | ||
|
||
/** | ||
* Interface for runners that allow ordering of tests. | ||
* | ||
* <p>Beware of using this interface to cope with order dependencies between tests. | ||
* Tests that are isolated from each other are less expensive to maintain and | ||
* can be run individually. | ||
* | ||
* @since 4.13 | ||
*/ | ||
public interface Orderable extends Sortable { | ||
|
||
/** | ||
* Orders the tests using <code>orderer</code> | ||
* | ||
* @throws InvalidOrderingException if orderer does something invalid (like remove or add | ||
* children) | ||
*/ | ||
void order(Orderer orderer) throws InvalidOrderingException; | ||
} |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A couple musings here (which I may not finish all at once)
It might be so much easier to instead just say:
@OrderWith public static Ordering ordering = new FastestOrdering("/tmp/test-times.txt");
or some such.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@dsaff I don't see how what you are proposing would be 100% impossible. You could write your own ordering that reads the file the first time
apply()
is called. If the location of the file should be specified by the test writer, then yourOrdering
could use a custom annotation.I'd hate to add more broiler-plate because of a possibly rare use case. Although I've personally never been bothered by adding a method with an annotation that returns something, other people have objected loudly to this (for example, with Rules).
The problem with passing a
Description
when you create an ordering is that an ordering applies to all children. Luckily, you can get aDescription
from aRunner
so a customOrdering
do a different algorithm based on theDescription
inside ofOrdering.apply()
. I could be convinced thatOrdering.order()
should also take in the parentDescription
.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So my apply() method would cast the Object to a Runner, call getDescription, and then look for the custom annotation in the Description? On balance, I'm not a fan of asking end-extenders to cast things, but could see doing it if there are benefits gained.
I can't personally call this use case "possibly rare", since it's occurred in the majority of test-case sorting scenarios I've written (and since test re-ordering was an essential part of my thesis, I may be in the sorry position of having personally seen a healthy percentage of all test-ordering schemes our species have attempted...)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@dsaff would you be fine with allowing a static method annotated with
OrderWith
as an additional way of specifying an ordering, and document whether the static method overrides the annotation?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@dsaff alternatively, as I suggested, we could add the parent
Description
to the parameters passed toOrdering.order()
, which should handle your use case.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry for the long delay, but I hope that we can release JUnit 4.13 in 2017 and I'd like to include this pull.
I updated the code to pass a context object that allows you to access the top-level description
of the class annotated with
@OrderWith
. This will handle David's use-case of " run the tests from the fastest to the slowest, based on a record of previous runtimes kept in a file". The filename couldeither be specified as an annotation on the class annotated with
@OrderWith
or it could bederived from the class name of the class annotated with
@OrderWith