Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add @Ignore to track unit tests other than first one #101

Closed
matthewmorgan opened this issue Mar 1, 2016 · 10 comments
Closed

Add @Ignore to track unit tests other than first one #101

matthewmorgan opened this issue Mar 1, 2016 · 10 comments

Comments

@matthewmorgan
Copy link
Contributor

In the JS and ES tracks, all tests except the first for any given problem are disabled by default. This way the user is encouraged to make their solution pass one test at a time, enabling the next test as he or she proceeds.

In JUnit we could use @Ignore in front of the @Test annotation to enable this.

There's a second step, how do we tell gradle to ignore @Ignore so all the tests get run during development and CI builds?

@kytrinyx
Copy link
Member

kytrinyx commented Mar 3, 2016

I'm not sure we could tell gradle to ignore the ignores, but we could do a sed to delete lines with @Ignore in them before running the tests on CI. The problem would be if you run the tests locally to verify changes you wouldn't get the benefit of all of the tests. Also, if you run sed locally, then you're changing the actual files, which means that now we'd risk committing them.

A lot of tracks write a script to copy everything to tmp/ first, and then do whatever manipulations, and then finally run all the tests. Not sure if that's the way to go here.

@matthewmorgan
Copy link
Contributor Author

I think that's pretty much what we're doing with ES and JS. (temp dirs) @jtigger what do you think about that?

@sit
Copy link
Contributor

sit commented Mar 4, 2016

I think we could whip up something using JUnit categories, which are supported by Gradle.

Instead of using @Ignore, we could cook up a @Category(NotReady.class) annotation that we add to all but the first test. The Gradle build that we ship to users could then exclude the NotReady category. Users would delete that annotation as they move through the exercise. The build that we use for dev/CI builds could run all the tests, without excluding any.

(We'd need to ship some artifact into maven central or somewhere that includes our category marker interface, io.exercism.xjava.NotReady, unless we can find some way to ship common code with of the exercises. Or we could maybe write a Gradle task that generates this class before compiling everything. Probably just using a maven artifact would be less magical.)

@matthewmorgan
Copy link
Contributor Author

@sit that sounds like a great idea. Is this something you have time to help with? If not I can give it a try.

@sit
Copy link
Contributor

sit commented Apr 1, 2016

@jtigger @matthewmorgan Took me a while to get around to it, but put together a PR that does what I suggested.

It'd require a little work to get a little artifact with the NotReady annotation in it uploaded to Maven Central; the main problem being figuring out how to stash a PGP key on whatever it is we're using for CI. @kytrinyx, thoughts? Or maybe we just include this little annotation file in each of the source trees we download.

@kytrinyx
Copy link
Member

kytrinyx commented Apr 1, 2016

Let's just include the annotation file in the source trees. I can't think of a way to get the PGP key onto CI.

@jtigger
Copy link
Contributor

jtigger commented Apr 2, 2016

Yeah, there simple is no way to ignore the @Ignore. This approach makes a lot of sense, @sit.

In fact, I think it is helpful to have the annotation "class" sitting there in the source to reinforce to new-to-Java that this is our construct, not built-in to JUnit.

👍

@kytrinyx
Copy link
Member

kytrinyx commented Apr 2, 2016

reinforce to new-to-Java that this is our construct, not built-in to JUnit.

That's a really good point.

@jtigger
Copy link
Contributor

jtigger commented Sep 24, 2016

While you can't ignore @Ignore, there's no reason we can't delete them, when needed. This is what #139 does.

  • I've added @Ignores to all but the first test method of each exercise.
  • in the root build.gradle a Copy task is prepended to :test that copies all the test files to another directory, removing @Ignores.
  • in the Journey Test, after the exercise is fetched, we remove @Ignores.

@jtigger
Copy link
Contributor

jtigger commented Sep 25, 2016

Reviewed this with @matthewmorgan, this morning (see #139).

We discussed that it could be overwhelming to see a 500+ line GETTING_STARTED.md... when we think of "Getting Started" we think of a very short getting-your-foot-in-the-door starter. As rewritten, this is a full walkthrough.

We've captured that improvement in #141.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants