Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TESTING.asciidoc fix examples using forbidden annotation #34515

Merged
merged 2 commits into from
Oct 23, 2018
Merged

TESTING.asciidoc fix examples using forbidden annotation #34515

Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
ad26075
TESTING.asciidoc fix examples using forbidden annotation
lipsill Oct 16, 2018
edb40c0
remove references to unused annotations
lipsill Oct 17, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 4 additions & 6 deletions TESTING.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -91,23 +91,22 @@ Run any test methods that contain 'esi' (like: ...r*esi*ze...).

You can also filter tests by certain annotations ie:

* `@Nightly` - tests that only run in nightly builds (disabled by default)
* `@Backwards` - backwards compatibility tests (disabled by default)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't actually use @Backwards or @BadApple either! At this point we really only use @AwaitsFix I think.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I left the @BadApple and @Backwards so that the filtering examples with more complicated boolean operators and grouping are valid. But you are right, as only @AwaitsFix is still used, the other annotations should be removed as well.

I was thinking to remove the @Backwards separately as there is also a section using the backwards filtering which I guess is outdated: https://github.com/elastic/elasticsearch/blob/ad26075e989de81df61ec9f2eabce2bbc00a4896/TESTING.asciidoc#backwards-compatibility-tests
and should be replaced by:
https://github.com/elastic/elasticsearch/blob/ad26075e989de81df61ec9f2eabce2bbc00a4896/TESTING.asciidoc#testing-backwards-compatibility
I simply did not want to mix it all together ;)

Just let me know if you prefer having the changes at the same PR or several dedicated ones ;)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How would you feel about removing the complex filtering examples in this PR because, not that we really only have the one annotation, we really can only document the one filter and then doing a separate PR for @Backwards like you say?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lipsill I addressed the outdated section in #34050 already.

* `@AwaitsFix` - tests that are waiting for a bugfix (disabled by default)
* `@BadApple` - tests that are known to fail randomly (disabled by default)

Those annotation names can be combined into a filter expression like:

------------------------------------------------
./gradlew test -Dtests.filter="@nightly and not @backwards"
./gradlew test -Dtests.filter="@awaitsfix and not @backwards"
------------------------------------------------

to run all nightly test but not the ones that are backwards tests. `tests.filter` supports
the boolean operators `and, or, not` and grouping ie:
to run all tests waiting for a bug fix but not the ones that are backwards tests.
`tests.filter` supports the boolean operators `and, or, not` and grouping ie:


---------------------------------------------------------------
./gradlew test -Dtests.filter="@nightly and not(@badapple or @backwards)"
./gradlew test -Dtests.filter="@awaitsfix and not(@badapple or @backwards)"
---------------------------------------------------------------

=== Seed and repetitions.
Expand Down Expand Up @@ -160,7 +159,6 @@ Test groups can be enabled or disabled (true/false).
Default value provided below in [brackets].

------------------------------------------------------------------
./gradlew test -Dtests.nightly=[false] - nightly test group (@Nightly)
./gradlew test -Dtests.weekly=[false] - weekly tests (@Weekly)
./gradlew test -Dtests.awaitsfix=[false] - known issue (@AwaitsFix)
------------------------------------------------------------------
Expand Down