Elasticsearch maven/gradle plugin

[nik9000]

Now that we're quite explicit about not supporting embedded Elasticsearch, maybe we can made a maven and a gradle plugin to that jvm projects can use to start Elasticsearch before running their integration tests?

It'd certainly be nice to have one.

[dadoonet]

For maven, this plugin exists for years now: https://github.com/alexcojocaru/elasticsearch-maven-plugin

BTW I wrote some guide to do that without a plugin using maven and ant:

• http://david.pilato.fr/blog/2016/07/29/elasticsearch-real-integration-tests/
• http://david.pilato.fr/blog/2016/08/03/elasticsearch-real-integration-tests-with-security-enabled/

[nik9000]

Nice!

I wonder if we should have something as part of our build so we don't break it. That plugin looks wonderful though!

[rjernst]

For gradle, it also exists already as part of our build tools. This is what the qa projects use, elasticsearch.rest-test (we could probably split it out to allow only starting up a cluster, eg elasticsearch.test-cluster).

[ywelsch]

For the transport client as well as for the test framework it would be nice to have tests that check whether the dependencies can be properly consumed (from Maven/Gradle/SBT) when using a local maven repo with all dependencies properly packaged. This would consist of a simple build file and a sample class file that connects to an Elasticsearch instance running locally and executing a few commands. We could then have assertions on the exact output we expect. We could then publish the same code as part of the documentation.

[nik9000]

@ywelsch that makes sense to me. I think maybe that is a separate issue that is blocked on having this.

[ywelsch]

@nik9000 you're right, it's a separate concern. I've opened #21338

[hub-cap]

If these both exist in some form, is there more work to do here? cc @nik9000 @rjernst

[elasticmachine]

Pinging @elastic/es-core-infra

[rjernst]

This can be closed. While we still don't have any official maven plugin for testing like this, something does exist, and it is not a priority to provide something officially.

[jsmucr]

It would be nice to have at least one example of a custom plugin along with the integration tests and a testing cluster setup. Sadly, not everything can be solved using the standalone analyzers and there's pretty much nothing to encourage developers into creating new plugins. In order to create a new plugin, I have to dig very deep into the ES code and spend weeks by eventually finding out that something that would make sense was deprecated in favor of something that's overly complicated. Disappointing. :(

[alpar-t]

@jsmucr we are sorry you feel that way.
Have you seen the example plugins we would also love to hear more on discuss you can also get help there.

[jsmucr]

@atorok I created the tests I currently use by reusing parts of the Elasticsearch test suite itself. Honestly, I didn't look at the examples much because I'm only interested in EDI files analysis. So in the end I ended up testing my plugin the same way you (I mean the devs) test the analysis plugins officially. But then I started to have problems with testing the plugin using the same mapping I use in the production because of lack of the standard analyzers and filters in the embedded ES. This led me here only to learn that the official way of testing is deprecated.

Later I figured out that it's not completely true that the functionality I was after isn't available. Just the tokenizer and filter factories were missing and unpublished. So I reimplemented these factories and republished it under my own name just for the testing purposes.