#8442 Java Pipeline Compiler

[original-brownbear]

Still WIP but 85% there:

Some notes for the time being:

• rbx-stdlib is not relevant anymore => removed
• We should have a stronger Gradle task for setting up Ruby that uses defined input and output paths, otherwise this stuff is pretty slow
• This gives us the ability to compile and run a pipeline from JUnit only :)
• Uses Gradle input and output definitions to not run redundant gem and bundler to save some time :)
• Manually invoking ./gradlew assemble bootstrap :ingest-converter:assemble from rake is admittedly kind of dirt, but invoking the full cycle assemble here would introduce an obvious circular dependency between Gradle and Rake ... so I opted for the shortcut here for a POC/Step1
• We should fix this up to also run the ITs
• Updated Readme accordingly

[original-brownbear]

@andrewvc FYI, this gives us very clean (imo) testing of stuff that requires Ruby code from JUnit + Gradle, just a one-off call now <= https://github.com/elastic/logstash/pull/8543/files#diff-c80ddff243fd1a3eb2407f3ccc53418bR22.

Obviously, requires a few changes to the whole Gradle-Rake interaction. Though the only "not under the hood" change is that ./gradle bootstrap has to come before the first use of a rake command.

Not sure if we want to go Gradle all the way now? I don't think we should merge this thing, and mix test changes + build system changes, but we def. need a change like this to get a clean Gradle test target.
The other option would be to not have Gradle self-contained for now and make running rake bootstrap a precondition for the JUnit tests (I kinda hate this one since it calls back to Gradle for the JRuby download ... but that will come with a little fewer changes required).

Either way, finally found a clean way to set this kind of thing up and getting something like this in would way speed up finishing up on the end-to-end codegen obviously :)

[original-brownbear]

@jakelandis maybe we could go over this one whenever you have some time (starting tomorrow, don't have much time today myself :)). I think with this kind of test (JUnit that heavily relies on Ruby code) on the horizon it may make sense to adjust the split between RSpec and JUnit run on CI (one option may be to abandon it altogether since the combined RSpec + JUnit run from #8517 is faster than an IT run anyhow right?).

[jakelandis]

@original-brownbear - sure, I can assist with adjusting our testing as needed.

However, I am bit hesitant to replace the 'pure' RSpec tests with the JUnit wrapper variant since the JUnit wrapper utilizes JRuby as a Java library as opposed to the implementation of Ruby. I don't want to get too far away from how the code is actually run.

I am in favor of adding in an extra step for the combined execution, or simply adjusting the JUnit tests to use gradle task that also runs the Rspecs. (the latter shouldn't require any infra changes)

[original-brownbear]

@jakelandis

JUnit wrapper utilizes JRuby as a Java library as opposed to the implementation of Ruby

+1 on this, the thing is, the whole discussion is tightly coupled to making the entry-point Java in #8161 , which itself would greatly benefit from heaving tests execute Java first (+ getting rid of the Ruby runtime singleton use). But yea, let's talk about this tomorrow, long story with a few twists :)

[original-brownbear]

Closing here in favor of the less invasive #8569