Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add test sourceset support to MRJAR plugin #106571

Merged
merged 9 commits into from
Mar 21, 2024

Conversation

rjernst
Copy link
Member

@rjernst rjernst commented Mar 20, 2024

Mrjar plugin supports main sourcesets automatically configured with newer java versions. This commit adds support for test sourcesets as well. The key difference between main and test is that test sourcesets also extend their main counterparts.

Mrjar plugin supports main sourcesets automatically configured with
newer java versions. This commit adds support for test sourcesets as
well. The key difference between main and test is that test sourcesets
also extend their main counterparts.
@rjernst rjernst added >enhancement :Delivery/Build Build or test infrastructure labels Mar 20, 2024
@rjernst rjernst requested a review from a team as a code owner March 20, 2024 17:10
@elasticsearchmachine elasticsearchmachine added Team:Delivery Meta label for Delivery team v8.14.0 labels Mar 20, 2024
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-delivery (Team:Delivery)

Copy link
Contributor

@mark-vieira mark-vieira left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like I'm missing the bit where we configure the test task. How are these source sets making it on to the testing classpath? Also, should we have individual test tasks for each source set? How do we plan on conditionally handling tests that are applicable only to a particular runtime version?

SourceSet sourceSet = javaExtension.getSourceSets().maybeCreate(sourcesetName);
for (String parentSourceSetName : parentSourceSets) {
GradleUtils.extendSourceSet(project, parentSourceSetName, sourcesetName);
List<Integer> testVersions = findSourceVersions(project, MRJAR_TEST_SOURCESET_PATTERN);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we need to do this twice. We can assume for every main source set, there is a corresponding test source set, and if it's empty that's fine the tasks will just get skipped with NO_SOURCE.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah ok, didn't think about NO_SOURCE.

String sourcesetName = SourceSet.TEST_SOURCE_SET_NAME + javaVersion;
if (mainVersions.contains(javaVersion)) {
// tests extend both their parent test source sets, and all of main sourcesets up to that version
testSourceSets.add(SourceSet.TEST_SOURCE_SET_NAME + javaVersion);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be:

Suggested change
testSourceSets.add(SourceSet.TEST_SOURCE_SET_NAME + javaVersion);
testSourceSets.add(SourceSet.MAIN_SOURCE_SET_NAME + javaVersion);

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, copy/paste issue

var jarTask = project.getTasks().withType(Jar.class).named(JavaPlugin.JAR_TASK_NAME);
jarTask.configure(task -> {
task.into("META-INF/versions/" + javaVersion, copySpec -> copySpec.from(sourceSet.getOutput()));
for (var version : versions) {
SourceSet sourceSet = javaExtension.getSourceSets().maybeCreate(baseSourcesetName + version);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are we attempting to create a source set here? Shouldn't this already exist? We also should definitely not be registering source sets in a task configuration action.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops, copy/paste issue, I'll fix

@rjernst
Copy link
Member Author

rjernst commented Mar 20, 2024

I feel like I'm missing the bit where we configure the test task.

In my mind I was thinking this was automatic with sourceset creation. Makes sense though that they are independent. I'll add that.

@rjernst
Copy link
Member Author

rjernst commented Mar 20, 2024

@mark-vieira I believe I addressed all your comments in 1b2d055

testTask.dependsOn(jarTask);

SourceSetContainer sourceSets = GradleUtils.getJavaSourceSets(project);
FileCollection mainRuntime = sourceSets.getByName(SourceSet.MAIN_SOURCE_SET_NAME + javaVersion).getOutput();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't we need to remove all the parent main source sets here. So if this was test23 we'd need to remove main21, main22 and main23 or else we'll hit jar hell.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Of course, you are right. I've fixed this to loop over the main sourcesets up to that point.


private void createTestTask(Project project, SourceSet sourceSet, int javaVersion) {
var jarTask = project.getTasks().withType(Jar.class).named(JavaPlugin.JAR_TASK_NAME);
project.getTasks().register(JavaPlugin.TEST_TASK_NAME + javaVersion, Test.class).configure(testTask -> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We'll want to have the check task depend on this so that they actually get run.

testTask.setClasspath(testRuntime.minus(mainRuntime).plus(project.files(jarTask)));

testTask.getJavaLauncher().set(
javaToolchains.launcherFor(spec -> { spec.getLanguageVersion().set(JavaLanguageVersion.of(javaVersion)); })
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: we can omit the curly braces in this lambda.

var jarTask = project.getTasks().withType(Jar.class).named(JavaPlugin.JAR_TASK_NAME);
jarTask.configure(task -> {
task.into("META-INF/versions/" + javaVersion, copySpec -> copySpec.from(sourceSet.getOutput()));
for (var version : versions) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking at this more, this feels a bit awkward. I feel like we should be doing this in addSourceSet instead.

@rjernst rjernst requested a review from mark-vieira March 20, 2024 22:46
@rjernst rjernst merged commit 3f8cc1e into elastic:main Mar 21, 2024
15 checks passed
@rjernst rjernst deleted the build/mrjar_test_sourcesets branch March 21, 2024 18:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Delivery/Build Build or test infrastructure >enhancement Team:Delivery Meta label for Delivery team v8.14.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants