Uses custom plugin to publish zips to maven

[downsrob]

Signed-off-by: Clay Downs [email protected]

Issue #, if available:
#348

Description of changes:
Adds the custom plugin to publish zips to maven to the build.gradle. Introduces a custom build.sh script.

CheckList:

• Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[codecov-commenter]

Codecov Report

Merging #366 (1d10b2e) into main (4f8e722) will decrease coverage by 0.08%.
The diff coverage is n/a.

@@             Coverage Diff              @@
##               main     #366      +/-   ##
============================================
- Coverage     75.30%   75.21%   -0.09%     
- Complexity     2156     2158       +2     
============================================
  Files           263      263              
  Lines         12413    12429      +16     
  Branches       1974     1975       +1     
============================================
+ Hits           9347     9348       +1     
- Misses         1980     1999      +19     
+ Partials       1086     1082       -4     

Impacted Files	Coverage Δ
...statemanagement/model/destination/CustomWebhook.kt	67.14% <0.00%> (-28.58%)	⬇️
...gement/indexstatemanagement/action/ShrinkAction.kt	69.11% <0.00%> (-2.95%)	⬇️
.../rollup/action/start/TransportStartRollupAction.kt	71.76% <0.00%> (-1.18%)	⬇️
...nt/rollup/action/stop/TransportStopRollupAction.kt	76.47% <0.00%> (-1.18%)	⬇️
...temanagement/opensearchapi/OpenSearchExtensions.kt	66.66% <0.00%> (-0.43%)	⬇️
.../opensearch/indexmanagement/rollup/model/Rollup.kt	86.04% <0.00%> (ø)
...atemanagement/step/shrink/AttemptMoveShardsStep.kt	60.42% <0.00%> (ø)
...pensearch/indexmanagement/IndexManagementPlugin.kt	89.06% <0.00%> (+0.08%)	⬆️
...t/settings/LegacyOpenDistroManagedIndexSettings.kt	98.56% <0.00%> (+0.13%)	⬆️
...earch/indexmanagement/transform/model/Transform.kt	85.89% <0.00%> (+0.42%)	⬆️
... and 3 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4f8e722...1d10b2e. Read the comment docs.

[bowenlan-amzn]

Wondering do we need to exclude task in settings.gradle like here

[thalurur]

The zip publishing is purely for the build dependency graph right? If so do we have to publish the zips to all plugins even the leaf plugins like the IM in that graph or is there any use case that enables with publishing these zips other than build dependency graph for the opensearch plugins?

[downsrob]

@thalurur For BWC testing we utilize a zip. The only use case I can see for the leaves would be if we hope to utilize the maven zips in the future for our BWC testing

[downsrob]

Wondering do we need to exclude task in settings.gradle like here

I believe that we do not. According to step 7 in the setup guide for this plugin, "Exclude the following tasks in settings.gradle file, if the build script exists for the plugin and has the tasks publishToMavenLocal and publishAllPublicationsToStagingRepository, ..."

[dblock]

cc: @prudhvigodithi

That link in job-scheduler looks suspicious, care to track it down? not sure why that was needed

[prudhvigodithi]

That link in job-scheduler looks suspicious, care to track it down? not sure why that was needed

Hey @dblock from document point 7, this targets ALL publications for local maven repo and staging maven repo that includes publishPluginZipPublicationToMavenLocal, publishPluginZipPublicationToStagingRepository, I mean its fine to allow it to run as well without excludedTaskNames, should work but it will push the zips twice as publishPluginZipPublicationToStagingRepository will be called before publishPluginZipPublicationToZipStagingRepository which is the actual publish task.

[prudhvigodithi]

We should be also adding

mkdir -p $OUTPUT/maven/org/opensearch
cp -r ./build/local-staging-repo/org/opensearch/. $OUTPUT/maven/org/opensearch 

As this will put the local staged maven artifacts under $OUTPUT/maven/org/opensearch and CI will be using this folder $OUTPUT/maven/org/opensearch to publish to end maven
Please refer to job-scheduler https://github.com/opensearch-project/job-scheduler/blob/main/scripts/build.sh#L77-L79

[downsrob]

Thanks! Added.

[prudhvigodithi]

Looks good.
Please try to run the the build component locally, once passed tar/builds/opensearch/maven/org/opensearch/plugin/opensearch-index-management/ folder should be created and zip should be added as per maven coordinates.
Thank you