Remove brittle snapshot publish script.

[mch2]

Right now the bundle build relies on snapshot artifacts being placed in a local /maven folder during build and a script is invoked that publishes them during in our nightly jenkins workflow.

This script is brittle and relies on artifacts being named properly to be picked up - #1180 #439

Rather than publishing to a local folder and publishing the snapshot artifacts on the behalf of each repository. Each component should include its own gradle task to publish to our snapshot repository. This task can be invoked directly from each component's build.sh.

        maven {
          name = 'Snapshots'
          url = 'https://aws.oss.sonatype.org/content/repositories/snapshots'
          credentials {
            username "$System.env.SONATYPE_USERNAME"
            password "$System.env.SONATYPE_PASSWORD"
          }

This would require a flag to the build workflow to optionally publish the snapshots inside of a build.sh so that local builds do not attempt to publish.

[dblock]

@mch2 If you can show us how you think it should be done for one of the projects we can split the work for the others

[mch2]

Sorry for the delay here.

I would prefer each repository include the exact gradle task above to publish the snapshot to Sonatype directly vs us publishing a pre-built directory. The build system would then need a separate publish step that looks for an option in the manifest and conditionally invoke this task. This would look exactly like this.

Ideally we could do this with publishing release builds as well but given our current workflow stages pre-built artifacts after they are tested & signed, this won't be possible.

[dblock]

This would also fix #1463.

I think we should attempt what @mch2 is recommending above for opensearch-project/ml-commons#122.

[bbarani]

@mch2 Whats the status of this issue?

[mch2]

Hi @bbarani, I did not pick this up and missed that this was assigned to me. I cut this as a suggestion but don't have time at the moment. There is a bit of complexity here to integrate this with the build system & update the jenkins scripts.

[bbarani]

@prudhvigodithi Are you taking care of this issue as part of Maven publish process for individual zips?

[prudhvigodithi]

@bbarani, The process of publishing to end nexus maven repo is still done by the same script, but this time zips will also be added to the same local repo.
However the current publish script works fine for snapshot and it does not mess up the POM file.
I also see the issue's addressed have been closed #1180 #439
Off course it is not solved how @mch2 suggested, but looks to me like this PR fixed it.
Also @dblock I guess this issue that was mentioned is also resolved right ? is there still circular dependency on job-scheduler-spi SNAPSHOT?, as I can build anomaly-detection (Example ./build.sh manifests/2.0.0/opensearch-2.0.0.yml --snapshot --component anomaly-detection) directly even before job-scheduler.
So can we close this issue #1182 or am I missing anything ? :)
@dblock @mch2

[dblock]

I think for as long as publish/publish-snapshot.sh exists this issue should not be closed and @mch2's solution is probably still the right one.

[prudhvigodithi]

I see, how about this issue @dblock right when I ran the build for anomaly-detection, I dint find any circular dependency on job-scheduler-spi SNAPSHOT, so is this still valid ?

[dblock]

I see, how about this issue @dblock right when I ran the build for anomaly-detection, I dint find any circular dependency on job-scheduler-spi SNAPSHOT, so is this still valid ?

That's still a problem, the description is pretty clear I think, but happy to answer any questions in that issue!

[bbarani]

@prudhvigodithi @peterzhuamazon Can you update this issue?

[gaiksaya]

Hey everyone,

Since we have set up AWS GH action integration, we can now access the maven creds and publish the snapshots for each repo. The publish script can be removed completely and let each repo publish to maven on each push.
We would need a campaign issue once PoC is successful.

I believe we can remove this code then? https://github.com/opensearch-project/opensearch-build/blob/main/jenkins/opensearch/distribution-build.jenkinsfile#L163-L170

Let me know what you think @bbarani @dblock? We can use https://github.com/opensearch-project/spring-data-opensearch to test that out.

cc: @prudhvigodithi @peterzhuamazon @rishabh6788

[dblock]

Oooooh that's great news. You're saying we don't need Jenkins to publish to maven or Rubygems or other places? Let's do this!

[dblock]

cc: @wbeckler ^

[saratvemulapalli]

Hey everyone,

Since we have set up AWS GH action integration, we can now access the maven creds and publish the snapshots for each repo. The publish script can be removed completely and let each repo publish to maven on each push. We would need a campaign issue once PoC is successful.

I believe we can remove this code then? https://github.com/opensearch-project/opensearch-build/blob/main/jenkins/opensearch/distribution-build.jenkinsfile#L163-L170

Let me know what you think @bbarani @dblock? We can use https://github.com/opensearch-project/spring-data-opensearch to test that out.

cc: @prudhvigodithi @peterzhuamazon @rishabh6788

This is amazing, if you are looking for a specimen you could use up opensearch-sdk-java[1][2]

[1] #3011
[2] opensearch-project/opensearch-sdk-java#75

[gaiksaya]

Oooooh that's great news. You're saying we don't need Jenkins to publish to maven or Rubygems or other places? Let's do this!

We still want to run everything that is production on jenkins. @bbarani Can add more thoughts to this.
This includes publishing to all platforms. The plan is not to build anything on jenkins as part of publishing. See issue #1234 for more details
That is the reason we tried this integration with rubygems so that ruby gems could be signed during build in GHA and it worked. For publishing snapshots I believe we can do the same. @bbarani Need your approval to go ahead with this approach.

Also what's the plan for plugins? I understand publishing snapshots is a part of the distribution workflow today so are we talking about publishing snapshots as a part of GH actions when pushed to main for plugins as well as individual components like opensearch-sdk-java or spring-data-opensearch??

[dblock]

Oooooh that's great news. You're saying we don't need Jenkins to publish to maven or Rubygems or other places? Let's do this!

We still want to run everything that is production on jenkins. @bbarani Can add more thoughts to this. This includes publishing to all platforms. The plan is not to build anything on jenkins as part of publishing.

I'd like to hear more why we want this., other than "exercising centralized control" :)

[gaiksaya]

Each component should include its own gradle task to publish to our snapshot repository. This task can be invoked directly from each component's build.sh.

I don't think build.sh is required in anyway here anymore. If each component repo will be publishing on its own, calling the publishing gradle task should be enough. Taking OpenSearch as an example, is ./gradlew publishAllPublicationsToSnapshotsRepository the right task to call to publish all snapshots to snapshot repository?

[gaiksaya]

Update:
We have the code ready to merge to give repositories access to AWS secrets to retrieve credentials for publishing snapshots to maven. However we are still waiting on setting up a user with access only to snapshot repository.

Few things to do once that is ready:

• Update the secrets with actual credentials in AWS secrets
• Add the above code base in required repository
• Add IAM role as a GH secret to retrieve AWS secrets. Example
• Add workflow with push as a trigger to repo that will retrieve the secret and use it to publish the snapshots. Command might be ./gradlew publishAllPublicationsToSnapshotsRepository

[gaiksaya]

Issue created with sonatype requesting to add a new user with only access to publish snapshots: https://issues.sonatype.org/browse/OSSRH-87579

[gaiksaya]

We tested using opensearch-sdk-java and we were able to achieve snapshot publishing using GHA and AWS integration.

https://aws.oss.sonatype.org/content/repositories/snapshots/org/opensearch/sdk/opensearch-sdk-java/
Workflow: https://github.com/opensearch-project/opensearch-sdk-java/actions/runs/3888010406/jobs/6634902820

Once the above issue with sonatype is resolved we can move forward integrating the same with all other component.
Thanks!

[gaiksaya]

Regarding plugins, @dblock @mch2 What do you suggest? Currently plugin snapshots are being published as part of distribution workflow. Can we use this same process for plugins too so that they can publish snapshots independently?

[gaiksaya]

Update: Sonatype added new user that only has access to publish snapshots. More details in this issue.
We should be good to proceed and more confident in publishing snapshots to maven now.
Thanks!

[prudhvigodithi]

Regarding plugins, @dblock @mch2 What do you suggest? Currently plugin snapshots are being published as part of distribution workflow. Can we use this same process for plugins too so that they can publish snapshots independently?

Yes @gaiksaya thats a good idea and removes the dependency on the distribution workflow, each plugin team can manage their own way of publishing the snapshots.

One main downside I see, on how they publish the snapshots of plugins to the maven repo, with distribution workflow the groupID, artifcatID and version are constant and ensures all the plugins are published under same maven coordinates, but when each plugin has its own control this can deviate.

Second one is having snapshot publish logic at multiple places, half with plugins and other with distribution workflow, we should completely allow the plugins to manage the maven publishing or should be done via the workflow.

So I'm ok to for this same process for plugins too so that they can publish snapshots independently but with defined guardrails.

Thank you