Get rid of hard-coded version in Jenkinsfile(s) in main

[dblock]

Is your feature request related to a problem? Please describe

In #6 we replaced @date by @1.0.0 in Jenkinsfile(s). Every time a version is incremented it will require doing that across many files, which is a lot of busy work.

Describe the solution you'd like

1. Hard-code @main.
2. Create a Release workflow that can be triggered by hand by any maintainer.
3. Replaces all @main by @version in that workflow.

So, to release current version (it's 1.0.0 now), maintained hits a button. Workflow checks out code, replaces all @main by @1.0.0, creates a branch called 1.0.0, commits the code to that branch, tags it as 1.0.0, creates a GitHub release, increments the version to 1.0.1 on main, commits and pushes everything.

Describe alternatives you've considered

No response

Additional context

No response

[peterzhuamazon]

I wonder if all the libs can load / preload a common-lib where the @main or @version is defined, so that we dont need to update all these files no matter what.

Writing a workflow to handle all the version update and send a PR seems like a hack to me.

Thanks.

[dblock]

+1 @peterzhuamazon if the version is located only in 1 place that works too

[gaiksaya]

There are 2 things:

1. Library version in Jenkins files
   Example: https://github.com/opensearch-project/opensearch-build/blob/main/jenkins/opensearch/distribution-build.jenkinsfile#L1

2. Library version within library.
   Example: https://github.com/opensearch-project/opensearch-build-libraries/blob/main/vars/archiveAssembleUpload.groovy#L9

For 1. The idea here is each file can have it own version that the file needs to be used. For example, we add some library that will only be used by dashboards. Now only dashboards jenkinsFile will use the new version (1.2.0).
In this way not every jenkinsFile needs to change the version but only the concerned one. we are starting with 1.0.0 but might change for each of them going forward. We can default to version located at one place (version.properties file) and override in each JenkinsFile going forward as required

For 2. In case of libraries used within libraries I believe main sounds good however, since we are using legacySCM the number does not matter as it locally checks out the vars content. (Need more research on that)

[peterzhuamazon]

New issues coming up after we have a merge of #13 this morning with main and 1.0.0/1.0.1 mismatch which cause jacoco reports generation to fail.

1. If using remote libs from opensearch-build-libraries repository with tag (ex: 1.0.0), make sure both the Jenkins Test file as well as the Jenkins Job file are overriding the libs version with the same tag (ex: 1.0.0), or Jacoco test will fail to generate reports. This would happen if defaultVersion in BuildPipelineTest.groovy (default to 'main') have a different HEAD commit id compares to tag commit id you defined to use.

super.setUp()
......
helper.registerSharedLibrary(
    library().name('jenkins')
        .defaultVersion('1.0.0')
        .allowOverride(true)
        .implicit(true)
        .targetPath('vars')
        .retriever(gitSource('https://github.com/opensearch-project/opensearch-build-libraries.git'))
        .build()
)

lib = library(identifier: '[email protected]', retriever: modernSCM([
    $class: 'GitSCMSource',
    remote: 'https://github.com/opensearch-project/opensearch-build-libraries.git',
]))

2. Even if you define above if your jenkinsfile is calling another lib from this repo, then the `remote` param in jenkinsfile is not taking effect while the `identifier` is. In my own test I force test file to have official repo while jenkinsfile having my own fork url, turns out:

super.setUp()
......
helper.registerSharedLibrary(
    library().name('jenkins')
        .defaultVersion('1.0.0')
        .allowOverride(true)
        .implicit(true)
        .targetPath('vars')
        .retriever(gitSource('https://github.com/opensearch-project/opensearch-build-libraries.git'))
        .build()
)

lib = library(identifier: '[email protected]', retriever: modernSCM([
    $class: 'GitSCMSource',
    remote: 'https://github.com/peterzhuamazon/opensearch-build-libraries.git',
]))

TestCCRPerfTest > testCCRPerfTestScript_Pipeline STANDARD_OUT
    Loading shared library jenkins with version 1.0.0
    git clone -b 1.0.0 --single-branch https://github.com/opensearch-project/opensearch-build-libraries.git [email protected]
Cloning into '[email protected]'...
......
    Loading shared library jenkins with version 2.0.0-tag
    git clone -b 2.0.0-tag --single-branch https://github.com/opensearch-project/opensearch-build-libraries.git [email protected]
Cloning into '[email protected]'...
warning: Could not find remote branch 2.0.0-tag to clone.
fatal: Remote branch 2.0.0-tag not found in upstream origin

3. If your jenkinsfile is calling another lib from libs repo in vars, then a suspicious `main` will always be called no matter what. We suspect this is coming from the `main` from all the local definition of `jenkins@main` in this repo for legacyscm, but it doesnt make sense that it is actively closing using gitSource:

    Loading shared library jenkins with version 1.0.1
    git clone -b 1.0.1 --single-branch https://github.com/peterzhuamazon/opensearch-build-libraries.git [email protected]
Cloning into '[email protected]'...

    Loading shared library jenkins with version 2.0.0-tag
    git clone -b 2.0.0-tag --single-branch https://github.com/peterzhuamazon/opensearch-build-libraries.git [email protected]
Cloning into '[email protected]'...

    Loading shared library jenkins with version main
    git clone -b main --single-branch https://github.com/peterzhuamazon/opensearch-build-libraries.git jenkins@main
Cloning into 'jenkins@main'...

Any commitid mismatch would cause jacoco to fail as it is using commitid as part of the identifier. All above 3 issues will cause Jacoco to fail.

Thanks.