🏃extend ci-conformance.sh testing capability

[chewong]

What this PR does / why we need it:
Extending ci-conformance.sh testing capability to allow other tests to be run on a CAPZ cluster - a temporary solution until kubernetes-sigs/cluster-api#2826 is fixed.

This PR created a modified version of ci-conformance.sh called ci-entrypoint.sh instead of directly re-naming ci-conformance.sh so we won't break any jobs temporarily (I will open a separate PR to delete duplicated files after all test configurations in test-infra point to ci-entrypoint.sh). ci-entrypoint.sh will be the new entrypoint for all E2E tests. It can accept k/k master branch or release branches, and has the option to run upstream E2E tests or bring your own E2E tests by appending custom commands to ./scripts/ci-entrypoint.sh.

An example of how this script will be used in test-infra to test in-tree Azure Disk driver:

...
  spec:
    containers:
    - image: gcr.io/k8s-testimages/kubekins-e2e:v20200509-be34c64-master
      command:
      - runner.sh
      - ./scripts/ci-entrypoint.sh
      - bash
      - -c
      - >-
        kubectl apply -f templates/addons/storageclass-azure-disk.yaml &&
        cd ${GOPATH}/src/sigs.k8s.io/azuredisk-csi-driver &&
        make e2e-test
      env:
      - name: SKIP_UPSTREAM_E2E_TESTS
        value: "true"
      - name: USE_CI_ARTIFACTS
        value: "true"
...

/assign @CecileRobertMichon

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #

Special notes for your reviewer:

Please confirm that if this PR changes any image versions, then that's the sole change this PR makes.

Release note:

🏃extend ci-conformance.sh testing compatibility 

[chewong]

/retest

[CecileRobertMichon]

/test ls

[k8s-ci-robot]

@CecileRobertMichon: The specified target(s) for /test were not found.
The following commands are available to trigger jobs:

• /test pull-cluster-api-provider-azure-test
• /test pull-cluster-api-provider-azure-build
• /test pull-cluster-api-provider-azure-integration
• /test pull-cluster-api-provider-azure-e2e
• /test pull-cluster-api-provider-azure-verify
• /test pull-cluster-api-provider-azure-conformance-v1alpha3

Use /test all to run all jobs.

In response to this:

/test ls

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[CecileRobertMichon]

/test pull-cluster-api-provider-azure-conformance-v1alpha3

[CecileRobertMichon]

/test pull-cluster-api-provider-azure-conformance-v1alpha3

[CecileRobertMichon]

@chewong what about opening a test-infra PR to change the pull conformance job to point to ci-entrypoint.sh right now so we can test the new script in this PR? the pull conformance job optional and not run by default so it won't block other PRs

[chewong]

/test pull-cluster-api-provider-azure-conformance-v1alpha3

[chewong]

/test pull-cluster-api-provider-azure-conformance-v1alpha3-master
/test pull-cluster-api-provider-azure-conformance-v1alpha3-1-18

[chewong]

/hold
Since the jobs are already pointing to ci-entrypoint.sh, let me directly rename ci-conformance.sh to ci-entrypoint.sh

[CecileRobertMichon]

@chewong what about the periodic jobs? I think we need to merge this as-is before changing those

[chewong]

@chewong what about the periodic jobs? I think we need to merge this as-is before changing those

Ah right, we can merge this as-is and I will modify existing periodic jobs and add additional ones before removing ci-conformance.sh.

[chewong]

Oops I forgot to set USE_CI_ARTIFACTS to true in kubernetes/test-infra#17613. That's why pull-cluster-api-provider-azure-conformance-v1alpha3-1-18 and pull-cluster-api-provider-azure-conformance-v1alpha3-master are testing against 1.17.4 instead of master and 1.18. Let me open another PR in test-infra.

[chewong]

/test pull-cluster-api-provider-azure-conformance-v1alpha3-master
/test pull-cluster-api-provider-azure-conformance-v1alpha3-1-18

[chewong]

/test pull-cluster-api-provider-azure-conformance-v1alpha3-1-18

[chewong]

/hold cancel

[chewong]

/test pull-cluster-api-provider-azure-conformance-v1alpha3-master
/test pull-cluster-api-provider-azure-conformance-v1alpha3-1-18

[chewong]

/retest

[CecileRobertMichon]

/lgtm
/approve

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: CecileRobertMichon, chewong

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [CecileRobertMichon]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment