Add smoke tests

[Amulyam24]

Signed-off-by: Amulyam24 [email protected]

What this PR does / why we need it:
Adds smoke tests for testing pre workload creation steps.
Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #524

Special notes for your reviewer:

/area provider/ibmcloud

1. Please confirm that if this PR changes any image versions, then that's the sole change this PR makes.

Release note:

NONE

[netlify]

✔️ Deploy Preview for kubernetes-sigs-cluster-api-ibmcloud ready!

🔨 Explore the source changes: d33f427

🔍 Inspect the deploy log: https://app.netlify.com/sites/kubernetes-sigs-cluster-api-ibmcloud/deploys/61fb660d26397e0007aeba31

😎 Browse the preview: https://deploy-preview-528--kubernetes-sigs-cluster-api-ibmcloud.netlify.app

[k8s-ci-robot]

Hi @Amulyam24. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[Amulyam24]

@mkumatag Right now running this is pretty straight forward. I'm not sure if any additional scenario can be introduced here since all the cases(creating kind cluster, deploying providers correctly) are covered via suite_test.go(which might be redundant once we start running e2e tests). PTAL and let me know your thoughts.

GOPROXY was changed due to the following error

Step 5/18 : ENV GOPROXY https://goproxy.io,direct                                                                                            
 ---> Using cache                                                                                                                            
 ---> feb3b3f1833e                                                                                                                           
Step 6/18 : RUN go mod download                                                                                                              
 ---> Running in a1bd6cc081e1                                                                                                                
go mod download: github.com/smartystreets/[email protected]: verifying module: github.com/smartystreets/assertion
[email protected]: Get "https://goproxy.io/sumdb/sum.golang.org/lookup/github.com/smartystreets/[email protected]
180507-b2de0cb4f26d": read tcp 172.17.0.2:48652->119.28.201.50:443: read: connection reset by peer                                           
go mod download: github.com/smartystreets/[email protected]: read tcp 172.17.0.2:48652->119.28.201.50:443: read: connection reset by peer      
The command '/bin/sh -c go mod download' returned a non-zero code: 1                                                                         
make: *** [Makefile:206: e2e-image] Error 1 

[mkumatag]

@mkumatag Right now running this is pretty straight forward. I'm not sure if any additional scenario can be introduced here since all the cases(creating kind cluster, deploying providers correctly) are covered via suite_test.go(which might be redundant once we start running e2e tests). PTAL and let me know your thoughts.

agree but I'm not sure we can run the e2e all the time but we can run the smoke for every PR and e2e optional(that's what most of the folks do as of now like aws etc..)

[mkumatag]

/ok-to-test

[mkumatag]

@Amulyam24 seems like its not able to find the customize binary!

  �[91mFailed to generate the manifest for "infrastructure-ibmcloud" / "v0.2.0"
  Unexpected error:
      <*errors.withStack | 0xc0009a4048>: {
          error: <*errors.withMessage | 0xc0000a4160>{
              cause: <*errors.withStack | 0xc0009a4018>{
                  error: <*exec.Error | 0xc000428020>{
                      Name: "kustomize",
                      Err: <*errors.errorString | 0xc0003cb3e0>{
                          s: "executable file not found in $PATH",
                      },
                  },
                  stack: [0x1546667, 0x1a85fba, 0x1a85188, 0x1ad746b, 0x1ad9cc5, 0x4d1a85, 0x4d1005, 0x7348fa, 0x73283a, 0x732205, 0x733afe, 0x733732, 0x757ec7, 0x757a73, 0x759d32, 0x765ee5, 0x765cfe, 0x1ad701e, 0x515042, 0x46b181],
              },
              msg: "failed to execute kustomize: ",
          },
          stack: [0x1a86051, 0x1a85188, 0x1ad746b, 0x1ad9cc5, 0x4d1a85, 0x4d1005, 0x7348fa, 0x73283a, 0x732205, 0x733afe, 0x733732, 0x757ec7, 0x757a73, 0x759d32, 0x765ee5, 0x765cfe, 0x1ad701e, 0x515042, 0x46b181],
      }
      failed to execute kustomize: : exec: "kustomize": executable file not found in $PATH
  occurred�[0m

  /home/prow/go/pkg/mod/sigs.k8s.io/cluster-api/[email protected]/framework/clusterctl/repository.go:89

  �[91mFull Stack Trace�[0m

[Amulyam24]

It is failing with another issue now

/home/prow/go/src/sigs.k8s.io/cluster-api-provider-ibmcloud/test/e2e/suite_test.go:106
failed to run clusterctl init
Unexpected error:
    <*errors.withStack | 0xc001b5f1a0>: {
        error: <*errors.withMessage | 0xc00104d020>{
            cause: <*errors.withStack | 0xc001b5f170>{
                error: <*errors.withMessage | 0xc00104d000>{
                    cause: <*runtime.missingKindErr | 0xc0011a33e0>{
                        data: "unstructured object has no kind",
                    },
                    msg: "failed to get cert-manager object /, Kind=, /",
                },
                stack: [0x1a2155e, 0x1a1f0a5, 0x1a22e71, 0x10d3f5b, 0x10d405c, 0x10d3fb9, 0x10d499f, 0x1a22d45, 0x1a1f005, 0x1a1ed05, 0x1a75a7a, 0x1a7c7dc, 0x1a7e112, 0x1ad7b90, 0x1ad9dbf, 0x4d1a85, 0x4d1005, 0x7348fa, 0x73283a, 0x732205, 0x733afe, 0x733732, 0x757ec7, 0x757a73, 0x759d32, 0x765ee5, 0x765cfe, 0x1ad701e, 0x515042, 0x46b181],
            },
            msg: "action failed after 10 attempts",
        },
        stack: [0x1a22da5, 0x1a1f005, 0x1a1ed05, 0x1a75a7a, 0x1a7c7dc, 0x1a7e112, 0x1ad7b90, 0x1ad9dbf, 0x4d1a85, 0x4d1005, 0x7348fa, 0x73283a, 0x732205, 0x733afe, 0x733732, 0x757ec7, 0x757a73, 0x759d32, 0x765ee5, 0x765cfe, 0x1ad701e, 0x515042, 0x46b181],
    }
    action failed after 10 attempts: failed to get cert-manager object /, Kind=, /: Object 'Kind' is missing in 'unstructured object has no kind'
occurred
/home/prow/go/pkg/mod/sigs.k8s.io/cluster-api/[email protected]/framework/clusterctl/client.go:84

[mkumatag]

/retest

[mkumatag]

We got 1.1.0 version released with a fix to use the latest cert-manager packages, let's update the go.mod file for the same part of this PR to fix the issue..

[Amulyam24]

@mkumatag, made the changes. The test passes now. PTAL.

[mkumatag]

I was expecting to show at least one spec here but I don't see any in the result, can we see why it shows 0 here in the count

[Amulyam24]

I was expecting to show at least one spec here but I don't see any in the result, can we see why it shows 0 here in the count

@mkumatag, There are not specs under Run sanity tests as of now(added TO-DO). The 2 skipped ones are a part of e2e_test.go. All the required sanity checks are run only as a part of BeforeSuite in the current e2e flow.

[mkumatag]

/lgtm
/approve

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Amulyam24, mkumatag

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [mkumatag]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment