Add option to execute e2e test both locally and CI

[mszostok]

Description

This PR enables e2e test on CI with Kuberentes 1.15

Done:

• Refactor code for running the e2e test. Tests can be executed both on CI and locally
  • running test locally:
    make test-e2e

    NOTE: Docker is required for running e2e tests.

  • running on prow:
    https://status.build.kyma-project.io/view/gcs/kyma-prow-logs/logs/post-test-sc-e2e-job/1162418742185955328
• Remove the outdated Jenkins folder/scripts
• Update documentation about e2e tests
• Move all CI scripts to a single folder

TODO:

• Test changed Travis configuration on fork repo - done
• Add readme about current CI state (what is on Prow, what is on Travis etc) - will be done in separate PR

Resolves #2659

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: mszostok

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [mszostok]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[MHBauer]

Having never run kind before, my local run ends with:

Step 7/7 : ENTRYPOINT ["/opt/services/service-catalog"]
 ---> Running in 38d99c6d38b2
Removing intermediate container 38d99c6d38b2
 ---> ecfa7d656527
Successfully built ecfa7d656527
Successfully tagged quay.io/kubernetes-service-catalog/service-catalog-amd64:v0.2.1-14-gcc1af7f
docker tag quay.io/kubernetes-service-catalog/service-catalog-amd64:v0.2.1-14-gcc1af7f quay.io/kubernetes-service-catalog/service-catalog-amd64:canary
rm -rf ""build/"service-catalog""/tmp"
docker tag quay.io/kubernetes-service-catalog/service-catalog-amd64:v0.2.1-14-gcc1af7f quay.io/kubernetes-service-catalog/service-catalog:v0.2.1-14-gcc1af7f
docker tag quay.io/kubernetes-service-catalog/service-catalog-amd64:canary quay.io/kubernetes-service-catalog/service-catalog:canary

#################################################################################################
# Fri Aug 16 15:30:55 PDT 2019
# - Loading Service Catalog image into cluster...
#################################################################################################

Error: Image: "service-catalog:canary" not present locally
Deleting cluster "kind-ci" ...
$KUBECONFIG is still set to use /Users/mhb/.kube/kind-config-kind-ci even though that file has been deleted, remember to unset it
make: *** [test-e2e] Error 1

Looks to me like it built the image, but not with the expected name in the next step?

[mszostok]

From logs I can see that cluster was created, so kind was installed successfully.

What was the output of - Building Service Catalog image from sources...?

at the end, there should be docker tag service-catalog-amd64:canary service-catalog:canary

[MHBauer]

Added above, last line is
docker tag quay.io/kubernetes-service-catalog/service-catalog-amd64:canary quay.io/kubernetes-service-catalog/service-catalog:canary

[mszostok]

ok, now I see the problem. You have the env REGISTRY configured.
see: https://github.com/kubernetes-sigs/service-catalog/blob/master/Makefile#L88-L95

I assumed that everyone will have that empty by default. To solve that issue I will change the

make service-catalog-image

and

make user-broker-image

into

env REGISTRY="" VERSION=canary ARCH=amd64 make service-catalog-image

env REGISTRY="" VERSION=canary ARCH=amd64 make user-broker-image

so that will change envs temporary only for that particular command, thanks to that everyone will have the same setup.

thanks for finding that 🙂

btw. there is also an option to execute env SKIP_DEPS_INSTALLATION=true make test-e2e if u have preinstalled helm and kind on your computer. It speeds up test execution and allows u to do that also without internet connection. But still building the service catalog image takes to long. I will try to optimize that but after merging the CRD impl.

[MHBauer]

Oh, you're right, good catch! It's in my envrc, I probably don't need it set anymore.

$ cat .envrc
# for running contrib/jenkins/test_walkthrough.sh
export REGISTRY=quay.io/kubernetes-service-catalog/
# export VERSION=v0.0.21
# export CLEANUP=true

# for make test-e2e
export SERVICECATALOGCONFIG=~/.kube/config
export KUBECONFIG=~/.kube/config

Giving it another go around.

[MHBauer]

Works good. I'm impressed.

/lgtm

[mszostok]

@MHBauer I only squashed commits, so ur lgtm was removed

[piotrmiskiewicz]

/lgtm

[MHBauer]

😄