Add migration tool, guide and add script to test if migration job from api-server to CRD is working

[mszostok]

Description

Changes proposed in this pr:

• Add migration tool and guide which allow clients to do easily migration from api-server to CRD approach.

  Scenario

  • pre-upgrade hook

    • Make API Server read-only
    • Check if API Server deployment exists. If deployment was not found we skip the migration.
    • Backup Service Catalog custom resources to files in a Persistent Volume.
    • Remove OwnerReference fields in all Secrets pointed by any ServiceBinding
    • Remove all Service Catalog resources

  • upgrade

    • remove the api-server
    • remove the etcd storage
    • adjust secrets, RBAC etc.
    • upgrade controller manager
    • install webhook server

  • post-upgrade

    • Scale down the Controller Manager to avoid any resource processing while backing up resources.
    • Restore all resources. The migration tool sets all necessary fields added in the Service Catalog 0.3.0.
    • Add proper OwnerReference to all Secrets pointed by ServiceBindings.
    • Scale up the Controller Manager.

    Full documentation with diagrams is added in this PR: https://github.com/kubernetes-sigs/service-catalog/pull/2710/files#diff-c13a535c132e80742d0b5f424960c632

• Add script to test if migration job from api-server to CRD is working (will be executed as CI pipeline).

  Scenario:

  1. Create k8s cluster (kind)
  2. Install Service Catalog from latest api-server release
  3. Created sample data (sample service broker, service instances etc.)
  4. Upgrade Service Catalog to the version built from sources
  5. Execute tests to check if data from step 3 are still working
  6. Shutdown cluster
  

  Pipeline in test-infra: Add Service Catalog pipeline for checking migration scenario kubernetes/test-infra#14464
  Example of this pipeline execution: https://storage.googleapis.com/kyma-prow-logs/logs/post-test-sc-upgrade-job/1176903408213299200/build-log.txt
  Fixes:

• Provide Migration Path for Apiserver -> CRDs #2634

• Add upgrade test to pipeline #2660

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: mszostok

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [mszostok]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[jberkhahn]

/retest

[jberkhahn]

I think you actually broke some stuff in the instance controller in pkg/controller...

[mszostok]

That’s true, there is sth wrong and I’m still working on it;)

But overall logic and documentation will not be changed

[mszostok]

@piotrmiskiewicz @jberkhahn PTAL

about issue: I had to adjust test because here we also fixed repetitive updates on status entry. Here are changes which I applied ae2caca

[mszostok]

Waiting for your reviews :) and also block tide merge. We will need to do that manually using the rebase and merge action to keep the commits history
/hold

[piotrmiskiewicz]

/lgtm

[jberkhahn]

"you have performed"

[jberkhahn]

"Upgrading Service Catalog from versions 0.2.x (and earlier) to 0.3.x requires a database migration."

[jberkhahn]

"backing up resources"

[jberkhahn]

"Backup and delete resources"

[jberkhahn]

"The resulting"

[piotrmiskiewicz]

/lgtm

[mszostok]

@jberkhahn changes applied:)

there is also PR for adding CI for running migration tests: kubernetes/test-infra#14464.

It was executed on our infra: https://storage.googleapis.com/kyma-prow-logs/logs/post-test-sc-upgrade-job/1176903408213299200/build-log.txt. You can also execute it locally:

./contrib/hack/ci/run-migration-tests.sh

[jberkhahn]

/lgtm