Propose - migration Knative resources from one cluster to another by kn plugin

[ZhuangYuZY]

In what area(s)?

Describe the feature:

Describe the feature:

Today is multiple clouds world, customer may work on multiple k8s clusters from multiple cloud providers to run Knative applications, like IBM Cloud, AWS, GCP. Even in one cloud provider, client may also have multiple k8s clusters which running Knative applications. So there is a requirement that customer wants to clone and migrate Knative resources from one cluster and another cluster, instead of re-deploy Knative application multiple times. It will give benefit that keeping Knative configuration and revision consistent in multiple clusters, it will be very important because some traffic control rules depend on revisions. Beside clone and migration, backup and recovery are also the important requirement for Knative users to handle risk of cluster broken, as deploy Knative application again can not recover revision history.

Proposal to do:

1. Export all Knative resources. It can generate all of json/yml files for Knative service, revision, and route/configuration.
2. Import all Knative resources to another cluster and namespace. It will give the capability to recreate the Knative service with all revision history.
3. With export and import, we can migrate or clone Knative resources from one cluster to another cluster.

Implement plugin "migration" for kn client with 3 sub-commands as below:

  1. kn migration export --source-namespace ns-source -o knative-resource.yml
  2. kn migration import --destination-namespace ns-dest -i knative-resource.yml  --force
  3. kn migration run --destination-namespace ns-dest --destination-kubeconfig xxx  --source-namespace ns-source --source-kubeconfig xxx  --force

[zhangtbj]

This is what we did for the proposal first:

jordanzt@zhangtaodembp:~/Works/workspaces/gopath/src/github.ibm.com/BlueMix-Fabric/kn-migration-tool$ ./kn-migration-tool migrate --help
Migrate knative services from source cluster to destination cluster

Usage:
  knmt migrate [flags]

Flags:
  -d, --delete                          Force delete the knative resource after migration on source cluster
  -t, --destination-kubeconfig string   The kubeconfig of the destination knative resources (default is KUBECONFIG2 from ENV property)
  -m, --destination-namespace string    The namespace of the destination knative resources (default "default")
  -f, --force                           Force update if the knative resource exists on destination cluster
  -h, --help                            help for migrate
  -s, --source-kubeconfig string        The kubeconfig of the source knative resources (default is KUBECONFIG from ENV property)
  -n, --source-namespace string         The namespace of the source knative resources (default "default")

Global Flags:
      --config string       config file (default is $HOME/.knmt.yaml)
      --kubeconfig string   kube config file (default is KUBECONFIG from ENV property)

We can migrate the knSerivce, route and configuration from one namespace of the source cluster to another cluster.

But we cannot copy the revision of the knService to the destination cluster, we have to do more investigation.

[maximilien]

Thanks for this @zhangtbj and @ZhuangYuZY. Please add an entry to the next Knative client working group meeting agenda to perhaps demonstrate this Kn plugin to the community.

I created #397 that will raise the related issue of how to discover, install, and manage Kn plugins. So for now just keep your plugin in a public repository for people to try.

[toVersus]

@zhangtbj IIUC, you cannot create Revisions directly by coping existing Revisions from source cluster to target cluster because Revisions are immutable and must be created via a Configuration as described below. I think you can import by applying existing Revision spec to a Revision template in a Configuration each time.

Revisions are created by updates to a Configuration.
https://github.com/knative/serving/blob/master/docs/spec/overview.md#revision

[zhangtbj]

Hi @toVersus , Thanks for the quick response.

I also think I can link the revision with config, But I didn't find the link in somewhere.

For example, I have an example helloworld ksvc which has two revisions:

kubectl get revision
NAME                           SERVICE NAME                   GENERATION   READY     REASON
helloworld-nxj2f               helloworld-nxj2f               3            True
helloworld-x9zvw               helloworld-x9zvw               2            True

But I cannot see the revision list or history in the ksvc or configuration configurations. I can only see the latestCreatedRevisionName and latestReadyRevisionName in the config:

spec:
  revisionTemplate:
    metadata:
      creationTimestamp: null
    spec:
      container:
        image: us.icr.io/knative_jordan/helloworld:2.0
        name: ""
        resources: {}
      timeoutSeconds: 300
status:
  conditions:
  - lastTransitionTime: 2019-09-02T13:07:33Z
    status: "True"
    type: Ready
  latestCreatedRevisionName: helloworld-nxj2f
  latestReadyRevisionName: helloworld-nxj2f
  observedGeneration: 3

Do you have any idea how to get the revision list or history of one ksvc or config? Thanks a lot! :)

[toVersus]

I think you can get a list of revisions in a ksvc by calling ListRevisions with service name option like below. Did that answer your question?

package main

import (
	"github.com/knative/client/pkg/kn/commands"
	"github.com/knative/client/pkg/serving/v1alpha1"
	"log"
)

func main() {
	knParams := &commands.KnParams{}
	knParams.Initialize()
	client, err := knParams.NewClient("default")
	if err != nil {
		log.Fatalf("Failed to create kn client: %s", err)
	}
	revisions, err := client.ListRevisions(v1alpha1.WithService("hello"))
	if err != nil {
		log.Fatalf("Failed to list revisions by service name ('hello'): %s", err)
	}

	log.Printf("%#v\n", revisions)
}

[zhangtbj]

Hi @toVersus Thanks for your help! Now I can get all revision list and migrate to the destination cluster. But now in the new cluster, the new knative service only uses the new generated revision. I have tried many methods, I found I don't have a way to ask the ksvc/route or configuration to change to use a special revision. Do you have any idea how to modify it?

Name                          Creation Time                      Domain                                                                               Ready
helloworld                    2019-09-04 18:30:42 +0800 CST      helloworld-default.knativepipeline.us-south.containers.appdomain.cloud               false
  |- Revision helloworld-nrnb8 ( Generation: 1 , Ready: false )
  |- Revision helloworld-rwsw6 ( Generation: 2 , Ready: true )
  |- Revision helloworld-sftrm ( Generation: 1 , Ready: false )

For example the helloworld-nrnb8 revision is the new generated revision. helloworld-rwsw6 and helloworld-sftrm are migrated from source cluster, now I would like this helloworld service change to point to the Revision helloworld-rwsw6. Is this possible? Thanks a lot!

[maximilien]

@zhangtbj and @ZhuangYuZY can we contribute the code to a contrib/plugins/kn-migration directory on this repo? Submit PR. If I recall correctly, I believe that was the agreement after yesterday’s call.

@sixolet and @evankanderson please chime in if you intended a different location. Thx

[github-actions]

This issue is stale because it has been open for 90 days with no
activity. It will automatically close after 30 more days of
inactivity. Reopen the issue with /reopen. Mark the issue as
fresh by adding the comment /remove-lifecycle stale.