crossplane · muvaf · Oct 14, 2022 · Oct 14, 2022 · Oct 14, 2022
@@ -1,10 +1,10 @@
 ---
 name: Bug Report
-about: Help us diagnose and fix bugs in Terrajet
+about: Help us diagnose and fix bugs in Upjet
 labels: bug
 ---
 <!--
-Thank you for helping to improve Terrajet!
+Thank you for helping to improve Upjet!
 
 Please be sure to search for open issues before raising a new one. We use issues
 for bug reports and feature requests. Please find us at https://slack.crossplane.io
@@ -13,7 +13,7 @@ for questions, support, and discussion.
 
 ### What happened?
 <!--
-Please let us know what behaviour you expected and how Terrajet diverged from
+Please let us know what behaviour you expected and how Upjet diverged from
 that behaviour.
 -->
 

@@ -1,10 +1,10 @@
 ---
 name: Feature Request
-about: Help us make Terrajet more useful
+about: Help us make Upjet more useful
 labels: enhancement
 ---
 <!--
-Thank you for helping to improve Terrajet!
+Thank you for helping to improve Upjet!
 
 Please be sure to search for open issues before raising a new one. We use issues
 for bug reports and feature requests. Please find us at https://slack.crossplane.io
@@ -18,7 +18,7 @@ Leading with this context helps frame the feature request so we can ensure we
 implement it sensibly.
 --->
 
-### How could Terrajet help solve your problem?
+### How could Upjet help solve your problem?
 <!--
-Let us know how you think Terrajet could help with your use case. 
+Let us know how you think Upjet could help with your use case. 
 -->
@@ -1,8 +1,8 @@
 <!--
-Thank you for helping to improve Terrajet!
+Thank you for helping to improve Upjet!
 
 Please read through https://git.io/fj2m9 if this is your first time opening a
-Terrajet pull request. Find us in https://slack.crossplane.io/messages/dev if
+Upjet pull request. Find us in https://slack.crossplane.io/messages/dev if
 you need any help contributing.
 -->
 
@@ -12,7 +12,7 @@ you need any help contributing.
 Briefly describe what this pull request does. Be sure to direct your reviewers'
 attention to anything that needs special consideration.
 
-We love pull requests that resolve an open Terrajet issue. If yours does, you
+We love pull requests that resolve an open Upjet issue. If yours does, you
 can uncomment the below line to indicate which issue your PR fixes, for example
 "Fixes #500":
 

@@ -0,0 +1,3 @@
+## Code of Conduct
+
+Upjet is under [the Apache 2.0 license](LICENSE) with [notice](LEGAL_NOTICE).
@@ -10,13 +10,9 @@ See [design document][design-doc] for more details.
 
 Feel free to test the following Crossplane providers built using Upjet:
 
-* [Provider Jet AWS](https://github.com/crossplane-contrib/provider-jet-aws/releases)
-* [Provider Jet Azure](https://github.com/crossplane-contrib/provider-jet-azure/releases)
-* [Provider Jet GCP](https://github.com/crossplane-contrib/provider-jet-gcp/releases)
-
-**NOTE**: Upjet is in its very early stages and we're making many changes that
-can affect the output and the runtime. Please check the generated code before
-running in production.
+* [Provider AWS](https://github.com/upbound/provider-aws/releases)
+* [Provider Azure](https://github.com/upbound/provider-azure/releases)
+* [Provider GCP](https://github.com/upbound/provider-gcp/releases)
 
 ## Generating a New Provider Using Upjet
 
@@ -56,17 +52,20 @@ license restrictions.
 * [Kubeform](https://github.com/kubeform/kubeform)
 * [Terraform Operator](https://github.com/isaaguilar/terraform-operator)
 
-## Code of Conduct
 
-terrajet adheres to the [Code of
-Conduct](https://github.com/crossplane/crossplane/blob/master/CODE_OF_CONDUCT.md)
-as the core Crossplane project.
+## Contributing
+
+* [Generating a Provider](docs/generating-a-provider.md)
+* [Configuring a Resource](docs/configuring-a-resource.md)
+* [Reference Generation](docs/reference-generation.md)
+* [New v1beta1 Resources](docs/new-v1beta1-resource.md)
+* [Moving Resources to v1beta1](docs/moving-resources-to-v1beta1.md)
+* [Testing Instructions](docs/testing-instructions.md)
+* [Testing Resources by Using Uptest](docs/testing-resources-by-using-uptest.md)
 
 ## Licensing
 
 All rights of upjet belongs to Upbound Inc.
 
-[![FOSSA Status](https://app.fossa.io/api/projects/git%2Bgithub.com%2Fcrossplane%2Fterrajet.svg?type=large)](https://app.fossa.io/projects/git%2Bgithub.com%2Fcrossplane%2Fterrajet?ref=badge_large)
-
 [design-doc]: https://github.com/crossplane/crossplane/blob/master/design/design-doc-terrajet.md
 [provider-template]: https://github.com/crossplane/provider-template
@@ -703,10 +703,10 @@ So, an interface must be passed to the related configuration field for adding in
 [Description]: https://github.com/hashicorp/terraform-plugin-sdk/blob/e3325b095ef501cf551f7935254ce942c44c1af0/helper/schema/schema.go#L120
 [Optional]: https://github.com/hashicorp/terraform-plugin-sdk/blob/e3325b095ef501cf551f7935254ce942c44c1af0/helper/schema/schema.go#L80
 [Computed]: https://github.com/hashicorp/terraform-plugin-sdk/blob/e3325b095ef501cf551f7935254ce942c44c1af0/helper/schema/schema.go#L139
-[tags_all for jet AWS resources]: https://github.com/crossplane-contrib/provider-jet-aws/blob/c045bae7736da4a9cc80e7fc0fc4cfcd78de60df/config/overrides.go#L86
-[boot_disk.initialize_params.labels]: https://github.com/crossplane-contrib/provider-jet-gcp/blob/f90456c4fc032c021c8179ef061f4803bc01b488/config/compute/config.go#L157
-[AWS region]: https://github.com/crossplane-contrib/provider-jet-aws/blob/a5b6a6fea65634c475a84583e1e1776a048a0df9/config/overrides.go#L325
-[this figure]: images/terrajet-externalname.png
+[tags_all for jet AWS resources]: https://github.com/upbound/provider-aws/blob/main/config/overrides.go#L62
+[boot_disk.initialize_params.labels]: https://github.com/upbound/provider-gcp/blob/main/config/compute/config.go#L121
+[AWS region]: https://github.com/upbound/provider-aws/blob/main/config/overrides.go#L32
+[this figure]: images/upjet-externalname.png
 [Initializers]: #initializers
 [InitializerFns]: https://github.com/upbound/upjet/blob/ae78a0a4c438f01717002e00fac761524aa6e951/pkg/config/resource.go#L289
 [NewInitializerFn]: https://github.com/upbound/upjet/blob/ae78a0a4c438f01717002e00fac761524aa6e951/pkg/config/resource.go#L207

@@ -0,0 +1,227 @@
+## Manual Migration Guide to Official Providers
+
+This document describes the steps that need to be applied to migrate from
+community providers to official providers manually. We plan to implement a
+client-based tool to automate this process. 
+
+For the sake of simplicity, we only focus on migrating managed resources 
+and compositions in this guide. These scenarios can be extended
+with other tools like ArgoCD, Flux, Helm, Kustomize, etc.
+
+### Migrating Managed Resources
+
+Migrating existing managed resources to official providers can be simplified
+as import scenarios. The aim is to modify the community provider's scheme to official
+providers and apply those manifests to import existing cloud resources.
+
+To prevent a conflict between two provider controllers reconciling for the same external resource,
+we're scaling down the old provider. This can also be eliminated with the new [pause annotation feature].
+
+
+1) Backup managed resource manifests
+```bash
+kubectl get managed -o yaml > backup-mrs.yaml
+```
+2) Update deletion policy to `Orphan` with the command below:
+```bash
+kubectl patch $(kubectl get managed -o name) -p '{"spec": {"deletionPolicy":"Orphan"}}' --type=merge
+```
+3) Install the official provider
+4) Install provider config
+5) Update managed resource manifests to the new API version `upbound.io`, external-name annotations and new field names/types. You can use
+[Upbound Marketplace] for comparing CRD schema changes. It is also planned to extend current documentation with external-name syntax in this [issue].
+```bash
+cp backup-mrs.yaml op-mrs.yaml
+vi op-mrs.yaml
+```
+6) Scale down Crossplane deployment
+```bash
+kubectl scale deploy crossplane --replicas=0
+```
+7) Scale down native provider deployment
+```bash
+kubectl scale deploy ${deployment_name} --replicas=0
+```
+8) Apply updated managed resources and wait until they become ready
+```bash
+kubectl apply -f op-mrs.yaml
+```
+9) Delete old MRs
+```bash
+kubectl delete -f backup-mrs.yaml
+kubectl patch -f backup-mrs.yaml -p '{"metadata":{"finalizers":[]}}' --type=merge
+```
+10) Delete old provider config
+```bash
+kubectl delete providerconfigs ${provider_config_name}
+```
+11) Delete old provider
+```bash
+kubectl delete providers ${provider_name}
+```
+12) Scale up Crossplane deployment
+```bash
+kubectl scale deploy crossplane --replicas=1
+``` 
+
+#### Migrating VPC Managed Resource
+
+In below, we display the required changes to migrate a native provider-aws VPC resource to an official 
+provider-aws VPC. As you can see, we have updated the API version and some field names/types in spec 
+and status subresources. To find out which fields to update, we need to compare the CRDs in the current
+provider version and the target official provider version. 
+
+```diff
+-   apiVersion: ec2.aws.crossplane.io/v1beta1
++   apiVersion: ec2.aws.upbound.io/v1beta1  
+    kind: VPC
+    metadata:
+      annotations:
+        crossplane.io/external-create-pending: "2022-09-23T12:20:31Z"
+        crossplane.io/external-create-succeeded: "2022-09-23T12:20:33Z"
+        crossplane.io/external-name: vpc-008f150c8f525bf24
+        kubectl.kubernetes.io/last-applied-configuration: |
+          {"apiVersion":"ec2.aws.crossplane.io/v1beta1","kind":"VPC","metadata":{"annotations":{},"name":"ezgi-vpc"},"spec":{"deletionPolicy":"Delete","forProvider":{"cidrBlock":"192.168.0.0/16","enableDnsHostNames":true,"enableDnsSupport":true,"instanceTenancy":"default","region":"us-west-2","tags":[{"key":"Name","value":"platformref-vpc"},{"key":"Owner","value":"Platform Team"},{"key":"crossplane-kind","value":"vpc.ec2.aws.crossplane.io"},{"key":"crossplane-name","value":"ezgi-plat-ref-aws-tcg6t-n6zph"},{"key":"crossplane-providerconfig","value":"default"}]},"providerConfigRef":{"name":"default"}}}
+      creationTimestamp: "2022-09-23T12:18:21Z"
+      finalizers:
+      - finalizer.managedresource.crossplane.io
+      generation: 2
+      name: ezgi-vpc
+      resourceVersion: "22685"
+      uid: 81211d98-57f2-4f2e-a6db-04bb75cc60ff
+    spec:
+      deletionPolicy: Delete
+      forProvider:
+        cidrBlock: 192.168.0.0/16
+-       enableDnsHostNames: true
++       enableDnsHostnames: true
+        enableDnsSupport: true
+        instanceTenancy: default
+        region: us-west-2
+        tags:
+-       - key: Name
+-         value: platformref-vpc
+-       - key: Owner
+-         value: Platform Team
+-       - key: crossplane-kind
+-         value: vpc.ec2.aws.crossplane.io
+-       - key: crossplane-name
+-         value: ezgi-vpc
+-       - key: crossplane-providerconfig
+-         value: default
++         Name: platformref-vpc
++         Owner: Platform Team
++         crossplane-kind: vpc.ec2.aws.upbound.io
++         crossplane-name: ezgi-vpc
++         crossplane-providerconfig: default
+      providerConfigRef:
+        name: default
+```
+
+
+### Migrating Crossplane Configurations
+
+Configuration migration can be more challenging. Because, in addition to managed resource migration, we need to 
+update our composition and claim files to match the new CRDs. Just like managed resource migration, we first start to import
+our existing resources to official provider and then update our configuration package version to point to the 
+official provider. 
+
+
+1) Backup managed resource manifests
+```bash
+kubectl get managed -o yaml > backup-mrs.yaml
+```
+2) Scale down Crossplane deployment
+```bash
+kubectl scale deploy crossplane --replicas=0
+```
+3) Update deletion policy to `Orphan` with the command below:
+```bash
+kubectl patch $(kubectl get managed -o name) -p '{"spec": {"deletionPolicy":"Orphan"}}' --type=merge
+```
+4) Update composition files to the new API version `upbound.io`, external-name annotations and new field names/types. You can use
+[Upbound Marketplace] for comparing CRD schema changes. It is also planned to extend current documentation with external-name syntax in this [issue].
+5) Update `crossplane.yaml` file with official provider dependency.
+6) Build and push the new configuration version
+7) Install Official Provider
+8) Install provider config
+9) Update managed resource manifests with the same changes done on composition files
+```bash
+cp backup-mrs.yaml op-mrs.yaml
+vi op-mrs.yaml
+```
+10) Scale down native provider deployment
+```bash
+kubectl scale deploy ${deployment_name} --replicas=0
+```
+11) Apply updated managed resources and wait until they become ready
+```bash
+kubectl apply -f op-mrs.yaml
+```
+12) Delete old MRs
+```bash
+kubectl delete -f backup-mrs.yaml
+kubectl patch -f backup-mrs.yaml -p '{"metadata":{"finalizers":[]}}' --type=merge
+```
+13) Update the configuration to the new version
+```bash
+cat <<EOF | kubectl apply -f -
+apiVersion: pkg.crossplane.io/v1
+kind: Configuration
+metadata:
+  name: ${configuration_name}
+spec:
+  package: ${configuration_registry}/${configuration_repository}:${new_version}
+EOF
+```
+14) Scale up Crossplane deployment
+```bash
+kubectl scale deploy crossplane --replicas=1
+```
+15) Delete old provider config
+```bash
+kubectl delete providerconfigs ${provider_config_name}
+```
+16) Delete old provider
+```bash
+kubectl delete providers ${provider_name}
+```
+
+#### Migrating VPC in a Composition
+
+In the below, there is a small code snippet from platform-ref-aws to update VPC resource.  
+
+```diff
+   resources:
+     - base:
+-        apiVersion: ec2.aws.crossplane.io/v1beta1
++        apiVersion: ec2.aws.upbound.io/v1beta1
+         kind: VPC
+         spec:
+           forProvider:
+             spec:
+               region: us-west-2
+               cidrBlock: 192.168.0.0/16
+               enableDnsSupport: true
+-              enableDnsHostNames: true
++              enableDnsHostnames: true
+-              tags:
+-              - key: Owner
+-                value: Platform Team
+-              - key: Name
+-                value: platformref-vpc
++              tags:
++                Owner: Platform Team
++                Name: platformref-vpc
+       name: platformref-vcp
+```
+
+
+PRs which fully update existing platform-refs can be found below:
+- platform-ref-aws: https://github.com/upbound/platform-ref-aws/pull/69
+- platform-ref-azure: https://github.com/upbound/platform-ref-azure/pull/10
+- platform-ref-gcp: https://github.com/upbound/platform-ref-gcp/pull/22
+
+[pause annotation feature]: https://github.com/upbound/product/issues/227
+[Upbound Marketplace]: https://marketplace.upbound.io/
+[issue]: https://github.com/upbound/official-providers/issues/792
@@ -0,0 +1,19 @@
+## Moving Untested Resources to v1beta1
+
+For outside contributors, we wanted to form a baseline for resource test
+efforts. Therefore, we created a map: `ExternalNameNotTestedConfigs`. This map
+contains the external name configurations of resources, but they were not tested.
+And also, the resources’ schemas and controllers will not be generated after
+running `make generate`/`make reviewable` commands.
+
+For the generation of this resource’s schema and controller, we need to add it to
+the `ExternalNameConfigs` map. After this addition, this resource’s schema and
+the controller will be started to generate. By default, every resource that was 
+added to this map will be generated in the `v1beta1` version.
+
+Here there are two important points. For starting to test efforts, you need a
+generated CRD and controller. And for this generation, you need to move your
+resource to the `ExternalNameConfigs` map. Then you can start testing and if the
+test effort is successful, the new entry can remain on the main map. However, if
+there are some problems in tests, and you cannot validate the resource please
+move the entry to `ExternalNameNotTestedConfigs` again.