Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Smaller Provider Migration guides #203

Merged
merged 1 commit into from
May 16, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
158 changes: 158 additions & 0 deletions docs/sp-migration-conf.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,158 @@

# Configuration Migration to Smaller Providers

1. Backup managed resource, composite and claim manifests:

```bash
kubectl get managed -o yaml > backup-mrs.yaml
kubectl get composite -o yaml > backup-composites.yaml
kubectl get claim -n $NAMESPACE -o yaml > backup-claims.yaml
```

2. Update deletion policy to `Orphan`:

P.S: If this field is used in the composition files, we need to have special treatment

```bash
kubectl patch $(kubectl get managed -o name) -p '{"spec":{"deletionPolicy":"Orphan"}}' --type=merge
```

3. Generate smaller provider manifests

```bash
version_aws=v0.37.0
version_azure=v0.34.0
version_gcp=v0.34.0

rm -f "sp-manual.yaml" && touch "sp-manual.yaml"
rm -f "sp-family-manual.yaml" && touch "sp-family-manual.yaml"
kubectl get managed --no-headers -o jsonpath='{range .items[*]}{.apiVersion}{"\n"}{end}' | grep -E '(aws|gcp|azure).upbound.io' | sort | uniq | while read -r line
do
service=$(echo "${line}" | cut -d. -f1)
provider=$(echo "${line}" | cut -d. -f2)
if [[ ${provider} == "upbound" ]]; then
# azure.upbound.io is an exception where apiVersion does not contain the service name
# we have those resources in the family package
provider="azure"
fi
eval version=\$"version_${provider}"

for sp in config ${service}; do
filename="sp-manual.yaml"
providername="${provider}-${sp}"

if [[ "${sp}" == "config" ]] || [[ "${sp}" == "azure" ]]; then
# azure.upbound.io is an exception where apiVersion does not contain the service name
# we have those resources in the family package
providername="family-$provider"
filename="sp-family-manual.yaml"
fi
if ! cat "${filename}" | grep provider-${providername}:${version} > /dev/null; then
echo "apiVersion: pkg.crossplane.io/v1
kind: Provider
metadata:
name: upbound-release-candidates-provider-${providername}
spec:
package: xpkg.upbound.io/upbound-release-candidates/provider-${providername}:${version}
revisionActivationPolicy: Manual" >> "${filename}"
echo "---" >> "${filename}"
fi
done
done
```

4. Install config providers with `revisionActivationPolicy: Manual`:

Verify that `sp-family-manual.yaml` files are generated with the correct content

```bash
cat sp-family-manual.yaml
```

Install the family provider(s)

```bash
kubectl apply -f sp-family-manual.yaml
```

Make sure the family provider(s) are in `Installed: False` and `Healthy: True` state:

```bash
kubectl get providers.pkg
```

5. Install smaller providers with `revisionActivationPolicy: Manual`:

Verify that `sp-manual.yaml` files are generated with the correct content

```bash
cat sp-manual.yaml
```

Install the family provider

```bash
kubectl apply -f sp-manual.yaml
```

Make sure the smaller providers are in `Installed: False` and `Healthy: True` state:

```bash
kubectl get providers.pkg
```

6. Remove monolith provider(s) from dependsOn. Build/push and update the configuration

7. Delete monolith provider(s):

```bash
kubectl delete provider.pkg $PROVIDER_NAME
```

8. Update smaller providers with `revisionActivationPolicy:Automatic`:

```bash
sed 's/revisionActivationPolicy: Manual/revisionActivationPolicy: Automatic/' sp-family-manual.yaml > sp-family-automatic.yaml

kubectl apply -f sp-family-automatic.yaml
```


```bash
sed 's/revisionActivationPolicy: Manual/revisionActivationPolicy: Automatic/' sp-manual.yaml > sp-automatic.yaml

kubectl apply -f sp-automatic.yaml
```

9. Verify that MRs and providers are ready:

```bash
kubectl get managed
kubectl get provider.pkg
```

10. Find new dependencies and add them to `dependsOn`:

```bash
providers=( "aws:v0.37.0" "gcp:v0.34.0" "azure:v0.34.0" )

for pp in ${providers[@]}; do
provider="${pp%%:*}"
version="${pp##*:}"
smaller=$(grep -irn ${provider}.upbound.io/v1beta1 ${CONF_PATH} | awk '{print $3}'| cut -d '.' -f 1 | sort | uniq)

for s in ${smaller[@]}; do
if [ $s != "azure" ]; then
echo "- provider: xpkg.upbound.io/upbound-release-candidates/provider-$provider-$s";echo " version: \">=$version\"";
fi
done
done
```

12. Build/push/update the configuration to the new version

13. Update deletion policy to `Delete`:

```bash
kubectl patch $(kubectl get managed -o name) -p '{"spec":{"deletionPolicy":"Delete"}}' --type=merge
```
135 changes: 135 additions & 0 deletions docs/sp-migration-mr.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,135 @@
# Managed Resource Migration to Smaller Providers

1. Backup managed resources:

```bash
kubectl get managed -o yaml > backup-mrs.yaml
```

2. Update deletion policy to `Orphan`:

P.S: If this field is used in the managed resources, we need to have special treatment

```bash
kubectl patch $(kubectl get managed -o name) -p '{"spec":{"deletionPolicy":"Orphan"}}' --type=merge
```

3. Generate smaller provider manifests

```bash
version_aws=v0.37.0
version_azure=v0.34.0
version_gcp=v0.34.0

rm -f "sp-manual.yaml" && touch "sp-manual.yaml"
rm -f "sp-family-manual.yaml" && touch "sp-family-manual.yaml"
kubectl get managed --no-headers -o jsonpath='{range .items[*]}{.apiVersion}{"\n"}{end}' | grep -E '(aws|gcp|azure).upbound.io' | sort | uniq | while read -r line
do
service=$(echo "${line}" | cut -d. -f1)
provider=$(echo "${line}" | cut -d. -f2)
if [[ ${provider} == "upbound" ]]; then
# azure.upbound.io is an exception where apiVersion does not contain the service name
# we have those resources in the family package
provider="azure"
fi
eval version=\$"version_${provider}"

for sp in config ${service}; do
filename="sp-manual.yaml"
providername="${provider}-${sp}"

if [[ "${sp}" == "config" ]] || [[ "${sp}" == "azure" ]]; then
# azure.upbound.io is an exception where apiVersion does not contain the service name
# we have those resources in the family package
providername="family-$provider"
filename="sp-family-manual.yaml"
fi
if ! cat "${filename}" | grep provider-${providername}:${version} > /dev/null; then
echo "apiVersion: pkg.crossplane.io/v1
kind: Provider
metadata:
name: upbound-release-candidates-provider-${providername}
spec:
package: xpkg.upbound.io/upbound-release-candidates/provider-${providername}:${version}
revisionActivationPolicy: Manual" >> "${filename}"
echo "---" >> "${filename}"
fi
done
done
```

4. Install config providers with `revisionActivationPolicy: Manual`:


Verify that `sp-config-manual.yaml` files are generated with the correct content

```bash
cat sp-config-manual.yaml
```

Install the config provider(s)

```bash
kubectl apply -f sp-config-manual.yaml
```

Make sure the config provider(s) are in `Installed: False` and `Healthy: True` state:

```bash
kubectl get providers.pkg
```

5. Install smaller providers with `revisionActivationPolicy: Manual`:


Verify that `sp-manual.yaml` files are generated with the correct content

```bash
cat sp-manual.yaml
```

Install the config provider

```bash
kubectl apply -f sp-manual.yaml
```

Make sure the smaller providers are in `Installed: False` and `Healthy: True` state:

```bash
kubectl get providers.pkg
```

6. Delete monolith provider(s):

```bash
kubectl delete provider.pkg $PROVIDER_NAME
```

7. Update smaller providers with `revisionActivationPolicy:Automatic`:

```bash
sed 's/revisionActivationPolicy: Manual/revisionActivationPolicy: Automatic/' sp-config-manual.yaml > sp-config-automatic.yaml

kubectl apply -f sp-config-automatic.yaml
```


```bash
sed 's/revisionActivationPolicy: Manual/revisionActivationPolicy: Automatic/' sp-manual.yaml > sp-automatic.yaml

kubectl apply -f sp-automatic.yaml
```

8. Verify that MRs and providers are ready:

```bash
kubectl get managed
kubectl get provider.pkg
```

9. Update deletion policy to `Delete`:

```bash
kubectl patch $(kubectl get managed -o name) -p '{"spec":{"deletionPolicy":"Delete"}}' --type=merge
```
54 changes: 54 additions & 0 deletions docs/sp-migration-revert.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
# Reverting back to Monolith Providers

1. Backup managed resource manifests:

```bash
kubectl get managed -o yaml > backup-mrs.yaml
```

2. Update deletion policy to `Orphan`:

P.S: If this field is used in the managed resources, we need to have special treatment

```bash
kubectl patch $(kubectl get managed -o name) -p '{"spec":{"deletionPolicy":"Orphan"}}' --type=merge
```

3. Install monolith provider:

```bash
kubectl apply -f - <<EOF
apiVersion: pkg.crossplane.io/v1
kind: Provider
Metadata:
name: $PROVIDER_NAME
spec:
package: $PACKAGE
revisionActivationPolicy: Manual
EOF
```

5. Delete smaller provider(s):

```bash
kubectl delete provider.pkg $(kubectl get provider.pkg |grep upbound-release-candidates |awk '{print $1}')
```

6. Update monolith providers with `revisionActivationPolicy:Automatic`:

```bash
kubectl patch provider.pkg $PROVIDER_NAME --type=merge -p='{"spec":{"revisionActivationPolicy":"Automatic"}}'
```

7. Verify that MRs and providers are ready:

```bash
kubectl get managed
kubectl get provider.pkg
```

8. Update deletion policy to `Delete`:

```bash
kubectl patch $(kubectl get managed -o name) -p '{"spec":{"deletionPolicy":"Delete"}}' --type=merge
```