Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Terrajet AWS Provider for Crossplane #272

Merged
merged 18 commits into from
Mar 1, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 24 additions & 10 deletions docs/add-ons/crossplane.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,26 +39,40 @@ You can optionally customize the Helm chart that deploys `Crossplane` via the fo
```

### Crossplane AWS Provider Deployment
AWS Provider for Crossplane gets deployed by default when you enable `enable_crossplane = true`.
The below configuration helps you to upgrade the AWS provider version and lets you define custom IAM policies to manage AWS resources through IRSA.
This module provides options to deploy the following AWS providers for Crossplane. These providers disabled by default, and it can be enabled using the config below.

Crossplane requires Admin like permissions to create and update resources similar to Terraform deploy role.
- [AWS Provider](https://github.com/crossplane/provider-aws)
- [Terrajet AWS Provider](https://github.com/crossplane-contrib/provider-jet-aws)

Please find more details from [AWS Provider](https://github.com/crossplane/provider-aws)
_NOTE: Crossplane requires Admin like permissions to create and update resources similar to Terraform deploy role.
This example config uses AdministratorAccess, but you should select a policy with the minimum permissions required to provision your resources._

Config to deploy [AWS Provider](https://github.com/crossplane/provider-aws)
```hcl
crossplane_provider_aws = {
provider_aws_version = "v0.23.0"
additional_irsa_policies = ["<ENTER_YOUR_IAM_POLICY>"]
}
# Creates ProviderConfig -> aws-provider
crossplane_aws_provider = {
enable = true
provider_aws_version = "v0.24.1" # Get the latest version from https://github.com/crossplane/provider-aws
additional_irsa_policies = ["arn:aws:iam::aws:policy/AdministratorAccess"]
}
```

Config to deploy [Terrajet AWS Provider](https://github.com/crossplane-contrib/provider-jet-aws)
```hcl
# Creates ProviderConfig -> jet-aws-provider
crossplane_jet_aws_provider = {
enable = true
provider_aws_version = "v0.4.1" # Get the latest version from https://github.com/crossplane-contrib/provider-jet-aws
additional_irsa_policies = ["arn:aws:iam::aws:policy/AdministratorAccess"]
}
```

Checkout the full [example](examples/crossplane) to deploy Crossplane with `kubernetes-addons` module
Checkout the full [example](../../examples/crossplane) to deploy Crossplane with `kubernetes-addons` module

### GitOps Configuration
The following properties made available for use when managing the add-on via GitOps.

Refer to [locals.tf](modules/kubernetes-addons/crossplane/locals.tf) for latest config. GitOps with ArgoCD Add-on repo is located [here](https://github.com/aws-samples/ssp-eks-add-ons/blob/main/chart/values.yaml)
Refer to [locals.tf](../../modules/kubernetes-addons/crossplane/locals.tf) for latest config. GitOps with ArgoCD Add-on repo is located [here](https://github.com/aws-samples/ssp-eks-add-ons/blob/main/chart/values.yaml)

```hcl
argocd_gitops_config = {
Expand Down
91 changes: 79 additions & 12 deletions examples/crossplane/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,34 @@ This example deploys the following Basic EKS Cluster with VPC
- Creates EKS Cluster Control plane with one managed node group
- Crossplane Add-on to EKS Cluster
- AWS Provider for Crossplane
- Terrajet AWS Provider for Crossplane

## Crossplane Design

```mermaid
graph TD;
subgraph AWS Cloud
id1(VPC)-->Private-Subnet1;
id1(VPC)-->Private-Subnet2;
id1(VPC)-->Private-Subnet3;
id1(VPC)-->Public-Subnet1;
id1(VPC)-->Public-Subnet2;
id1(VPC)-->Public-Subnet3;
Public-Subnet1-->InternetGateway
Public-Subnet2-->InternetGateway
Public-Subnet3-->InternetGateway
Public-Subnet3-->Single-NATGateway
Private-Subnet1-->EKS{{"EKS #9829;"}}
Private-Subnet2-->EKS
Private-Subnet3-->EKS
EKS==>ManagedNodeGroup;
ManagedNodeGroup-->|enable_crossplane=true|id2([Crossplane]);
subgraph Kubernetes Add-ons
id2([Crossplane])-.->|crossplane_aws_provider.enable=true|id3([AWS-Provider]);
id2([Crossplane])-.->|crossplane_jet_aws_provider.enable=true|id4([Terrajet-AWS-Provider]);
end
end
```

## How to Deploy
### Prerequisites:
Expand Down Expand Up @@ -53,35 +81,74 @@ This following command used to update the `kubeconfig` in your local machine whe

`~/.kube/config` file gets updated with cluster details and certificate from the below command

$ aws eks --region <enter-your-region> update-kubeconfig --name <cluster-name>
```shell script
aws eks --region <enter-your-region> update-kubeconfig --name <cluster-name>
```

#### Step6: List all the worker nodes by running the command below

$ kubectl get nodes
```shell script
kubectl get nodes
```

#### Step7: List all the pods running in `crossplane` namespace

```shell script
kubectl get pods -n crossplane
```

### AWS Provider for Crossplane
This example shows how to deploy S3 bucket using Crossplane AWS provider

- Open the file below

```shell script
vi ~/examples/crossplane/crossplane-aws-examples/aws-provider-s3.yaml
```
- Edit the below `aws-provider-s3.yaml` to update the new bucket name

- Enter the new `bucket name` and `region` in YAML file. Save the file using :wq!

#### Step7: List all the pods running in `kube-system` namespace
- Apply the K8s manifest

```shell script
cd ~/examples/crossplane/crossplane-aws-examples/
kubectl apply -f aws-provider-s3.yaml
```

$ kubectl get pods -n kube-system
- Login to AWS Console and verify the new S3 bucket

### Deploy S3 bucket using Crossplane
To Delete the bucket
```shell script
cd ~/examples/crossplane/crossplane-aws-examples/
kubectl delete -f aws-provider-s3.yaml
```
### Terrajet AWS Provider for Crossplane
This example shows how to deploy S3 bucket using Crossplane Terrajet AWS Provider

- Edit the `s3.yaml` to update the new bucket name
- Open the file below

```shell script
vi ~/examples/crossplane/crossplane-aws-examples/s3.yaml
vi ~/examples/crossplane/crossplane-aws-examples/jet-aws-provider-s3.yaml
```
Enter the new bucket name and region in YAML file
Save the file using :wq!
- Edit the below `jet-aws-provider-s3.yaml` to update the new bucket name

- Use `kubectl` to apply the `s3.yaml`
- Enter the new `bucket name` and `region` in YAML file. Save the file using :wq!

- Apply the K8s manifest

```shell script
cd ~/examples/crossplane/crossplane-aws-examples/
kubectl apply -f s3.yaml
cd ~/examples/crossplane/crossplane-aws-examples/
kubectl apply -f jet-aws-provider-s3.yaml
```

- Login to AWS Console and verify the new S3 bucket

To Delete the bucket
```shell script
cd ~/examples/crossplane/crossplane-aws-examples/
kubectl delete -f jet-aws-provider-s3.yaml
```

## How to Destroy
The following command destroys the resources created by `terraform apply`
Expand Down
Original file line number Diff line number Diff line change
@@ -1,19 +1,19 @@
apiVersion: s3.aws.crossplane.io/v1beta1
kind: Bucket
metadata:
name: new-bucket
name: sample-xplane-aws-s3-bucket
annotations:
# This will be the actual bucket name. It must be globally unique, so you
# probably want to change it before trying to apply this example.
crossplane.io/external-name: <ENTER_NEW_BUCKET_NAME>
crossplane.io/external-name: "<ENTER_YOUR_NEW_BUCKET_NAME>"
spec:
forProvider:
locationConstraint: <ENTER_YOUR_REGION>
locationConstraint: "<ENTER_YOUR_REGION>" # choose your own region
acl: private
publicAccessBlockConfiguration:
blockPublicAcls: true
blockPublicPolicy: true
ignorePublicAcls: true
restrictPublicBuckets: true
providerConfigRef:
name: default
name: aws-provider-config
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
apiVersion: s3.aws.jet.crossplane.io/v1alpha2
kind: Bucket
metadata:
name: xplane-jet-aws-s3-bucket
annotations:
# This will be the actual bucket name. It must be globally unique, so you
# probably want to change it before trying to apply this example.
crossplane.io/external-name: "<ENTER_YOUR_NEW_BUCKET_NAME>"
spec:
forProvider:
region: "<ENTER_YOUR_REGION>" # choose your own region
acl: private
tags:
Name: "<ENTER_YOUR_NEW_BUCKET_NAME>"
providerConfigRef:
name: jet-aws-provider-config
19 changes: 16 additions & 3 deletions examples/crossplane/main.tf
Original file line number Diff line number Diff line change
Expand Up @@ -143,9 +143,22 @@ module "kubernetes-addons" {
# Refer to docs/add-ons/crossplane.md for advanced configuration
enable_crossplane = true

# Optional config to deploy specific version of AWS Provider and attach additional IAM policies to manage AWS resources using Crossplane
crossplane_provider_aws = {
provider_aws_version = "v0.23.0"
# You can choose to install either of crossplane_aws_provider or crossplane_jet_aws_provider to work with AWS
# Creates ProviderConfig -> aws-provider
vara-bonthu marked this conversation as resolved.
Show resolved Hide resolved
crossplane_aws_provider = {
enable = true
provider_aws_version = "v0.24.1"
# NOTE: Crossplane requires Admin like permissions to create and update resources similar to Terraform deploy role.
# This example config uses AmazonS3FullAccess for demo purpose only, but you should select a policy with the minimum permissions required to provision your resources.
additional_irsa_policies = ["arn:aws:iam::aws:policy/AmazonS3FullAccess"]
}

# Creates ProviderConfig -> jet-aws-provider
crossplane_jet_aws_provider = {
enable = true
provider_aws_version = "v0.4.1"
# NOTE: Crossplane requires Admin like permissions to create and update resources similar to Terraform deploy role.
# This example config uses AmazonS3FullAccess for demo purpose only, but you should select a policy with the minimum permissions required to provision your resources.
additional_irsa_policies = ["arn:aws:iam::aws:policy/AmazonS3FullAccess"]
}
}
4 changes: 2 additions & 2 deletions modules/irsa/main.tf
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ resource "kubernetes_service_account_v1" "irsa" {
}

resource "aws_iam_role" "irsa" {
name = format("%s-%s-%s", var.eks_cluster_id, trim(var.kubernetes_service_account, "*"), "irsa")
name = format("%s-%s-%s", var.eks_cluster_id, trim(var.kubernetes_service_account, "-*"), "irsa")
description = "AWS IAM Role for the Kubernetes service account ${var.kubernetes_service_account}."
assume_role_policy = join("", data.aws_iam_policy_document.irsa_with_oidc.*.json)
path = var.iam_role_path
Expand All @@ -51,7 +51,7 @@ resource "aws_iam_role" "irsa" {

tags = merge(
{
"Name" = "${var.eks_cluster_id}-${var.kubernetes_service_account}-irsa",
"Name" = format("%s-%s-%s", var.eks_cluster_id, trim(var.kubernetes_service_account, "-*"), "irsa"),
"app.kubernetes.io/managed-by" = "terraform-ssp-amazon-eks"
},
var.tags
Expand Down
13 changes: 10 additions & 3 deletions modules/kubernetes-addons/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,9 @@ SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

## Providers

No providers.
| Name | Version |
|------|---------|
| <a name="provider_aws"></a> [aws](#provider\_aws) | >= 3.66.0 |

## Modules

Expand Down Expand Up @@ -63,7 +65,11 @@ No providers.

## Resources

No resources.
| Name | Type |
|------|------|
| [aws_caller_identity.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) | data source |
| [aws_partition.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/partition) | data source |
| [aws_region.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/region) | data source |

## Inputs

Expand Down Expand Up @@ -96,8 +102,9 @@ No resources.
| <a name="input_aws_open_telemetry_addon_config"></a> [aws\_open\_telemetry\_addon\_config](#input\_aws\_open\_telemetry\_addon\_config) | AWS Open Telemetry Distro add-on config | `any` | `{}` | no |
| <a name="input_cert_manager_helm_config"></a> [cert\_manager\_helm\_config](#input\_cert\_manager\_helm\_config) | Cert Manager Helm Chart config | `any` | `{}` | no |
| <a name="input_cluster_autoscaler_helm_config"></a> [cluster\_autoscaler\_helm\_config](#input\_cluster\_autoscaler\_helm\_config) | Cluster Autoscaler Helm Chart config | `any` | `{}` | no |
| <a name="input_crossplane_aws_provider"></a> [crossplane\_aws\_provider](#input\_crossplane\_aws\_provider) | AWS Provider config for Crossplane | <pre>object({<br> enable = bool<br> provider_aws_version = string<br> additional_irsa_policies = list(string)<br> })</pre> | <pre>{<br> "additional_irsa_policies": [],<br> "enable": false,<br> "provider_aws_version": "v0.24.1"<br>}</pre> | no |
| <a name="input_crossplane_helm_config"></a> [crossplane\_helm\_config](#input\_crossplane\_helm\_config) | Crossplane Helm Chart config | `any` | `null` | no |
| <a name="input_crossplane_provider_aws"></a> [crossplane\_provider\_aws](#input\_crossplane\_provider\_aws) | AWS Provider config for Crossplane | <pre>object({<br> provider_aws_version = string<br> additional_irsa_policies = list(string)<br> })</pre> | <pre>{<br> "additional_irsa_policies": [],<br> "provider_aws_version": "v0.23.0"<br>}</pre> | no |
| <a name="input_crossplane_jet_aws_provider"></a> [crossplane\_jet\_aws\_provider](#input\_crossplane\_jet\_aws\_provider) | AWS Provider Jet AWS config for Crossplane | <pre>object({<br> enable = bool<br> provider_aws_version = string<br> additional_irsa_policies = list(string)<br> })</pre> | <pre>{<br> "additional_irsa_policies": [],<br> "enable": false,<br> "provider_aws_version": "v0.24.1"<br>}</pre> | no |
| <a name="input_eks_cluster_id"></a> [eks\_cluster\_id](#input\_eks\_cluster\_id) | EKS Cluster Id | `string` | n/a | yes |
| <a name="input_eks_worker_security_group_id"></a> [eks\_worker\_security\_group\_id](#input\_eks\_worker\_security\_group\_id) | EKS Worker Security group Id created by EKS module | `string` | `""` | no |
| <a name="input_enable_agones"></a> [enable\_agones](#input\_enable\_agones) | Enable Agones GamServer add-on | `bool` | `false` | no |
Expand Down
32 changes: 16 additions & 16 deletions modules/kubernetes-addons/crossplane/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,18 +15,14 @@ Crossplane Add-on can be deployed as follows
enable_crossplane = true
```

AWS Provider for Crossplane will be installed by default with IRSA.
Crossplane requires Admin like permissions to create and update resources similar to Terraform deploy role.
This example uses AdministratorAccess, but you should select a policy with the minimum permissions required to provision your resources.
Please find more details from [AWS Provider](https://github.com/crossplane/provider-aws)
This module allows you to deploy the following AWS providers for Crossplane. These providers disabled by default.

```hcl
crossplane_provider_aws = {
provider_aws_version = "v0.23.0"
additional_irsa_policies = ["arn:aws:iam::aws:policy/AdministratorAccess"]
}
```
- [AWS Provider](https://github.com/crossplane/provider-aws)
- [Provider Jet AWS](https://github.com/crossplane-contrib/provider-jet-aws)

Refer to [docs](../../../docs/add-ons/crossplane.md) on how to deploy AWS Providers.

___
<!--- BEGIN_TF_DOCS --->
## Requirements

Expand All @@ -42,36 +38,40 @@ Please find more details from [AWS Provider](https://github.com/crossplane/provi
| <a name="provider_aws"></a> [aws](#provider\_aws) | n/a |
| <a name="provider_kubectl"></a> [kubectl](#provider\_kubectl) | >= 1.13.1 |
| <a name="provider_kubernetes"></a> [kubernetes](#provider\_kubernetes) | n/a |
| <a name="provider_time"></a> [time](#provider\_time) | n/a |

## Modules

| Name | Source | Version |
|------|--------|---------|
| <a name="module_aws_provider_irsa"></a> [aws\_provider\_irsa](#module\_aws\_provider\_irsa) | ../../../modules/irsa | n/a |
| <a name="module_helm_addon"></a> [helm\_addon](#module\_helm\_addon) | ../helm-addon | n/a |
| <a name="module_jet_aws_provider_irsa"></a> [jet\_aws\_provider\_irsa](#module\_jet\_aws\_provider\_irsa) | ../../../modules/irsa | n/a |

## Resources

| Name | Type |
|------|------|
| [aws_iam_policy.aws_provider](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) | resource |
| [aws_iam_policy.jet_aws_provider](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) | resource |
| [kubectl_manifest.aws_controller_config](https://registry.terraform.io/providers/gavinbunney/kubectl/latest/docs/resources/manifest) | resource |
| [kubectl_manifest.aws_provider](https://registry.terraform.io/providers/gavinbunney/kubectl/latest/docs/resources/manifest) | resource |
| [kubectl_manifest.aws_provider_config](https://registry.terraform.io/providers/gavinbunney/kubectl/latest/docs/resources/manifest) | resource |
| [kubectl_manifest.controller_config](https://registry.terraform.io/providers/gavinbunney/kubectl/latest/docs/resources/manifest) | resource |
| [kubectl_manifest.jet_aws_controller_config](https://registry.terraform.io/providers/gavinbunney/kubectl/latest/docs/resources/manifest) | resource |
| [kubectl_manifest.jet_aws_provider](https://registry.terraform.io/providers/gavinbunney/kubectl/latest/docs/resources/manifest) | resource |
| [kubectl_manifest.jet_aws_provider_config](https://registry.terraform.io/providers/gavinbunney/kubectl/latest/docs/resources/manifest) | resource |
| [kubernetes_namespace_v1.crossplane](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/resources/namespace_v1) | resource |
| [time_sleep.wait_30_seconds](https://registry.terraform.io/providers/hashicorp/time/latest/docs/resources/sleep) | resource |
| [aws_caller_identity.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) | data source |
| [aws_iam_policy_document.s3_policy](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
| [aws_partition.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/partition) | data source |

## Inputs

| Name | Description | Type | Default | Required |
|------|-------------|------|---------|:--------:|
| <a name="input_crossplane_provider_aws"></a> [crossplane\_provider\_aws](#input\_crossplane\_provider\_aws) | AWS Provider config for Crossplane | <pre>object({<br> provider_aws_version = string<br> additional_irsa_policies = list(string)<br> })</pre> | n/a | yes |
| <a name="input_account_id"></a> [account\_id](#input\_account\_id) | Current AWS Account ID | `string` | n/a | yes |
| <a name="input_aws_partition"></a> [aws\_partition](#input\_aws\_partition) | AWS Identifier of the current partition e.g., aws or aws-cn | `string` | n/a | yes |
| <a name="input_aws_provider"></a> [aws\_provider](#input\_aws\_provider) | AWS Provider config for Crossplane | <pre>object({<br> enable = bool<br> provider_aws_version = string<br> additional_irsa_policies = list(string)<br> })</pre> | n/a | yes |
| <a name="input_eks_cluster_id"></a> [eks\_cluster\_id](#input\_eks\_cluster\_id) | EKS cluster Id | `string` | n/a | yes |
| <a name="input_helm_config"></a> [helm\_config](#input\_helm\_config) | Helm provider config for the Argo Rollouts | `any` | `{}` | no |
| <a name="input_jet_aws_provider"></a> [jet\_aws\_provider](#input\_jet\_aws\_provider) | AWS Provider Jet AWS config for Crossplane | <pre>object({<br> enable = bool<br> provider_aws_version = string<br> additional_irsa_policies = list(string)<br> })</pre> | n/a | yes |
| <a name="input_manage_via_gitops"></a> [manage\_via\_gitops](#input\_manage\_via\_gitops) | Determines if the add-on should be managed via GitOps. | `bool` | `false` | no |
| <a name="input_tags"></a> [tags](#input\_tags) | Common Tags for AWS resources | `map(string)` | `{}` | no |

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
apiVersion: pkg.crossplane.io/v1alpha1
kind: ControllerConfig
metadata:
name: aws-config
name: aws-controller-config
annotations:
eks.amazonaws.com/role-arn: ${iam-role-arn}
spec:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
apiVersion: aws.crossplane.io/v1beta1
kind: ProviderConfig
metadata:
name: default
name: aws-provider-config
spec:
credentials:
source: InjectedIdentity
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,8 @@
apiVersion: pkg.crossplane.io/v1
kind: Provider
metadata:
name: provider-aws
name: ${aws-provider-name}
spec:
package: crossplane/provider-aws:${provider-aws-version}
controllerConfigRef:
name: aws-config
name: aws-controller-config
Loading