Skip to content

Commit

Permalink
feat: Add transparent encryption with cilium and wiregaurd (aws-ia#1130)
Browse files Browse the repository at this point in the history 
Co-authored-by: Bryant Biggs <[email protected]>
  • Loading branch information
@askulkarni2 @bryantbiggs @allamand
2 people authored and allamand committed Dec 15, 2022
1 parent 6497a40 commit 4044e82
Show file tree
Hide file tree
Showing 11 changed files with 429 additions and 1 deletion.
127 changes: 127 additions & 0 deletions examples/wireguard-with-cilium/README.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,127 @@
# Transparent Encryption with Cilium and Wirguard

This example shows how to provision an EKS cluster with:
- Managed node group based on Bottlerocket AMI
- Cilium configured in CNI chaining mode with VPC CNI and with Wiregaurd transparent encryption enabled

## Reference Documentation:

- [Cilium CNI Chaning Documentation](https://docs.cilium.io/en/v1.12/gettingstarted/cni-chaining-aws-cni/)
- [Cilium Wiregaurd Encryption Documentation](https://docs.cilium.io/en/v1.12/gettingstarted/encryption-wireguard/)

## Prerequisites:

Ensure that you have the following tools installed locally:

1. [aws cli](https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html)
2. [kubectl](https://Kubernetes.io/docs/tasks/tools/)
3. [terraform](https://learn.hashicorp.com/tutorials/terraform/install-cli)

## Deploy

To provision this example with a sample app for testing:

```sh
terraform init
terraform apply
```

To provision this example without sample app for testing:

```sh
terraform init
terraform apply -var enable_example=false
```

Enter `yes` at command prompt to apply

## Validate

The following command will update the `kubeconfig` on your local machine and allow you to interact with your EKS Cluster using `kubectl` to validate the deployment.

1. Run `update-kubeconfig` command:

```sh
aws eks --region <REGION> update-kubeconfig --name <CLUSTER_NAME>
```

2. List the daemonsets

```sh
kubectl get ds -n kube-system

# Output should look something similar
NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
aws-node 2 2 2 2 2 <none> 156m
cilium 2 2 2 2 2 kubernetes.io/os=linux 152m
kube-proxy 2 2 2 2 2 <none> 156m
```

3. Open a shell inside the cilium container

```sh
kubectl -n kube-system exec -ti ds/cilium -- bash
```

4. Verify Encryption is enabled

```sh
cilium status | grep Encryption

# Output should look something similar
Encryption: Wireguard [cilium_wg0 (Pubkey: b2krgbHgaCsVWALMnFLiS/RekhhcE36PXEjQ7T8+mW0=, Port: 51871, Peers: 1)]
```

5. Install tcpdump

```sh
apt-get update
apt-get install -y tcpdump
```

6. Start a packet capture and verify you don't see payload in clear text

```sh
tcpdump -A -c 3 -i cilium_wg0

# Output should look similar below (truncated for brevity)

tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on cilium_wg0, link-type RAW (Raw IP), capture size 262144 bytes
05:28:30.234209 IP ip-10-0-11-73.ec2.internal.58086 > ip-10-0-10-160.ec2.internal.http: Flags [S], seq 2831772984, win 62727, options [mss 8961,sackOK,TS val 3834644316 ecr 0,nop,wscale 7], length 0
E..<].@.?...
..I
.
....P..m8........&.....#....
...\........
05:28:30.234306 IP ip-10-0-10-160.ec2.internal.http > ip-10-0-11-73.ec2.internal.58086: Flags [S.], seq 131501951, ack 2831772985, win 62643, options [mss 8961,sackOK,TS val 1959385110 ecr 3834644316,nop,wscale 7], length 0
E..<..@.?...
.
.
..I.P........m9....*.....#....
t......\....
05:28:30.234930 IP ip-10-0-11-73.ec2.internal.58086 > ip-10-0-10-160.ec2.internal.http: Flags [.], ack 1, win 491, options [nop,nop,TS val 3834644317 ecr 1959385110], length 0
E..4].@.?...
..I
.
....P..m9...............
...]t...
3 packets captured
9 packets received by filter
1 packet dropped by kernel
```
7. Exit the container shell

```sh
exit
```

## Destroy

To teardown and remove the resources created in this example:

```sh
terraform destroy -target=module.eks_blueprints_kubernetes_addons -auto-approve
terraform destroy -target=module.eks_blueprints -auto-approve
terraform destroy -auto-approve
```
240 changes: 240 additions & 0 deletions examples/wireguard-with-cilium/main.tf
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,240 @@
provider "aws" {
region = local.region
}

provider "kubernetes" {
host = module.eks_blueprints.eks_cluster_endpoint
cluster_ca_certificate = base64decode(module.eks_blueprints.eks_cluster_certificate_authority_data)
token = data.aws_eks_cluster_auth.this.token
}

provider "helm" {
kubernetes {
host = module.eks_blueprints.eks_cluster_endpoint
cluster_ca_certificate = base64decode(module.eks_blueprints.eks_cluster_certificate_authority_data)
token = data.aws_eks_cluster_auth.this.token
}
}

provider "kubectl" {
apply_retry_count = 10
host = module.eks_blueprints.eks_cluster_endpoint
cluster_ca_certificate = base64decode(module.eks_blueprints.eks_cluster_certificate_authority_data)
load_config_file = false
token = data.aws_eks_cluster_auth.this.token
}

data "aws_eks_cluster_auth" "this" {
name = module.eks_blueprints.eks_cluster_id
}

data "aws_availability_zones" "available" {}

locals {
name = basename(path.cwd)
# var.cluster_name is for Terratest
cluster_name = local.name
cluster_version = "1.23"
region = "us-west-2"

vpc_cidr = "10.0.0.0/16"
azs = slice(data.aws_availability_zones.available.names, 0, 3)

tags = {
Blueprint = local.name
GithubRepo = "github.com/aws-ia/terraform-aws-eks-blueprints"
}
}

#---------------------------------------------------------------
# EKS Blueprints
#---------------------------------------------------------------

module "eks_blueprints" {
source = "../.."

cluster_name = local.cluster_name
cluster_version = local.cluster_version

vpc_id = module.vpc.vpc_id
private_subnet_ids = module.vpc.private_subnets

managed_node_groups = {
# BottleRocket ships with kernel 5.10 so there is no need
# to do anything special
bottlerocket = {
node_group_name = "mg5"
instance_types = ["m5.large"]
min_size = 2
desired_size = 2
max_size = 2
ami_type = "BOTTLEROCKET_x86_64"
subnet_ids = module.vpc.private_subnets
}
}

tags = local.tags
}

module "eks_blueprints_kubernetes_addons" {
source = "../../modules/kubernetes-addons"

eks_cluster_id = module.eks_blueprints.eks_cluster_id
eks_cluster_endpoint = module.eks_blueprints.eks_cluster_endpoint
eks_oidc_provider = module.eks_blueprints.oidc_provider
eks_cluster_version = module.eks_blueprints.eks_cluster_version

# Wait on the `kube-system` profile before provisioning addons
data_plane_wait_arn = module.eks_blueprints.managed_node_group_arn[0]

# Add-ons
enable_cilium = true
cilium_enable_wireguard = true

tags = local.tags
}

#---------------------------------------------------------------
# Supporting Resources
#---------------------------------------------------------------

module "vpc" {
source = "terraform-aws-modules/vpc/aws"
version = "~> 3.0"

name = local.name
cidr = local.vpc_cidr

azs = local.azs
public_subnets = [for k, v in local.azs : cidrsubnet(local.vpc_cidr, 8, k)]
private_subnets = [for k, v in local.azs : cidrsubnet(local.vpc_cidr, 8, k + 10)]

enable_nat_gateway = true
single_nat_gateway = true
enable_dns_hostnames = true

# Manage so we can name
manage_default_network_acl = true
default_network_acl_tags = { Name = "${local.name}-default" }
manage_default_route_table = true
default_route_table_tags = { Name = "${local.name}-default" }
manage_default_security_group = true
default_security_group_tags = { Name = "${local.name}-default" }

public_subnet_tags = {
"kubernetes.io/role/elb" = 1
}

private_subnet_tags = {
"kubernetes.io/role/internal-elb" = 1
}

tags = local.tags
}

#---------------------------------------------------------------
# Sample App for Testing
#---------------------------------------------------------------

resource "kubectl_manifest" "server" {
count = var.enable_example ? 1 : 0

yaml_body = yamlencode({
apiVersion = "v1"
kind = "Pod"
metadata = {
name = "server"
labels = {
blog = "wireguard"
name = "server"
}
}
spec = {
containers = [
{
name = "server"
image = "nginx"
}
]
topologySpreadConstraints = [
{
maxSkew = 1
topologyKey = "kubernetes.io/hostname"
whenUnsatisfiable = "DoNotSchedule"
labelSelector = {
matchLabels = {
blog = "wireguard"
}
}
}
]
}
})

depends_on = [
module.eks_blueprints_kubernetes_addons
]
}

resource "kubectl_manifest" "service" {
count = var.enable_example ? 1 : 0

yaml_body = yamlencode({
apiVersion = "v1"
kind = "Service"
metadata = {
name = "server"
}
spec = {
selector = {
name = "server"
}
ports = [
{
port = 80
}
]
}
})
}

resource "kubectl_manifest" "client" {
count = var.enable_example ? 1 : 0

yaml_body = yamlencode({
apiVersion = "v1"
kind = "Pod"
metadata = {
name = "client"
labels = {
blog = "wireguard"
name = "client"
}
}
spec = {
containers = [
{
name = "client"
image = "busybox"
command = ["watch", "wget", "server"]
}
]
topologySpreadConstraints = [
{
maxSkew = 1
topologyKey = "kubernetes.io/hostname"
whenUnsatisfiable = "DoNotSchedule"
labelSelector = {
matchLabels = {
blog = "wireguard"
}
}
}
]
}
})

depends_on = [
kubectl_manifest.server[0]
]
}
4 changes: 4 additions & 0 deletions examples/wireguard-with-cilium/outputs.tf
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
output "configure_kubectl" {
description = "Configure kubectl: make sure you're logged in with the correct AWS profile and run the following command to update your kubeconfig"
value = module.eks_blueprints.configure_kubectl
}
6 changes: 6 additions & 0 deletions examples/wireguard-with-cilium/variables.tf
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# tflint-ignore: terraform_unused_declarations
variable "enable_example" {
description = "Enable example to test this blueprint"
type = bool
default = true
}
Loading

0 comments on commit 4044e82

Please sign in to comment.