Add MWAA module and example (#585)

Co-authored-by: Bryant Biggs <[email protected]>

README.md

@@ -220,6 +220,7 @@ If you are interested in contributing to EKS Blueprints, see the [Contribution g
 | <a name="output_cluster_security_group_arn"></a> [cluster\_security\_group\_arn](#output\_cluster\_security\_group\_arn) | Amazon Resource Name (ARN) of the cluster security group |
 | <a name="output_cluster_security_group_id"></a> [cluster\_security\_group\_id](#output\_cluster\_security\_group\_id) | EKS Control Plane Security Group ID |
 | <a name="output_configure_kubectl"></a> [configure\_kubectl](#output\_configure\_kubectl) | Configure kubectl: make sure you're logged in with the correct AWS profile and run the following command to update your kubeconfig |
+| <a name="output_eks_cluster_arn"></a> [eks\_cluster\_arn](#output\_eks\_cluster\_arn) | Amazon EKS Cluster Name |
 | <a name="output_eks_cluster_certificate_authority_data"></a> [eks\_cluster\_certificate\_authority\_data](#output\_eks\_cluster\_certificate\_authority\_data) | Base64 encoded certificate data required to communicate with the cluster |
 | <a name="output_eks_cluster_endpoint"></a> [eks\_cluster\_endpoint](#output\_eks\_cluster\_endpoint) | Endpoint for your Kubernetes API server |
 | <a name="output_eks_cluster_id"></a> [eks\_cluster\_id](#output\_eks\_cluster\_id) | Amazon EKS Cluster Name |

examples/managed-workflow-apache-airflow/README.md

@@ -0,0 +1,88 @@
+## EKS Cluster w/ Amazon Manged Workflows for Apache Airflopw (MWAA)
+
+The example demonstrates how to use Amazon Managed Workflows for Apache Airflow (MWAA) with Amazon EKS.
+
+This example was originated from the steps provided on MWAA documentation on the link below:
+[mwaa-eks-example](https://docs.aws.amazon.com/mwaa/latest/userguide/mwaa-eks-example.html)
+
+### Considerations
+
+1. Ideally we recommend adding the steps to sync requirements/sync dags to the MWAA S3 Bucket as part of a CI/CD pipeline. Generally Dags development have a different lifecycle than the Terraform code to provision infrastructure.
+However for simplicity we are providing steps for that using Terraform running AWS CLI commands on null_resource.
+
+## Prerequisites:
+
+Ensure that you have the following tools installed locally:
+
+1. [aws cli](https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html)
+2. [kubectl](https://Kubernetes.io/docs/tasks/tools/)
+3. [terraform](https://learn.hashicorp.com/tutorials/terraform/install-cli)
+
+## Deploy
+
+To provision this example:
+
+```sh
+terraform init
+terraform apply
+```
+
+Enter `yes` at command prompt to apply
+
+
+## Validate
+
+The following command will update the `kubeconfig` on your local machine and allow you to interact with your EKS Cluster using `kubectl` to validate the deployment.
+
+1. Run `update-kubeconfig` command:
+
+```sh
+aws eks --region <REGION> update-kubeconfig --name <CLUSTER_NAME>
+```
+
+2. List the nodes running currently
+
+```sh
+kubectl get nodes
+
+# Output should look like below
+NAME                                        STATUS                        ROLES    AGE     VERSION
+ip-10-0-30-125.us-west-2.compute.internal   Ready                         <none>   2m19s   v1.22.9-eks-810597c
+```
+
+3. Log into Apache Airflow UI
+
+- Open the Environments page on the Amazon MWAA console
+- Choose an environment
+- Under the `Details` section, click the link for the Airflow UI
+
+4. Triger the DAG workflow to execute
+
+In the Airflow UI, enable the example and then trigger it.
+
+![Enable the DAG kubernetes_pod_example ](images/kubernetes_pod_example_dag.png)
+
+![Trigger the DAG kubernetes_pod_example ](images/dag_tree.png)
+
+5. Verify that the pod was executed successfully
+
+After it runs and completes successfully, use the following command to verify the pod:
+
+```sh
+kubectl get pods -n mwaa
+```
+
+You should see output similar to the following:
+
+```sh
+NAME                                             READY   STATUS      RESTARTS   AGE
+mwaa-pod-test.4bed823d645844bc8e6899fd858f119d   0/1     Completed   0          25s
+```
+
+## Destroy
+
+To teardown and remove the resources created in this example:
+
+```sh
+terraform destroy -auto-approve
+```

examples/managed-workflow-apache-airflow/dags/mwaa_pod_example.py

@@ -0,0 +1,33 @@
+from airflow import DAG
+from datetime import datetime
+from airflow.providers.cncf.kubernetes.operators.kubernetes_pod import (
+    KubernetesPodOperator,
+)
+
+default_args = {
+    "owner": "aws",
+    "depends_on_past": False,
+    "start_date": datetime(2019, 2, 20),
+    "provide_context": True,
+}
+
+dag = DAG("kubernetes_pod_example", default_args=default_args, schedule_interval=None)
+
+# use a kube_config stored in s3 dags folder for now
+kube_config_path = "/usr/local/airflow/dags/kube_config.yaml"
+
+podRun = KubernetesPodOperator(
+    namespace="mwaa",
+    image="ubuntu:18.04",
+    cmds=["bash"],
+    arguments=["-c", "ls"],
+    labels={"foo": "bar"},
+    name="mwaa-pod-test",
+    task_id="pod-task",
+    get_logs=True,
+    dag=dag,
+    is_delete_operator_pod=False,
+    config_file=kube_config_path,
+    in_cluster=False,
+    cluster_context="mwaa", # Must match kubeconfig context
+)

examples/managed-workflow-apache-airflow/dags/requirements.txt

@@ -0,0 +1,2 @@
+kubernetes==11.0.0
+apache-airflow-providers-cncf-kubernetes==2.0.2