-
Notifications
You must be signed in to change notification settings - Fork 671
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
* update Opta AWS doc Signed-off-by: Samhita Alla <[email protected]> * nit Signed-off-by: Samhita Alla <[email protected]> * add iam content Signed-off-by: Samhita Alla <[email protected]>
- Loading branch information
1 parent
992c466
commit 46c059e
Showing
1 changed file
with
106 additions
and
84 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,130 +1,152 @@ | ||
| .. _deployment-aws-opta: | ||
|
|
||
| AWS (EKS) Automated Setup with Opta | ||
| ----------------------------------- | ||
| AWS (EKS) Automated Setup With Opta | ||
| =================================== | ||
|
|
||
| In order to handle production load robustly, securely and with high availability, there are a number of important tasks that need to | ||
| be done independently from the sandbox deployment: | ||
| Several essential tasks need to be taken care of independently from the sandbox deployment to achieve high availability and handle production | ||
| load robustly and securely. | ||
|
|
||
| * The kubernetes cluster needs to run securely and robustly | ||
| * The sandbox's object store must be replaced by a production grade storage system | ||
| * The sandbox's PostgreSQL database must be replaced by a production grade deployment of postgres | ||
| * A production grade task queueing system must be provisioned and configured | ||
| * A production grade notification system must be provisioned and configured | ||
| * All the above must be done in a secure fashion | ||
| * (Optionally) An official dns domain must be created | ||
| * (Optionally) A production grade email sending system must be provisioned and configured | ||
| * The Kubernetes cluster needs to run securely and robustly | ||
| * The sandbox's object store must be replaced with a production-grade storage system | ||
| * The sandbox's PostgreSQL database must be replaced with a production-grade deployment of PostgreSQL | ||
| * A production-grade task queueing system must be provisioned and configured | ||
| * A production-grade notification system must be provisioned and configured | ||
| * All the above have to be done in a secure manner | ||
| * (Optionally) An official DNS domain must be created | ||
| * (Optionally) A production-grade email sending system must be provisioned and configured | ||
|
|
||
| A Flyte user may provision and orchestrate this setup by themselves, but the Flyte team has partnered with the | ||
| `Opta <https://github.com/run-x/opta>`_ team to create a streamlined production deployment strategy for AWS with | ||
| ready-to-use templates provided in the `Flyte repo <https://github.com/flyteorg/flyte/tree/master/opta/aws>`_. The following demo and documentation specifies how to use and further configure them. | ||
| ready-to-use templates provided in the `Flyte repo <https://github.com/flyteorg/flyte/tree/master/opta/aws>`__. | ||
|
|
||
| The following demo and documentation specify how to use, and further configure them. | ||
|
|
||
| .. youtube:: CMp04-mdtQQ | ||
|
|
||
| Deploying Opta Environment and Service for Flyte | ||
| ************************************************ | ||
| **The Environment** | ||
| To begin using Opta, please first `download the latest version <https://docs.opta.dev/installation/>`_ and all the listed | ||
| ------------------------------------------------ | ||
|
|
||
| 1. Environment | ||
| ************** | ||
|
|
||
| To begin using Opta, `download the latest version <https://docs.opta.dev/installation/>`__ and all the listed | ||
| prerequisites and make sure that you have | ||
| `admin/fullwrite AWS credentials setup on your terminal <https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-envvars.html>`_. | ||
| With that prepared, go to the `Opta AWS subdirectory <https://github.com/flyteorg/flyte/tree/master/opta/aws>`_ in the Flyte repo, and open up env.yaml in your editor. Please find and | ||
| replace the following values with your desired ones: | ||
| `admin/fullwrite AWS credentials setup on your terminal <https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-envvars.html>`__. | ||
| With that prepared, | ||
|
|
||
| * <account_id>: your AWS account ID | ||
| * <region>: your AWS region | ||
| * <domain>: your desired domain for your Flyte deployment (should be a domain which you own or a subdomain thereof - this environment will promptyly take ownership of the domain/subdomain so make sure it will only be used for this purpose) | ||
| * <env_name>: a name for the new isolated cloud environment which is going to be created (e.g. flyte-prod) | ||
| * <your_company>: your company or organization's name | ||
| .. _opta-aws-directory: | ||
|
|
||
| Once complete please run ``opta apply -c env.yaml`` and follow the prompts. | ||
| * Clone Flyte repo: ``git clone [email protected]:flyteorg/flyte.git`` | ||
| * Go to the ``flyte/opta/aws`` directory | ||
| * Open the ``env.yaml`` file in your editor and replace the following values with your desired values: | ||
|
|
||
| **DNS Delegation** | ||
| Once Opta's apply for the environment is completed, you will need to complete dns delegation to fully setup public | ||
| traffic access. You may find instructions on `how to do so here <https://docs.opta.dev/tutorials/ingress/>`__. | ||
| * <account_id>: your AWS account ID | ||
| * <region>: your AWS region | ||
| * <domain>: your desired domain for your Flyte deployment (should be a domain which you own or a subdomain thereof - this environment will promptly take ownership of the domain/subdomain, so make sure it will only be used for this purpose) | ||
| * <env_name>: a name for the new isolated cloud environment which is going to be created (e.g., flyte-prod) | ||
| * <your_company>: your company or organization's name | ||
|
|
||
| **The Flyte Deployment** | ||
| Once dns deployment delegation is complete, you may deploy the Flyte service and affiliated resources. Go to the Opta | ||
| subdirectory in the Flyte repo, and open up flyte.yaml in your editor. Please find and replace the following values with | ||
| your desired ones: | ||
| Once complete, run ``opta apply -c env.yaml`` and follow the prompts. | ||
|
|
||
| 2. DNS Delegation | ||
| ***************** | ||
| Next, you will need to complete DNS delegation to set up public | ||
| traffic access fully. You may find instructions on how to do so `here <https://docs.opta.dev/tutorials/ingress/>`__. | ||
|
|
||
| 3. Flyte Deployment | ||
| ******************* | ||
| Once DNS deployment delegation is complete, you may deploy the Flyte service and affiliated resources. | ||
| Open ``flyte.yaml`` present in ``flyte/opta/aws`` in your editor. | ||
| Replace the following values with your desired values: | ||
|
|
||
| * <account_id>: your AWS account ID | ||
| * <region>: your AWS region | ||
|
|
||
| Once complete please run ``opta apply -c flyte.yaml`` and follow the prompts. | ||
| Once complete, run ``opta apply -c flyte.yaml`` and follow the prompts. | ||
|
|
||
| Understanding the Opta Yamls | ||
| **************************** | ||
| The Opta yaml files | ||
| Understanding the Opta YAMLs | ||
| ---------------------------- | ||
|
|
||
| **Production Grade Environment** | ||
| The Opta env.yaml is responsible for setting up the base infrastructure necessary for most cloud resources. The base | ||
| module sets up the VPC and subnets (both public and private) used by the environment as well as the shared KMS keys. | ||
| The dns sets up the hosted zone for domain and ssl certificates once completed. The k8s-cluster creates the | ||
| Kubernetes cluster and node pool (with encrypted disk storage). And lastly the k8s-base module sets up the resources | ||
| Production-grade Environment | ||
| **************************** | ||
| The Opta ``env.yaml`` is responsible for setting up the base infrastructure necessary for most cloud resources. The base | ||
| module sets up the VPC and subnets (both public and private) used by the environment and the shared KMS keys. | ||
| The DNS sets up the hosted zone for domain and SSL certificates. The k8s-cluster creates the | ||
| Kubernetes cluster and node pool (with encrypted disk storage). And lastly, the k8s-base module sets up the resources | ||
| within Kubernetes like the autoscaler, metrics server, and ingress. | ||
|
|
||
| **Production Grade Database** | ||
| The aws-postgres module in flyte.yaml creates an Aurora Postgresql database with disk encryption and regular snapshot | ||
| backups. You can read more about it `here <https://docs.opta.dev/modules-reference/service-modules/aws/#postgres>`__ | ||
|
|
||
| **Production Grade Object Store** | ||
| The aws-s3 module in flyte.yaml creates a new S3 bucket for Flyte, including disk encryption. You can read more about it | ||
| `here <https://docs.opta.dev/modules-reference/service-modules/aws/#aws-s3>`__ | ||
|
|
||
| **Production Grade Notification System** | ||
| Flyte uses a combination of the AWS Simple Notification Service (SNS) and Simple Queueing service for a notification | ||
| system. flyte.yaml creates both the SNS topic and SQS queue (via the notifcationsQueue and topic modules), which are | ||
| encrypted with unique KMS keys and only the flyte roles can access them. You can read more about the queues | ||
| Production-grade Database | ||
| ************************* | ||
| The aws-postgres module in ``flyte.yaml`` creates an Aurora PostgreSQL database with disk encryption and regular snapshot | ||
| backups. You can read more about it `here <https://docs.opta.dev/modules-reference/service-modules/aws/#postgres>`__. | ||
|
|
||
| Production-grade Object Store | ||
| ***************************** | ||
| The aws-s3 module in ``flyte.yaml`` creates a new S3 bucket for Flyte, including disk encryption. You can read more about it | ||
| `here <https://docs.opta.dev/modules-reference/service-modules/aws/#aws-s3>`__. | ||
|
|
||
| Production-grade Notification System | ||
| ************************************ | ||
| Flyte uses a combination of the AWS' Simple Notification Service (SNS) and Simple Queue Service (SQS) for the notification | ||
| system. ``flyte.yaml`` creates both the SNS topic and SQS queue (via the notifcationsQueue and topic modules), which are | ||
| encrypted with unique KMS keys and only the Flyte roles can access them. You can read more about the queues | ||
| `here <https://docs.opta.dev/modules-reference/service-modules/aws/#aws-sqs>`__ and the topics | ||
| `here <https://docs.opta.dev/modules-reference/service-modules/aws/#aws-sns>`__. | ||
|
|
||
| **Production Grade Queueing System** | ||
| Flyte uses SQS to power its task scheduling system, and flyte.yaml creates said queue (via the schedulesQueue | ||
| module) with encryption and principle of least privilege rbac access like the other SQS queue above. | ||
|
|
||
| **Secure IAM Roles for Data and Control Planes** | ||
| Production-grade Queueing System | ||
| ******************************** | ||
| Flyte uses SQS to power its task scheduling system, and ``flyte.yaml`` creates said queue (via the schedulesQueue | ||
| module) with encryption and principle of least privilege RBAC access like the SQS queue mentioned above. | ||
|
|
||
|
|
||
| **Flyte Deployment via Helm** | ||
| A Flyte deployment contains around 50 kubernetes resources. | ||
| Secure IAM Roles for Data and Control Planes | ||
| ******************************************** | ||
| The aws-iam-role module in ``flyte.yaml`` creates IAM roles for the data and control planes of the Flyte service. You can read more about it | ||
| `here <https://docs.opta.dev/reference/aws/service_modules/aws-iam-role/>`__. | ||
|
|
||
| Additional Setup | ||
| **************** | ||
| By now you should be set up for most production deployments, but there are some extra steps which we recommend that | ||
| ---------------- | ||
|
|
||
| By now, you should be set up for most production deployments, but there are some extra steps that we recommend that | ||
| most users consider. | ||
|
|
||
| **Email Setup** | ||
| Email Setup | ||
| *********** | ||
|
|
||
| Flyte has the power to send email notifications, which can be enabled in Opta via | ||
| `AWS' Simple Email Service <https://aws.amazon.com/ses/>`_ with a few extra steps (NOTE: make sure to have completed dns | ||
| `AWS' Simple Email Service <https://aws.amazon.com/ses/>`__ with a few extra steps (NOTE: make sure to have completed DNS | ||
| delegation first): | ||
| 1. Simply go to env.yaml and uncomment out the last line ( `- type: aws-ses` ) | ||
|
|
||
| 2. Run ``opta apply -c env.yaml`` again | ||
| 1. Go to ``env.yaml`` and uncomment the last line ( `- type: aws-ses` ) | ||
| 2. Run ``opta apply -c env.yaml`` (again) | ||
|
|
||
| This will enable SES on your account and environment domain -- you may be prompted to fill in some user-specific input to take your account out of SES sandbox if not done already. | ||
| It may take a day for AWS to enable production SES on your account (you will be kept notified via email addresses inputted on the user | ||
| prompt) but that should not prevent you from moving forward. | ||
|
|
||
| This will enable SES on your account and environment domain -- you may be prompted to fill in some user-specific input to take your account out of SES sandbox if not done already. | ||
| It may take a day for AWS to enable production SES on your account (you will be kept notified via the email addresses inputted on the user | ||
| prompt) but that should not prevent you from moving forward. | ||
| 3. Lastly, go ahead and uncomment the 'Uncomment out for SES' line in the ``flyte.yaml`` and rerun ``opta apply -c flyte.yaml``. | ||
|
|
||
| 3. Lastly, go ahead and uncomment out the 'Uncomment out for SES' line in the flyte.yaml and rerun ``opta apply -c flyte.yaml``. | ||
| You will now be able to receive emails sent by Flyte as soon as AWS approves your account. You may also specify other | ||
| non-default email senders via the Heml chart values. | ||
|
|
||
| You will now be able to receive emails sent by Flyte as soon as AWS approves your account. You may also specify other | ||
| non-default email senders via the helm chart values. | ||
| Flyte RBAC | ||
| ********** | ||
|
|
||
| **Flyte Rbac** | ||
| All Flyte deployments are currently insecure on the application level by default (e.g. open/accessible to everyone) so it | ||
| is strongly recommended that users `add authentication <https://docs.flyte.org/projects/cookbook/en/latest/auto/deployment/cluster/auth_setup.html#authentication-setup>`_. | ||
| All Flyte deployments are currently insecure at the application level by default (e.g., open/accessible to everyone), | ||
| so we strongly recommend users to add :ref:`add authentication <deployment-cluster-config-auth-setup>`. | ||
|
|
||
| **Extra configuration** | ||
| It is possible to add extra configuration to your Flyte deployment by modifying the values passed in the helm chart | ||
| used by Opta. Please refer to the possible values allowed from the `Flyte helm chart <https://github.com/flyteorg/flyte/tree/master/charts/flyte>`_ | ||
| and update the values field of the Flyte module in the flyte.yaml file accordingly. | ||
| Extra Configuration | ||
| ******************* | ||
|
|
||
| It is possible to add extra configuration to your Flyte deployment by modifying the values passed in the Helm chart | ||
| used by Opta. Refer to the possible values allowed in `Flyte Helm chart <https://github.com/flyteorg/flyte/tree/master/charts/flyte>`__ | ||
| and update the values field of Flyte module in the ``flyte.yaml`` file accordingly. | ||
|
|
||
|
|
||
| Raw Helm Deployment | ||
| ******************* | ||
| It is certainly possible to deploy a production Flyte cluster directly using the helm chart if a user does not wish to | ||
| use Opta. To do so properly, one will need to ensure they have completed the initial security/ha/robustness checklist | ||
| from above, and then use `helm <https://helm.sh/>`_ to deploy the `Flyte helm chart <https://github.com/flyteorg/flyte/tree/master/charts/flyte>`_. | ||
| ------------------- | ||
| It is certainly possible to deploy a production Flyte cluster directly using Helm chart if a user does not wish to | ||
| use Opta. To do so properly, one will need to ensure they have completed the initial security/high-availability/robustness checklist, | ||
| and then use `Helm <https://helm.sh/>`__ to deploy `Flyte Helm chart <https://github.com/flyteorg/flyte/tree/master/charts/flyte>`__. | ||
|
|
||
| .. role:: raw-html-m2r(raw) | ||
| :format: html |