From 110eb7059bbc82f073861457ed48718a3931fb3c Mon Sep 17 00:00:00 2001 From: Yuvraj Date: Tue, 31 Aug 2021 19:51:08 +0530 Subject: [PATCH] Added docs for helm and plugin Signed-off-by: Yuvraj --- rsts/deployment/aws/manual.rst | 23 ++++---- rsts/deployment/index.rst | 7 +++ rsts/deployment/plugin_setup/index.rst | 33 +++++++++++ rsts/deployment/plugin_setup/mpi_operator.rst | 57 +++++++++++++++++++ 4 files changed, 108 insertions(+), 12 deletions(-) create mode 100644 rsts/deployment/plugin_setup/index.rst create mode 100644 rsts/deployment/plugin_setup/mpi_operator.rst diff --git a/rsts/deployment/aws/manual.rst b/rsts/deployment/aws/manual.rst index 5e6f871f47e..d71c51a48d5 100644 --- a/rsts/deployment/aws/manual.rst +++ b/rsts/deployment/aws/manual.rst @@ -407,11 +407,11 @@ Time for Helm Installing Flyte ----------------- -#. Clone the Flyte repo +#. Add helm registry for flyte .. code-block:: bash - git clone https://github.com/flyteorg/flyte + helm repo add flyteorg https://flyteorg.github.io/flyte #. Update values @@ -446,19 +446,12 @@ Search and replace the following - ARN of the self-signed (or official) certificate - ``arn:aws:acm:us-east-2:173113148371:certificate/763d12d5-490d-4e1e-a4cc-4b28d143c2b4`` -#. Update helm dependencies - -.. code-block:: bash - - helm dep update - - #. Install Flyte .. code-block:: bash cd helm - helm install -n flyte -f values-eks.yaml --create-namespace flyte . + helm install -n flyte -f values-eks.yaml --create-namespace flyte flyteorg/flyte-core #. Verify all the pods have come up correctly @@ -479,7 +472,7 @@ Upgrading Flyte .. code-block:: bash - helm upgrade -n flyte -f values-eks.yaml --create-namespace flyte . + helm upgrade -n flyte -f values-eks.yaml --create-namespace flyte flyteorg/flyte-core Connecting to Flyte =================== @@ -501,7 +494,13 @@ Flyte can be accessed using the UI console or your terminal * Connecting to flytectl CLI -Add : to ~/.flyte/config.yaml eg ; +Generate flytectl config for your flyte cluster. Use `--insecure` flag for self-signed cert; + +.. code-block:: bash + # Generate config for production cluster + flytectl config init --host= --storage + +Manually update the `insecureSkipVerify: true` config in `~/.flyte/config.yaml` (only required if using a self-signed cert. Caution: not to be used in production) .. code-block:: diff --git a/rsts/deployment/index.rst b/rsts/deployment/index.rst index e4bc234844d..9810b5a7a0b 100644 --- a/rsts/deployment/index.rst +++ b/rsts/deployment/index.rst @@ -48,6 +48,12 @@ solution). The following pages will help you effectively deploy and manage an en --- + Plugin Setup + ^^^^^^^^^^^^ + Plugin Setup guides with detailed instructions. + + --- + .. link-button:: deployment-cluster-config :type: ref :text: Cluster Configuration @@ -66,3 +72,4 @@ solution). The following pages will help you effectively deploy and manage an en gcp/index cluster_config/index sandbox + plugin_setup/index diff --git a/rsts/deployment/plugin_setup/index.rst b/rsts/deployment/plugin_setup/index.rst new file mode 100644 index 00000000000..025ae0fbc35 --- /dev/null +++ b/rsts/deployment/plugin_setup/index.rst @@ -0,0 +1,33 @@ +.. _deployment-cluster-config: + +############## +Cluster Config +############## + + +.. panels:: + :header: text-center + + .. link-button:: deployment-plugin-setup-mpi-operator + :type: ref + :text: MPI Plugin + :classes: btn-block stretched-link + ^^^^^^^^^^^^ + Guide to setting up MPI Plugin + + --- + + .. link-button:: deployment-plugin-setup-spark-plugin + :type: ref + :text: Spark Plugin + :classes: btn-block stretched-link + ^^^^^^^^^^^^ + Guide to setting up Spark Plugin + + +.. toctree:: + :maxdepth: 1 + :name: Plugin Setup + :hidden: + + mpi_operator diff --git a/rsts/deployment/plugin_setup/mpi_operator.rst b/rsts/deployment/plugin_setup/mpi_operator.rst new file mode 100644 index 00000000000..43e4c132c2a --- /dev/null +++ b/rsts/deployment/plugin_setup/mpi_operator.rst @@ -0,0 +1,57 @@ +.. _deployment-plugin-setup-mpi-operator: + +MPI Operator Setup +------------------------ + +.. _install-mpi-appendix: + +############## +Install MPI Operator +############## + +Start the sandbox for testing + +.. code-block:: bash + + flytectl sandbox start + +Clone MPI Operator + +.. code-block:: bash + + git clone https://github.com/kubeflow/mpi-operator.git && cd mpi-operator/manifests + +Install MPI Operator + +.. code-block:: bash + + kustomize build overlays/kubeflow | kubectl apply -f - + +Create a values-mpi.yaml + +.. code-block:: + + configmap: + enabled_plugins: + # -- Tasks specific configuration [structure](https://pkg.go.dev/github.com/flyteorg/flytepropeller/pkg/controller/nodes/task/config#GetConfig) + tasks: + # -- Plugins configuration, [structure](https://pkg.go.dev/github.com/flyteorg/flytepropeller/pkg/controller/nodes/task/config#TaskPluginConfig) + task-plugins: + # -- [Enabled Plugins](https://pkg.go.dev/github.com/flyteorg/flyteplugins/go/tasks/config#Config). Enable sagemaker*, athena if you install the backend + # plugins + enabled-plugins: + - container + - sidecar + - k8s-array + - mpi + default-for-task-types: + container: container + sidecar: sidecar + container_array: k8s-array + mpi: mpi + +Upgrade flyte helm release + +.. code-block:: bash + + helm upgrade -n flyte -f values-mpi.yaml flyteorg/flyte --kubeconfig=~/.flyte/k3s/k3s.yaml \ No newline at end of file