From 0483505b4bbf526189cf4b3e97ecadcb943b24d3 Mon Sep 17 00:00:00 2001 From: Ketan Umare <16888709+kumare3@users.noreply.github.com> Date: Fri, 28 May 2021 18:17:12 -0700 Subject: [PATCH] Introduce Flyte Helm Chart (#916) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * update docs theme (#876) * update docs theme Signed-off-by: cosmicBboy * remove sphinx material and readthedocs search Signed-off-by: cosmicBboy * update flytekit link Signed-off-by: cosmicBboy * update Signed-off-by: cosmicBboy * add readthedocs sphinx search Signed-off-by: cosmicBboy * update community link Signed-off-by: cosmicBboy Signed-off-by: Haytham Abuelfutuh * Helm chart for Flyte - initial commit Signed-off-by: Sören Brunk Signed-off-by: Haytham Abuelfutuh * remove new lines Signed-off-by: Sören Brunk Signed-off-by: Haytham Abuelfutuh * Added autogenerated Helm docs Signed-off-by: Sören Brunk Signed-off-by: Haytham Abuelfutuh * improve templates Signed-off-by: Sören Brunk Signed-off-by: Haytham Abuelfutuh * Support gRPC loadbalncing for EKS with AWS ALB Signed-off-by: Sören Brunk Signed-off-by: Haytham Abuelfutuh * Use external chart dependencies for contour and spark-operator Bump versions Signed-off-by: Sören Brunk Signed-off-by: Haytham Abuelfutuh * Helm - Update task plugin config Based on the following kustomize changes: - cd4a3d5a Trim sandbox (#727) - 9b59ac68 Update propeller version and config (#580) Signed-off-by: Sören Brunk Signed-off-by: Haytham Abuelfutuh * Helm - Updating Spark config to force default credentials chain for AWS Based on the following kustomize change: - 0e2dd723 Updating Spark config to force default credentials chain for AWS (#674) Signed-off-by: Sören Brunk Signed-off-by: Haytham Abuelfutuh * Helm - Update contour GRPC annotation Based on the following kustomize change: - 46864a53 Access GRPC services using projectcontour (#686) Signed-off-by: Sören Brunk Signed-off-by: Haytham Abuelfutuh * Helm - Enabling Event version 1 by default Based on the following kustomize change: - 658aecd7 Enabling Event version 1 by default (#689) Signed-off-by: Sören Brunk Signed-off-by: Haytham Abuelfutuh * Helm - Improve getting started experience Based on the following kustomize change: - fbe7e684 Improve getting started experience (#694) Signed-off-by: Sören Brunk Signed-off-by: Haytham Abuelfutuh * Helm - Exponential error back-off in case of system errors Based on the following kustomize change: - 0844a44b Exponential error back-off in case of system errors (#693) Signed-off-by: Sören Brunk Signed-off-by: Haytham Abuelfutuh * Helm - Trim sandbox Based on the following kustomize change: - cd4a3d5a Trim sandbox (#727) Signed-off-by: Sören Brunk Signed-off-by: Haytham Abuelfutuh * Helm - Update default RAM Request for tasks in sandbox Based on the following kustomize change: - 72c57053 Update default RAM Request for tasks in sandbox (#750) Signed-off-by: Sören Brunk Signed-off-by: Haytham Abuelfutuh * Helm - Update Admin task resources Based on the following kustomize change: - b13de214 Update Admin task resources (#753) Signed-off-by: Sören Brunk Signed-off-by: Haytham Abuelfutuh * Helm - Add kubernetes dashboard and update deployments Based on the following kustomize changes: - a14fd47f Update deployments with latest k8s library deps (#826) - f149af8e Add kubernetes dashboard to sandbox overlay (#789) Signed-off-by: Sören Brunk Signed-off-by: Haytham Abuelfutuh * Helm - Update EKS Ingress Use default Docker images in EKS Helm values. Based on the following kustomize changes: - 4e0eed69 Update EKS Ingress and Instructions (#825) Signed-off-by: Sören Brunk Signed-off-by: Haytham Abuelfutuh * Move GRPC annotations from ingress to services where possible Signed-off-by: Sören Brunk Signed-off-by: Haytham Abuelfutuh * Improve installation docs Signed-off-by: Sören Brunk Signed-off-by: Haytham Abuelfutuh * Downgrade Ingress API version for K8S 1.18 compatibility Signed-off-by: Sören Brunk Signed-off-by: Haytham Abuelfutuh * Add tls option for ingress Signed-off-by: Sören Brunk Signed-off-by: Haytham Abuelfutuh * Remove redundant default values from EKS config Signed-off-by: Sören Brunk Signed-off-by: Haytham Abuelfutuh * Simplify Helm storage configuration Signed-off-by: Sören Brunk Signed-off-by: Haytham Abuelfutuh * [wip] Helm components (#921) Signed-off-by: Haytham Abuelfutuh * Working helm deployment Signed-off-by: Haytham Abuelfutuh * more docs Signed-off-by: Ketan Umare Signed-off-by: Haytham Abuelfutuh * Add references to flyteadmin config Signed-off-by: Katrina Rogan Signed-off-by: Haytham Abuelfutuh * Improved value.yaml docs Signed-off-by: Ketan Umare Signed-off-by: Haytham Abuelfutuh * updated yaml Signed-off-by: Ketan Umare Signed-off-by: Haytham Abuelfutuh * Updated working basic eks config Signed-off-by: Ketan Umare Signed-off-by: Haytham Abuelfutuh * Improve installation docs and bump versions Signed-off-by: Sören Brunk Signed-off-by: Haytham Abuelfutuh * Fix service account annotation structure in templates Signed-off-by: Sören Brunk Signed-off-by: Haytham Abuelfutuh * Fully working Signed-off-by: Ketan Umare Signed-off-by: Haytham Abuelfutuh * Update image tags, fix customDataConfig and expose defaultIamRole as a template field Signed-off-by: Haytham Abuelfutuh * Allow remote data config for admin Signed-off-by: Haytham Abuelfutuh * Use bet version of ingress (#988) "Hey so, I noticed that you are using the Ingress class out of beta, which is kind of an issue-- the Ingress class was moved out of beta in the 1.19 release, which happened in late august. If flyte is run in clusters older than this (e.g. 1.18), then this will fail. EKS only started offering 1.19 a couple of months ago, and the GKE regular channel defaults to 1.18, so this really won't work for most of your users. Do you guys mind if I file a pr to move back to the beta ingress version? I don't see you guys using anyhting required from the GA version, and beta has been serving everyone well for the past 6 years" Signed-off-by: JD Signed-off-by: Haytham Abuelfutuh * Add auth services to ingress Signed-off-by: Haytham Abuelfutuh * Support imagePullSecrets for service accounts Signed-off-by: Haytham Abuelfutuh * Update Console version Signed-off-by: Haytham Abuelfutuh * Add Auth support to Helm chart - Move grpc routes into template function Based on - 840f250d Add Auth Ingress endpoints (#982) - 1ede3750 Authentication Config Examples (#970) - 77941a32 explicitly grant flyteworkflow finalizer permissions to flytepropeller (#930) Signed-off-by: Sören Brunk Signed-off-by: Haytham Abuelfutuh * update flytepropeller Signed-off-by: Haytham Abuelfutuh * Add Pod Webhook deployment to Helm chart (#972) * Add Pod Webhook deployment to Helm chart Based on a8c01e5a Pod Webhook deployment & Doc (#857) Signed-off-by: Sören Brunk * Remove K8s Job for Pod Webhook Signed-off-by: Haytham Abuelfutuh Co-authored-by: Haytham Abuelfutuh Signed-off-by: Haytham Abuelfutuh * dark theme updates (#883) Signed-off-by: cosmicBboy Signed-off-by: Haytham Abuelfutuh * Release automation addedd for flyte definition (#835) * wip: added script for release automation Signed-off-by: yuvraj * wip: added manual github workflow Signed-off-by: Yuvraj * wip: added release workflow Signed-off-by: Yuvraj * wip: fix e2e testcase Signed-off-by: Yuvraj * wip: small fixes Signed-off-by: Yuvraj * wip: revert version in kustomize Signed-off-by: Yuvraj * wip: revert kustomize changes Signed-off-by: Yuvraj * wip: added generated kustomize output Signed-off-by: yuvraj * update latest version of component Signed-off-by: Yuvraj * wip: added generated kustomize output Signed-off-by: yuvraj * wip: added message input in workflow Signed-off-by: yuvraj * wip: remove release automation from kustomize command Signed-off-by: yuvraj * wip: generate kustomize output Signed-off-by: yuvraj * wip: revert webhook and build kustomize output Signed-off-by: yuvraj * updated release workflow Signed-off-by: yuvraj * fix typo Signed-off-by: yuvraj Signed-off-by: Haytham Abuelfutuh * Milestone 0.12.0 (#888) * Update Flyte Components Signed-off-by: Flyte-Bot Signed-off-by: Haytham Abuelfutuh * update endtoend to flytekit 0.17 Signed-off-by: wild-endeavor Signed-off-by: Haytham Abuelfutuh * empty changelog Signed-off-by: wild-endeavor Signed-off-by: Haytham Abuelfutuh * Update ChangeLog Signed-off-by: Haytham Abuelfutuh * before pull from flyte-bot-update-releases Signed-off-by: Haytham Abuelfutuh Co-authored-by: EngHabu Co-authored-by: wild-endeavor Co-authored-by: HackMD Signed-off-by: Haytham Abuelfutuh * Update index.rst (#884) * Update index.rst Add Flyte users to Who's Using Flyte and Meet Flyte. * Update rsts/index.rst Co-authored-by: Ketan Umare <16888709+kumare3@users.noreply.github.com> * Update index.rst Co-authored-by: Ketan Umare <16888709+kumare3@users.noreply.github.com> Signed-off-by: Haytham Abuelfutuh * Dashboard updates (#886) * Adding additional metrics in Flyteadmin dashboard Signed-off-by: Ketan Umare * Flytepropeller metrics added Signed-off-by: Ketan Umare * updated Signed-off-by: Ketan Umare * updated Signed-off-by: Ketan Umare * Added queue metrics Signed-off-by: Ketan Umare * updated name Signed-off-by: Ketan Umare Signed-off-by: Haytham Abuelfutuh * Add init container that will pre-create buckets in minio (#905) * Add init container that will pre-create buckets in minio Signed-off-by: Jeev B * move minio tag override to overlay Signed-off-by: Jeev B Signed-off-by: Haytham Abuelfutuh * Add LF AI & Data Logo to README (#902) Signed-off-by: Haytham Abuelfutuh * Explaining the state machine used by FlytePropeller and Flyte (#903) * Explaining the state machine used by FlytePropeller and Flyte - this document helps to explain the various states a workflow, node and task transitions through. - TODO add a small table that helps users understand what a state in the UI represents Signed-off-by: Ketan Umare * updated docs (addressed comments) Signed-off-by: Ketan Umare * comments addressed - image for nodes improved Signed-off-by: Ketan Umare Signed-off-by: Haytham Abuelfutuh * Fix default eks storage config (#907) * Fix default eks storage config Signed-off-by: Haytham Abuelfutuh * kustomize Signed-off-by: Haytham Abuelfutuh * bump Signed-off-by: Haytham Abuelfutuh * Remove flyteadmin config options which now have defaults (#917) Signed-off-by: Haytham Abuelfutuh * Contribution Guide (#922) Signed-off-by: Haytham Abuelfutuh * explicitly grant flyteworkflow finalizer permissions to flytepropeller (#930) * explicitly grant flyteworkflow finalizer permissions to flytepropeller Signed-off-by: akumor * add kustomize output after adding flyteworkflows/finalizers rbac Signed-off-by: akumor Signed-off-by: Haytham Abuelfutuh * Add dynamic running state to node state machine (#929) Signed-off-by: Haytham Abuelfutuh * Updated Boilerplate Code (#932) Signed-off-by: Samhita Alla Signed-off-by: Haytham Abuelfutuh * Added goreleaser for publishing menifest (#927) * added goreleaser for publishing menifest Signed-off-by: yuvraj * Rename release file name Signed-off-by: yuvraj * Added script for artifacts Signed-off-by: yuvraj * More changes Signed-off-by: yuvraj * More changes Signed-off-by: yuvraj * fix typo Signed-off-by: yuvraj * minor changes Signed-off-by: yuvraj Signed-off-by: Haytham Abuelfutuh * Added flytectl in sandbox (#952) * Added sandbox changes for new release process Signed-off-by: yuvraj * curl added in sandbox build image Signed-off-by: yuvraj Signed-off-by: Haytham Abuelfutuh * Modify repo name in Makefile (#953) * Updated Boilerplate Code Signed-off-by: Samhita Alla * modified export line Signed-off-by: Samhita Alla Signed-off-by: Haytham Abuelfutuh * Signed-off-by: Samhita Alla aallasamhita@gmail.com (#955) Co-authored-by: Niels Bantilan niels.bantilan@gmail.com Documentation revamp restructuring according to the RFC: https://docs.google.com/document/d/1Yp5cYujKT6UMv17Y6r1djMchaMZWAlPNUTZnbw1sZvc/edit#heading=h.vozt1qaes3ur Signed-off-by: cosmicBboy Co-authored-by: Samhita Alla Signed-off-by: Haytham Abuelfutuh * added tagline (#956) Signed-off-by: Samhita Alla Signed-off-by: Haytham Abuelfutuh * Fix broken Kustomize link in docs (#944) Signed-off-by: Tom Forbes Signed-off-by: Haytham Abuelfutuh * update getting started guide (#958) since we're consolidating our docs, we want to use the getting started example originally from flytesnacks. Here we're moving it into the flyte repo since the flytesnacks example should be the canonical getting started guide Signed-off-by: cosmicBboy Signed-off-by: Haytham Abuelfutuh * Authentication Config Examples (#970) * Fix broken Kustomize link in docs (#944) Signed-off-by: Tom Forbes Signed-off-by: Haytham Abuelfutuh * update getting started guide (#958) since we're consolidating our docs, we want to use the getting started example originally from flytesnacks. Here we're moving it into the flyte repo since the flytesnacks example should be the canonical getting started guide Signed-off-by: cosmicBboy Signed-off-by: Haytham Abuelfutuh * Authentication Config Examples Signed-off-by: Haytham Abuelfutuh * Feedback Signed-off-by: Haytham Abuelfutuh * fix map specification in cluster resource template Signed-off-by: Haytham Abuelfutuh * attempting stdlib fix Signed-off-by: Haytham Abuelfutuh * Fix merge issue Signed-off-by: Haytham Abuelfutuh * Update Signed-off-by: Haytham Abuelfutuh * Spellcheck Signed-off-by: Haytham Abuelfutuh * regenerate Signed-off-by: Haytham Abuelfutuh * Update migration.rst Signed-off-by: Haytham Abuelfutuh * Update setup.rst Signed-off-by: Haytham Abuelfutuh Co-authored-by: Tom Forbes Co-authored-by: Niels Bantilan Co-authored-by: SandraGH5 <80421934+SandraGH5@users.noreply.github.com> Signed-off-by: Haytham Abuelfutuh * Flyte milestone v0.13 release (#979) Signed-off-by: Haytham Abuelfutuh * Add Auth Ingress endpoints (#982) Signed-off-by: Haytham Abuelfutuh * Updated Readme.md with CII Badge (#990) Signed-off-by: Haytham Abuelfutuh * Wait for deployments to be created using rollout before checking for availability (#991) Signed-off-by: Ketan Umare Signed-off-by: Haytham Abuelfutuh * update links to autogenerated flytesnacks pages (#999) * update links to autogenerated flytesnacks pages due to https://github.com/flyteorg/flytesnacks/pull/200 we need to update a bunch of links referring to auto-generated flytesnacks examples Signed-off-by: cosmicBboy * update link Signed-off-by: cosmicBboy * update Signed-off-by: cosmicBboy * update links cookbook -> user guide Signed-off-by: cosmicBboy Signed-off-by: Haytham Abuelfutuh * Include troubleshooting guide in the docs (#987) * troubleshooting guide in the docs Signed-off-by: Samhita Alla * modified list index Signed-off-by: Samhita Alla * added disk space condition Signed-off-by: Samhita Alla * modified list index Signed-off-by: Samhita Alla * moved troubleshooting guide to community Signed-off-by: Samhita Alla Signed-off-by: Haytham Abuelfutuh * Add data council video Signed-off-by: Haytham Abuelfutuh * Added tip for make start to check troubleshooting page (#1023) * Added tip for make start to check troubleshooting page Signed-off-by: Prafulla Mahindrakar Signed-off-by: Haytham Abuelfutuh * Update README.md (#1022) * Update README.md Signed-off-by: Ketan Umare * Update README (#1024) * updated readme Signed-off-by: Samhita Alla * updated readme Signed-off-by: Samhita Alla * removed css Signed-off-by: Samhita Alla * moved from html to md Signed-off-by: Samhita Alla * Update README.md Co-authored-by: Samhita Alla Co-authored-by: SandraGH5 <80421934+SandraGH5@users.noreply.github.com> Signed-off-by: Haytham Abuelfutuh * Rename admin server config section Signed-off-by: Haytham Abuelfutuh * Fix deployment indentation Signed-off-by: Haytham Abuelfutuh * Update image versions Signed-off-by: Haytham Abuelfutuh * disable tls by default because of: https://github.com/helm/helm/issues/5238 Signed-off-by: Haytham Abuelfutuh * default imagepullsecrets Signed-off-by: Haytham Abuelfutuh * Disable SSL Redirect for GRPC Signed-off-by: Haytham Abuelfutuh * Fix spark config map Signed-off-by: Haytham Abuelfutuh * Update imagePullSecrets Signed-off-by: Haytham Abuelfutuh * Add make helm to generate static helm charts Signed-off-by: Haytham Abuelfutuh * add --dependency-update to helm command Signed-off-by: Haytham Abuelfutuh * Don't hardcode namespace in flyte-admin-auth secret Signed-off-by: Sören Brunk Signed-off-by: Haytham Abuelfutuh * generate helm Signed-off-by: Haytham Abuelfutuh * Helm template upgrade to auto-generate co-pilot configuration (#1056) * Helm template upgrade to auto-generate co-pilot configuration Signed-off-by: Ketan Umare * using common storage base for all storage and co-pilot Signed-off-by: Ketan Umare Signed-off-by: Haytham Abuelfutuh * Install deps once Signed-off-by: Haytham Abuelfutuh * Install helm-docs if not found Signed-off-by: Haytham Abuelfutuh * bump Signed-off-by: Haytham Abuelfutuh * Fix merge Signed-off-by: Haytham Abuelfutuh * Update helm docs Signed-off-by: Haytham Abuelfutuh Co-authored-by: Niels Bantilan Co-authored-by: Ruslan Stanevich Co-authored-by: Sören Brunk Co-authored-by: Katrina Rogan Co-authored-by: Haytham Abuelfutuh Co-authored-by: Juan Diego Palomino Co-authored-by: Yuvraj <10830562+evalsocket@users.noreply.github.com> Co-authored-by: Flyte Bot Co-authored-by: EngHabu Co-authored-by: wild-endeavor Co-authored-by: HackMD Co-authored-by: SandraGH5 <80421934+SandraGH5@users.noreply.github.com> Co-authored-by: Jeev B Co-authored-by: Samhita Alla Co-authored-by: Katrina Rogan Co-authored-by: Jan Kumor Co-authored-by: Tom Forbes Co-authored-by: pmahindrakar-oss <77798312+pmahindrakar-oss@users.noreply.github.com> --- .github/workflows/tests.yml | 2 + .gitignore | 2 + Makefile | 14 +- deployment/eks/flyte_helm_generated.yaml | 2077 ++++++++ deployment/gcp/flyte_helm_generated.yaml | 4066 ++++++++++++++++ deployment/sandbox/flyte_helm_generated.yaml | 4200 +++++++++++++++++ doc-requirements.txt | 3 + helm/.helmignore | 23 + helm/Chart.yaml | 20 + helm/README.md | 232 + helm/README.md.gotmpl | 52 + helm/templates/_helpers.tpl | 216 + .../admin/cluster_resource_configmap.yaml | 13 + helm/templates/admin/configmap.yaml | 41 + helm/templates/admin/cronjob.yaml | 42 + helm/templates/admin/deployment.yaml | 154 + helm/templates/admin/rbac.yaml | 57 + helm/templates/admin/secret-auth.yaml | 6 + helm/templates/admin/service.yaml | 34 + helm/templates/common/ingress.yaml | 212 + helm/templates/common/namespace.yaml | 9 + helm/templates/common/secret.yaml | 3 + helm/templates/console/configmap.yaml | 7 + helm/templates/console/deployment.yaml | 44 + helm/templates/console/service.yaml | 18 + helm/templates/datacatalog/configmap.yaml | 18 + helm/templates/datacatalog/deployment.yaml | 67 + helm/templates/datacatalog/rbac.yaml | 16 + helm/templates/datacatalog/service.yaml | 27 + helm/templates/minio/deployment.yaml | 50 + helm/templates/minio/service.yaml | 21 + helm/templates/postgres/deployment.yaml | 45 + helm/templates/postgres/service.yaml | 21 + helm/templates/propeller/configmap.yaml | 50 + .../propeller/crds/flyteworkflow.yaml | 14 + helm/templates/propeller/deployment.yaml | 60 + helm/templates/propeller/rbac.yaml | 98 + helm/templates/propeller/secret-auth.yaml | 8 + helm/templates/propeller/webhook.yaml | 158 + .../pytorch-operator/crds/pytorchjobs.yaml | 44 + .../pytorch-operator/deployment.yaml | 48 + .../templates/pytorch-operator/namespace.yaml | 6 + helm/templates/pytorch-operator/rbac.yaml | 108 + helm/templates/pytorch-operator/service.yaml | 20 + helm/templates/redis/service.yaml | 21 + helm/templates/redis/statefulset.yaml | 70 + helm/values-eks.yaml | 505 ++ helm/values-gcp.yaml | 48 + helm/values-sandbox.yaml | 239 + helm/values.yaml | 842 ++++ script/generate_helm.sh | 39 + 51 files changed, 14189 insertions(+), 1 deletion(-) create mode 100644 deployment/eks/flyte_helm_generated.yaml create mode 100644 deployment/gcp/flyte_helm_generated.yaml create mode 100644 deployment/sandbox/flyte_helm_generated.yaml create mode 100644 helm/.helmignore create mode 100644 helm/Chart.yaml create mode 100644 helm/README.md create mode 100644 helm/README.md.gotmpl create mode 100644 helm/templates/_helpers.tpl create mode 100644 helm/templates/admin/cluster_resource_configmap.yaml create mode 100644 helm/templates/admin/configmap.yaml create mode 100644 helm/templates/admin/cronjob.yaml create mode 100644 helm/templates/admin/deployment.yaml create mode 100644 helm/templates/admin/rbac.yaml create mode 100644 helm/templates/admin/secret-auth.yaml create mode 100644 helm/templates/admin/service.yaml create mode 100644 helm/templates/common/ingress.yaml create mode 100644 helm/templates/common/namespace.yaml create mode 100644 helm/templates/common/secret.yaml create mode 100644 helm/templates/console/configmap.yaml create mode 100644 helm/templates/console/deployment.yaml create mode 100644 helm/templates/console/service.yaml create mode 100644 helm/templates/datacatalog/configmap.yaml create mode 100644 helm/templates/datacatalog/deployment.yaml create mode 100644 helm/templates/datacatalog/rbac.yaml create mode 100644 helm/templates/datacatalog/service.yaml create mode 100644 helm/templates/minio/deployment.yaml create mode 100644 helm/templates/minio/service.yaml create mode 100644 helm/templates/postgres/deployment.yaml create mode 100644 helm/templates/postgres/service.yaml create mode 100644 helm/templates/propeller/configmap.yaml create mode 100644 helm/templates/propeller/crds/flyteworkflow.yaml create mode 100644 helm/templates/propeller/deployment.yaml create mode 100644 helm/templates/propeller/rbac.yaml create mode 100644 helm/templates/propeller/secret-auth.yaml create mode 100644 helm/templates/propeller/webhook.yaml create mode 100644 helm/templates/pytorch-operator/crds/pytorchjobs.yaml create mode 100644 helm/templates/pytorch-operator/deployment.yaml create mode 100644 helm/templates/pytorch-operator/namespace.yaml create mode 100644 helm/templates/pytorch-operator/rbac.yaml create mode 100644 helm/templates/pytorch-operator/service.yaml create mode 100644 helm/templates/redis/service.yaml create mode 100644 helm/templates/redis/statefulset.yaml create mode 100644 helm/values-eks.yaml create mode 100644 helm/values-gcp.yaml create mode 100644 helm/values-sandbox.yaml create mode 100644 helm/values.yaml create mode 100644 script/generate_helm.sh diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml index 6c690be8be..743b7e1556 100644 --- a/.github/workflows/tests.yml +++ b/.github/workflows/tests.yml @@ -11,6 +11,8 @@ jobs: - uses: actions/checkout@v1 - name: Kustomize and diff run: DELTA_CHECK=true make kustomize + - name: Helm and diff + run: DELTA_CHECK=true make helm - uses: engineerd/setup-kind@v0.5.0 with: version: "v0.10.0" diff --git a/.gitignore b/.gitignore index bd8ac5ca2c..5abe39323f 100644 --- a/.gitignore +++ b/.gitignore @@ -16,3 +16,5 @@ docs/searchindex.js docs/ release/ __pycache__/ +/helm/charts/ +Chart.lock diff --git a/Makefile b/Makefile index 6b38fd26ff..a5449e4b92 100644 --- a/Makefile +++ b/Makefile @@ -12,6 +12,10 @@ update_boilerplate: kustomize: KUSTOMIZE_VERSION=3.9.2 bash script/generate_kustomize.sh +.PHONY: helm +helm: + bash script/generate_helm.sh + .PHONY: release_automation release_automation: bash script/release.sh @@ -50,4 +54,12 @@ stats: .PHONY: prepare_artifacts prepare_artifacts: - bash script/prepare_artifacts.sh \ No newline at end of file + bash script/prepare_artifacts.sh + +.PHONY: helm_install +helm_install: + helm install flyte --debug ./helm -f helm/values-sandbox.yaml --create-namespace + +.PHONY: helm_upgrade +helm_upgrade: + helm upgrade flyte --debug ./helm -f helm/values-sandbox.yaml --create-namespace diff --git a/deployment/eks/flyte_helm_generated.yaml b/deployment/eks/flyte_helm_generated.yaml new file mode 100644 index 0000000000..4ac91f10a5 --- /dev/null +++ b/deployment/eks/flyte_helm_generated.yaml @@ -0,0 +1,2077 @@ +--- +# Source: flyte/templates/pytorch-operator/namespace.yaml +apiVersion: v1 +kind: Namespace +metadata: + name: pytorch-operator +--- +# Source: flyte/charts/sparkoperator/templates/serviceaccount.yaml +apiVersion: v1 +kind: ServiceAccount +metadata: + name: flyte-sparkoperator + labels: + helm.sh/chart: sparkoperator-1.0.6 + app.kubernetes.io/name: sparkoperator + app.kubernetes.io/instance: flyte + app.kubernetes.io/version: "v1beta2-1.2.0-3.0.0" + app.kubernetes.io/managed-by: Helm +--- +# Source: flyte/charts/sparkoperator/templates/spark-serviceaccount.yaml +apiVersion: v1 +kind: ServiceAccount +metadata: + name: flyte-spark + namespace: default + labels: + helm.sh/chart: sparkoperator-1.0.6 + app.kubernetes.io/name: sparkoperator + app.kubernetes.io/instance: flyte + app.kubernetes.io/version: "v1beta2-1.2.0-3.0.0" + app.kubernetes.io/managed-by: Helm +--- +# Source: flyte/templates/propeller/webhook.yaml +# Create a Service Account for webhook +apiVersion: v1 +kind: ServiceAccount +metadata: + name: flyte-pod-webhook + namespace: default +--- +# Source: flyte/templates/pytorch-operator/rbac.yaml +apiVersion: v1 +kind: ServiceAccount +metadata: + name: pytorch-operator + namespace: pytorch-operator + labels: + app.kubernetes.io/name: pytorch-operator + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm +--- +# Source: flyte/templates/admin/secret-auth.yaml +apiVersion: v1 +kind: Secret +metadata: + name: flyte-admin-auth + namespace: default +type: Opaque +--- +# Source: flyte/templates/common/secret.yaml +apiVersion: v1 +kind: Secret +metadata: + name: db-pass +stringData: + pass.txt: awesomesauce +type: Opaque +--- +# Source: flyte/templates/propeller/secret-auth.yaml +apiVersion: v1 +kind: Secret +metadata: + name: flyte-propeller-auth + namespace: default +type: Opaque +stringData: + client_secret: foobar +--- +# Source: flyte/templates/propeller/webhook.yaml +# Create an empty secret that the first propeller pod will populate +apiVersion: v1 +kind: Secret +metadata: + name: flyte-pod-webhook + namespace: default +type: Opaque +--- +# Source: flyte/templates/admin/cluster_resource_configmap.yaml +apiVersion: v1 +kind: ConfigMap +metadata: + name: clusterresource-template + namespace: default + labels: + app.kubernetes.io/name: flyteadmin + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm +data: + aa_namespace.yaml: | + apiVersion: v1 + kind: Namespace + metadata: + name: {{ namespace }} + spec: + finalizers: + - kubernetes + + aab_default_service_account.yaml: | + apiVersion: v1 + kind: ServiceAccount + metadata: + name: default + namespace: {{ namespace }} + annotations: + eks.amazonaws.com/role-arn: {{ defaultIamRole }} + + ab_project_resource_quota.yaml: | + apiVersion: v1 + kind: ResourceQuota + metadata: + name: project-quota + namespace: {{ namespace }} + spec: + hard: + limits.cpu: {{ projectQuotaCpu }} + limits.memory: {{ projectQuotaMemory }} + + ad_spark_role.yaml: | + apiVersion: rbac.authorization.k8s.io/v1beta1 + kind: Role + metadata: + name: spark-role + namespace: {{ namespace }} + rules: + - apiGroups: + - "" + resources: + - pods + verbs: + - '*' + - apiGroups: + - "" + resources: + - services + verbs: + - '*' + - apiGroups: + - "" + resources: + - configmaps + verbs: + - '*' + + ae_spark_service_account.yaml: | + apiVersion: v1 + kind: ServiceAccount + metadata: + name: spark + namespace: {{ namespace }} + annotations: + eks.amazonaws.com/role-arn: {{ defaultIamRole }} + + af_spark_role_binding.yaml: | + apiVersion: rbac.authorization.k8s.io/v1beta1 + kind: RoleBinding + metadata: + name: spark-role-binding + namespace: {{ namespace }} + roleRef: + apiGroup: rbac.authorization.k8s.io + kind: Role + name: spark-role + subjects: + - kind: ServiceAccount + name: spark + namespace: {{ namespace }} + + zz_copilot_config.yaml: | + kind: ConfigMap + apiVersion: v1 + metadata: + name: flyte-data-config + namespace: {{ namespace }} + data: + config.yaml: | + storage: + type: s3 + container: "" + connection: + auth-type: iam + region: + enable-multicontainer: true +--- +# Source: flyte/templates/admin/configmap.yaml +apiVersion: v1 +kind: ConfigMap +metadata: + name: flyte-admin-config + namespace: default + labels: + app.kubernetes.io/name: flyteadmin + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm +data: + db.yaml: | + database: + dbname: flyteadmin + host: + passwordPath: /etc/db/pass.txt + port: 5432 + username: flyte + domain.yaml: | + domains: + - id: development + name: development + - id: staging + name: staging + - id: production + name: production + logger.yaml: | + logger: + level: 5 + show-source: true + server.yaml: | + auth: + appAuth: + thirdPartyConfig: + flyteClient: + clientId: flytectl + redirectUri: https://localhost:53593/callback + scopes: + - offline + - all + authorizedUris: + - https://localhost:30081 + - http://flyteadmin:80 + - http://flyteadmin.flyte.svc.cluster.local:80 + userAuth: + openId: + baseUrl: https://accounts.google.com + clientId: 657465813211-6eog7ek7li5k7i7fvgv2921075063hpe.apps.googleusercontent.com + scopes: + - profile + - openid + flyteadmin: + eventVersion: 1 + metadataStoragePrefix: + - metadata + - admin + metricsScope: 'flyte:' + profilerPort: 10254 + roleNameKey: iam.amazonaws.com/role + testing: + host: http://flyteadmin + server: + grpcPort: 8089 + httpPort: 8088 + security: + allowCors: true + allowedHeaders: + - Content-Type + allowedOrigins: + - '*' + secure: false + useAuth: false + remoteData.yaml: | + remoteData: + region: us-east-1 + scheme: local + signedUrls: + durationMinutes: 3 + storage.yaml: | + storage: + type: s3 + container: "" + connection: + auth-type: iam + region: + limits: + maxDownloadMBs: 10 + task_resource_defaults.yaml: | + task_resources: + defaults: + cpu: 1000m + memory: 1000Mi + storage: 1000Mi + limits: + cpu: 2 + gpu: 1 + memory: 8Gi + storage: 2000Mi + cluster_resources.yaml: | + cluster_resources: + customData: + - production: + - projectQuotaCpu: + value: "5" + - projectQuotaMemory: + value: 4000Mi + - staging: + - projectQuotaCpu: + value: "2" + - projectQuotaMemory: + value: 3000Mi + - development: + - projectQuotaCpu: + value: "4" + - projectQuotaMemory: + value: 3000Mi + refresh: 5m + templatePath: /etc/flyte/clusterresource/templates +--- +# Source: flyte/templates/console/configmap.yaml +apiVersion: v1 +kind: ConfigMap +metadata: + name: flyte-console-config + namespace: default + labels: + app.kubernetes.io/name: flyteconsole + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm +data: + BASE_URL: /console + CONFIG_DIR: /etc/flyte/config + DISABLE_AUTH: "1" +--- +# Source: flyte/templates/datacatalog/configmap.yaml +apiVersion: v1 +kind: ConfigMap +metadata: + name: datacatalog-config + namespace: default + labels: + app.kubernetes.io/name: datacatalog + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm +data: + db.yaml: | + database: + dbname: flyteadmin + host: + passwordPath: /etc/db/pass.txt + port: 5432 + username: flyte + logger.yaml: | + logger: + level: 5 + show-source: true + server.yaml: | + application: + grpcPort: 8089 + grpcServerReflection: true + httpPort: 8080 + datacatalog: + metrics-scope: datacatalog + profiler-port: 10254 + storage-prefix: metadata/datacatalog + storage.yaml: | + storage: + type: s3 + container: "" + connection: + auth-type: iam + region: + limits: + maxDownloadMBs: 10 +--- +# Source: flyte/templates/propeller/configmap.yaml +apiVersion: v1 +kind: ConfigMap +metadata: + name: flyte-propeller-config + namespace: default + labels: + app.kubernetes.io/name: flyteadmin + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm +data: + admin.yaml: | + admin: + clientId: flytepropeller + clientSecretLocation: /etc/secrets/client_secret + endpoint: flyteadmin:81 + insecure: true + event: + capacity: 1000 + rate: 500 + type: admin + catalog.yaml: | + catalog-cache: + endpoint: datacatalog:89 + insecure: true + type: datacatalog + copilot.yaml: | + plugins: + k8s: + co-pilot: + image: ghcr.io/lyft/flyteplugins/flytecopilot:dc4bdbd61cac88a39a5ff43e40f026bdbc2c78a2 + name: flyte-copilot- + start-timeout: 30s + core.yaml: | + propeller: + downstream-eval-duration: 30s + enable-admin-launcher: true + gc-interval: 12h + kube-client-config: + burst: 25 + qps: 100 + timeout: 30s + leader-election: + enabled: true + lease-duration: 15s + lock-config-map: + name: propeller-leader + namespace: flyte + renew-deadline: 10s + retry-period: 2s + limit-namespace: all + max-workflow-retries: 50 + metadata-prefix: metadata/propeller + metrics-prefix: flyte + prof-port: 10254 + queue: + batch-size: -1 + batching-interval: 2s + queue: + base-delay: 5s + capacity: 1000 + max-delay: 120s + rate: 100 + type: maxof + sub-queue: + capacity: 1000 + rate: 100 + type: bucket + type: batch + rawoutput-prefix: s3://s3-bucket-for-flyte/ + workers: 40 + workflow-reeval-duration: 30s + webhook: + certDir: /etc/webhook/certs + serviceName: flyte-pod-webhook + enabled_plugins.yaml: | + tasks: + task-plugins: + default-for-task-types: + container: container + container_array: k8s-array + hive: athena + pytorch: pytorch + sidecar: sidecar + spark: spark + enabled-plugins: + - container + - sidecar + - spark + - k8s-array + - pytorch + - athena + k8s.yaml: | + plugins: + k8s: + default-cpus: 100m + default-env-vars: [] + default-memory: 100Mi + logger.yaml: | + logger: + level: 5 + show-source: true + resource_manager.yaml: | + propeller: + resourcemanager: + redis: + hostKey: mypassword + hostPath: redis-resource-manager:6379 + resourceMaxQuota: 10000 + type: redis + spark.yaml: | + plugins: + spark: + spark-config-default: + - spark.hadoop.mapreduce.fileoutputcommitter.algorithm.version: "2" + - spark.kubernetes.allocation.batch.size: "50" + - spark.hadoop.fs.s3a.acl.default: BucketOwnerFullControl + - spark.hadoop.fs.s3n.impl: org.apache.hadoop.fs.s3a.S3AFileSystem + - spark.hadoop.fs.AbstractFileSystem.s3n.impl: org.apache.hadoop.fs.s3a.S3A + - spark.hadoop.fs.s3.impl: org.apache.hadoop.fs.s3a.S3AFileSystem + - spark.hadoop.fs.AbstractFileSystem.s3.impl: org.apache.hadoop.fs.s3a.S3A + - spark.hadoop.fs.s3a.impl: org.apache.hadoop.fs.s3a.S3AFileSystem + - spark.hadoop.fs.AbstractFileSystem.s3a.impl: org.apache.hadoop.fs.s3a.S3A + - spark.hadoop.fs.s3a.multipart.threshold: "536870912" + - spark.blacklist.enabled: "true" + - spark.blacklist.timeout: 5m + - spark.task.maxfailures: "8" + storage.yaml: | + storage: + type: s3 + container: "" + connection: + auth-type: iam + region: + limits: + maxDownloadMBs: 10 + cache.yaml: | + cache: + max_size_mbs: 1024 + target_gc_percent: 70 + task_logs.yaml: | + plugins: + logs: + cloudwatch-enabled: true + cloudwatch-log-group: + cloudwatch-region: + kubernetes-enabled: false +--- +# Source: flyte/templates/propeller/crds/flyteworkflow.yaml +apiVersion: apiextensions.k8s.io/v1beta1 +kind: CustomResourceDefinition +metadata: + name: flyteworkflows.flyte.lyft.com +spec: + group: flyte.lyft.com + names: + kind: FlyteWorkflow + plural: flyteworkflows + shortNames: + - fly + singular: flyteworkflow + scope: Namespaced + version: v1alpha1 +--- +# Source: flyte/templates/pytorch-operator/crds/pytorchjobs.yaml +apiVersion: apiextensions.k8s.io/v1beta1 +kind: CustomResourceDefinition +metadata: + name: pytorchjobs.kubeflow.org +spec: + additionalPrinterColumns: + - JSONPath: .status.conditions[-1:].type + name: State + type: string + - JSONPath: .metadata.creationTimestamp + name: Age + type: date + group: kubeflow.org + names: + kind: PyTorchJob + plural: pytorchjobs + singular: pytorchjob + scope: Namespaced + subresources: + status: {} + validation: + openAPIV3Schema: + properties: + spec: + properties: + pytorchReplicaSpecs: + properties: + Master: + properties: + replicas: + maximum: 1 + minimum: 1 + type: integer + Worker: + properties: + replicas: + minimum: 1 + type: integer + versions: + - name: v1 + served: true + storage: true +--- +# Source: flyte/charts/sparkoperator/templates/rbac.yaml +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: flyte-sparkoperator + labels: + helm.sh/chart: sparkoperator-1.0.6 + app.kubernetes.io/name: sparkoperator + app.kubernetes.io/instance: flyte + app.kubernetes.io/version: "v1beta2-1.2.0-3.0.0" + app.kubernetes.io/managed-by: Helm +rules: +- apiGroups: + - "" + resources: + - pods + verbs: + - "*" +- apiGroups: + - "" + resources: + - services + - configmaps + - secrets + verbs: + - create + - get + - delete + - update +- apiGroups: + - extensions + - networking.k8s.io + resources: + - ingresses + verbs: + - create + - get + - delete +- apiGroups: + - "" + resources: + - nodes + verbs: + - get +- apiGroups: + - "" + resources: + - events + verbs: + - create + - update + - patch +- apiGroups: + - "" + resources: + - resourcequotas + verbs: + - get + - list + - watch +- apiGroups: + - apiextensions.k8s.io + resources: + - customresourcedefinitions + verbs: + - create + - get + - update + - delete +- apiGroups: + - admissionregistration.k8s.io + resources: + - mutatingwebhookconfigurations + - validatingwebhookconfigurations + verbs: + - create + - get + - update + - delete +- apiGroups: + - sparkoperator.k8s.io + resources: + - sparkapplications + - sparkapplications/status + - scheduledsparkapplications + - scheduledsparkapplications/status + verbs: + - "*" +--- +# Source: flyte/templates/admin/rbac.yaml +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: flyteadmin + labels: + app.kubernetes.io/name: flyteadmin + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm +rules: +- apiGroups: + - "" + - flyte.lyft.com + - rbac.authorization.k8s.io + resources: + - configmaps + - flyteworkflows + - namespaces + - pods + - resourcequotas + - roles + - rolebindings + - secrets + - services + - serviceaccounts + - spark-role + verbs: + - '*' +--- +# Source: flyte/templates/propeller/rbac.yaml +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: flytepropeller + labels: + app.kubernetes.io/name: flytepropeller + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm +rules: +# Allow RO access to PODS +- apiGroups: + - "" + resources: + - pods + verbs: + - get + - list + - watch +# Allow Event recording access +- apiGroups: + - "" + resources: + - events + verbs: + - create + - update + - delete + - patch +# Allow Access All plugin objects +- apiGroups: + - '*' + resources: + - '*' + verbs: + - get + - list + - watch + - create + - update + - delete + - patch +# Allow Access to CRD +- apiGroups: + - apiextensions.k8s.io + resources: + - customresourcedefinitions + verbs: + - get + - list + - watch + - create + - delete + - update +# Allow Access to all resources under flyte.lyft.com +- apiGroups: + - flyte.lyft.com + resources: + - flyteworkflows + - flyteworkflows/finalizers + verbs: + - get + - list + - watch + - create + - update + - delete + - patch + - post + - deletecollection +--- +# Source: flyte/templates/propeller/webhook.yaml +# Create a ClusterRole for the webhook +# https://kubernetes.io/docs/admin/authorization/rbac/ +kind: ClusterRole +apiVersion: rbac.authorization.k8s.io/v1 +metadata: + name: flyte-pod-webhook + namespace: default +rules: + - apiGroups: + - "*" + resources: + - mutatingwebhookconfigurations + - secrets + - pods + verbs: + - get + - create + - update + - patch +--- +# Source: flyte/templates/pytorch-operator/rbac.yaml +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: pytorch-operator-admin + labels: + app.kubernetes.io/name: pytorch-operator + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm +rules: [] +aggregationRule: + clusterRoleSelectors: + - matchLabels: + rbac.authorization.kubeflow.org/aggregate-to-kubeflow-pytorchjobs-admin: "true" +--- +# Source: flyte/templates/pytorch-operator/rbac.yaml +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: pytorch-operator-edit + labels: + app.kubernetes.io/name: pytorch-operator + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm +rules: +- apiGroups: + - kubeflow.org + resources: + - pytorchjobs + - pytorchjobs/status + verbs: + - get + - list + - watch + - create + - delete + - deletecollection + - patch + - update +--- +# Source: flyte/templates/pytorch-operator/rbac.yaml +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: pytorch-operator-view + labels: + app.kubernetes.io/name: pytorch-operator + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm +rules: +- apiGroups: + - kubeflow.org + resources: + - pytorchjobs + - pytorchjobs/status + verbs: + - get + - list + - watch +--- +# Source: flyte/templates/pytorch-operator/rbac.yaml +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: pytorch-operator + labels: + app.kubernetes.io/name: pytorch-operator + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm +rules: +- apiGroups: + - kubeflow.org + resources: + - pytorchjobs + - pytorchjobs/status + verbs: + - '*' +- apiGroups: + - apiextensions.k8s.io + resources: + - customresourcedefinitions + verbs: + - '*' +- apiGroups: + - "" + resources: + - pods + - services + - endpoints + - events + verbs: + - '*' +--- +# Source: flyte/charts/sparkoperator/templates/rbac.yaml +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding +metadata: + name: flyte-sparkoperator + labels: + helm.sh/chart: sparkoperator-1.0.6 + app.kubernetes.io/name: sparkoperator + app.kubernetes.io/instance: flyte + app.kubernetes.io/version: "v1beta2-1.2.0-3.0.0" + app.kubernetes.io/managed-by: Helm +subjects: + - kind: ServiceAccount + name: flyte-sparkoperator + namespace: default +roleRef: + kind: ClusterRole + name: flyte-sparkoperator + apiGroup: rbac.authorization.k8s.io +--- +# Source: flyte/templates/admin/rbac.yaml +apiVersion: rbac.authorization.k8s.io/v1beta1 +kind: ClusterRoleBinding +metadata: + name: flyteadmin-binding + labels: + app.kubernetes.io/name: flyteadmin + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: flyteadmin +subjects: +- kind: ServiceAccount + name: flyteadmin + namespace: default +--- +# Source: flyte/templates/propeller/rbac.yaml +apiVersion: rbac.authorization.k8s.io/v1beta1 +kind: ClusterRoleBinding +metadata: + name: flytepropeller + labels: + app.kubernetes.io/name: flytepropeller + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: flytepropeller +subjects: +- kind: ServiceAccount + name: flytepropeller + namespace: default +--- +# Source: flyte/templates/propeller/webhook.yaml +# Create a binding from Role -> ServiceAccount +kind: ClusterRoleBinding +apiVersion: rbac.authorization.k8s.io/v1 +metadata: + name: flyte-pod-webhook + namespace: default +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: flyte-pod-webhook +subjects: + - kind: ServiceAccount + name: flyte-pod-webhook + namespace: default +--- +# Source: flyte/templates/pytorch-operator/rbac.yaml +apiVersion: rbac.authorization.k8s.io/v1beta1 +kind: ClusterRoleBinding +metadata: + name: pytorch-operator + labels: + app.kubernetes.io/name: pytorch-operator + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: pytorch-operator +subjects: +- kind: ServiceAccount + name: pytorch-operator + namespace: pytorch-operator +--- +# Source: flyte/charts/sparkoperator/templates/spark-rbac.yaml +apiVersion: rbac.authorization.k8s.io/v1 +kind: Role +metadata: + name: spark-role + namespace: default + labels: + helm.sh/chart: sparkoperator-1.0.6 + app.kubernetes.io/name: sparkoperator + app.kubernetes.io/instance: flyte + app.kubernetes.io/version: "v1beta2-1.2.0-3.0.0" + app.kubernetes.io/managed-by: Helm +rules: +- apiGroups: + - "" + resources: + - pods + verbs: + - "*" +- apiGroups: + - "" + resources: + - services + verbs: + - "*" +--- +# Source: flyte/charts/sparkoperator/templates/spark-rbac.yaml +apiVersion: rbac.authorization.k8s.io/v1 +kind: RoleBinding +metadata: + name: spark + namespace: default + labels: + helm.sh/chart: sparkoperator-1.0.6 + app.kubernetes.io/name: sparkoperator + app.kubernetes.io/instance: flyte + app.kubernetes.io/version: "v1beta2-1.2.0-3.0.0" + app.kubernetes.io/managed-by: Helm +subjects: +- kind: ServiceAccount + name: flyte-spark + namespace: default +roleRef: + kind: Role + name: spark-role + apiGroup: rbac.authorization.k8s.io +--- +# Source: flyte/templates/admin/service.yaml +apiVersion: v1 +kind: Service +metadata: + name: flyteadmin + namespace: default + labels: + app.kubernetes.io/name: flyteadmin + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm + annotations: + external-dns.alpha.kubernetes.io/hostname: flyte.example.com + projectcontour.io/upstream-protocol.h2c: grpc + service.beta.kubernetes.io/aws-load-balancer-connection-idle-timeout: "600" +spec: + type: LoadBalancer + loadBalancerSourceRanges: + [0.0.0.0/0] + ports: + - name: http + port: 80 + protocol: TCP + targetPort: 8088 + - name: grpc + port: 81 + protocol: TCP + targetPort: 8089 + - name: redoc + protocol: TCP + port: 87 + targetPort: 8087 + - name: http-metrics + protocol: TCP + port: 10254 + selector: + app.kubernetes.io/name: flyteadmin + app.kubernetes.io/instance: flyte +--- +# Source: flyte/templates/console/service.yaml +apiVersion: v1 +kind: Service +metadata: + name: flyteconsole + namespace: default + labels: + app.kubernetes.io/name: flyteconsole + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm + annotations: + external-dns.alpha.kubernetes.io/hostname: flyte.example.com + service.beta.kubernetes.io/aws-load-balancer-connection-idle-timeout: "600" +spec: + type: LoadBalancer + ports: + - port: 80 + protocol: TCP + targetPort: 8080 + selector: + app.kubernetes.io/name: flyteconsole + app.kubernetes.io/instance: flyte +--- +# Source: flyte/templates/datacatalog/service.yaml +apiVersion: v1 +kind: Service +metadata: + name: datacatalog + namespace: default + labels: + app.kubernetes.io/name: datacatalog + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm + annotations: + projectcontour.io/upstream-protocol.h2c: grpc + service.beta.kubernetes.io/aws-load-balancer-connection-idle-timeout: "600" +spec: + type: LoadBalancer + ports: + - name: grpc-2 + port: 8089 + protocol: TCP + targetPort: 8089 + - name: http + port: 88 + protocol: TCP + targetPort: 8088 + - name: grpc + port: 89 + protocol: TCP + targetPort: 8089 + selector: + app.kubernetes.io/name: datacatalog + app.kubernetes.io/instance: flyte +--- +# Source: flyte/templates/propeller/webhook.yaml +# Service +apiVersion: v1 +kind: Service +metadata: + name: flyte-pod-webhook + namespace: default + annotations: + projectcontour.io/upstream-protocol.h2c: grpc +spec: + selector: + app: flyte-pod-webhook + ports: + - name: https + protocol: TCP + port: 443 + targetPort: 9443 +--- +# Source: flyte/templates/pytorch-operator/service.yaml +apiVersion: v1 +kind: Service +metadata: + name: pytorch-operator + namespace: pytorch-operator + labels: + app.kubernetes.io/name: pytorch-operator + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm +spec: + type: ClusterIP + ports: + - name: monitoring-port + port: 8443 + targetPort: 8443 + selector: + app.kubernetes.io/name: pytorch-operator + app.kubernetes.io/instance: flyte +--- +# Source: flyte/templates/redis/service.yaml +apiVersion: v1 +kind: Service +metadata: + name: redis-resource-manager + namespace: default + labels: + app.kubernetes.io/name: redis + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm +spec: + type: ClusterIP + ports: + - name: redis + port: 6379 + protocol: TCP + targetPort: redis + selector: + app.kubernetes.io/name: redis + app.kubernetes.io/instance: flyte +--- +# Source: flyte/charts/sparkoperator/templates/deployment.yaml +# If the admission webhook is enabled, then a post-install step is required +# to generate and install the secret in the operator namespace. + +# In the post-install hook, the token corresponding to the operator service account +# is used to authenticate with the Kubernetes API server to install the secret bundle. + +apiVersion: apps/v1 +kind: Deployment +metadata: + name: flyte-sparkoperator + labels: + helm.sh/chart: sparkoperator-1.0.6 + app.kubernetes.io/name: sparkoperator + app.kubernetes.io/instance: flyte + app.kubernetes.io/version: "v1beta2-1.2.0-3.0.0" + app.kubernetes.io/managed-by: Helm +spec: + replicas: 1 + selector: + matchLabels: + app.kubernetes.io/name: sparkoperator + app.kubernetes.io/instance: flyte + strategy: + type: Recreate + template: + metadata: + annotations: + prometheus.io/scrape: "true" + prometheus.io/port: "10254" + prometheus.io/path: /metrics + labels: + app.kubernetes.io/name: sparkoperator + app.kubernetes.io/instance: flyte + spec: + serviceAccountName: flyte-sparkoperator + securityContext: + {} + containers: + - name: sparkoperator + image: gcr.io/spark-operator/spark-operator:v1beta2-1.2.0-3.0.0 + imagePullPolicy: IfNotPresent + securityContext: + {} + ports: + - name: "metrics" + containerPort: 10254 + + args: + - -v=2 + - -logtostderr + - -namespace= + - -ingress-url-format= + - -controller-threads=10 + - -resync-interval=30 + - -enable-batch-scheduler=false + - -enable-metrics=true + - -metrics-labels=app_type + - -metrics-port=10254 + - -metrics-endpoint=/metrics + - -metrics-prefix= + - -enable-resource-quota-enforcement=false + resources: + limits: + cpu: 1000m + memory: 1000Mi + requests: + cpu: 50m + memory: 250Mi +--- +# Source: flyte/templates/admin/deployment.yaml +apiVersion: apps/v1 +kind: Deployment +metadata: + name: flyteadmin + namespace: default + labels: + app.kubernetes.io/name: flyteadmin + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm +spec: + replicas: 2 + selector: + matchLabels: + app.kubernetes.io/name: flyteadmin + app.kubernetes.io/instance: flyte + template: + metadata: + annotations: + configChecksum: "2a74fd52cd1aea9ab4a05907782a3f50be7777930a292f8c74de653dc71ff19" + labels: + app.kubernetes.io/name: flyteadmin + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm + spec: + initContainers: + - command: + - flyteadmin + - --config + - /etc/flyte/config/*.yaml + - migrate + - run + image: "ghcr.io/flyteorg/flyteadmin:v0.5.7" + imagePullPolicy: "IfNotPresent" + name: run-migrations + volumeMounts: + - mountPath: /etc/db + name: db-pass + - mountPath: /etc/flyte/config + name: config-volume + - command: + - flyteadmin + - --config + - /etc/flyte/config/*.yaml + - migrate + - seed-projects + - flytesnacks + - flytetester + - flyteexamples + image: "ghcr.io/flyteorg/flyteadmin:v0.5.7" + imagePullPolicy: "IfNotPresent" + name: seed-projects + volumeMounts: + - mountPath: /etc/db + name: db-pass + - mountPath: /etc/flyte/config + name: config-volume + - command: + - flyteadmin + - --config + - /etc/flyte/config/*.yaml + - clusterresource + - sync + image: "ghcr.io/flyteorg/flyteadmin:v0.5.7" + imagePullPolicy: "IfNotPresent" + name: sync-cluster-resources + volumeMounts: + - mountPath: /etc/db + name: db-pass + - mountPath: /etc/flyte/clusterresource/templates + name: resource-templates + - mountPath: /etc/flyte/config + name: config-volume + - name: generate-secrets + image: "ghcr.io/flyteorg/flyteadmin:v0.5.7" + imagePullPolicy: "IfNotPresent" + command: ["/bin/sh", "-c"] + args: + [ + "flyteadmin --config=/etc/flyte/config/*.yaml secrets init --localPath /etc/secrets/auth && flyteadmin --config=/etc/flyte/config/*.yaml secrets create --fromPath /etc/secrets/auth", + ] + volumeMounts: + - name: config-volume + mountPath: /etc/flyte/config + env: + - name: POD_NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace + containers: + - command: + - flyteadmin + - --config + - /etc/flyte/config/*.yaml + - serve + image: "ghcr.io/flyteorg/flyteadmin:v0.5.7" + imagePullPolicy: "IfNotPresent" + name: flyteadmin + ports: + - containerPort: 8088 + - containerPort: 8089 + resources: + limits: + cpu: 250m + ephemeral-storage: 200Mi + memory: 500Mi + requests: + cpu: 50m + ephemeral-storage: 200Mi + memory: 200Mi + volumeMounts: + - mountPath: /etc/db + name: db-pass + - mountPath: /srv/flyte + name: shared-data + - mountPath: /etc/flyte/config + name: config-volume + - name: auth + mountPath: /etc/secrets/ + - command: + - sh + - -c + - ln -s /usr/share/nginx/html /usr/share/nginx/html/openapi && sh /usr/local/bin/docker-run.sh + env: + - name: PAGE_TITLE + value: Flyte Admin OpenAPI + - name: SPEC_URL + value: /api/v1/openapi + - name: PORT + value: "8087" + image: docker.io/redocly/redoc + imagePullPolicy: IfNotPresent + name: redoc + ports: + - containerPort: 8087 + resources: + limits: + cpu: "0.1" + memory: 200Mi + serviceAccountName: flyteadmin + volumes: + - name: db-pass + secret: + secretName: db-pass + - emptyDir: {} + name: shared-data + - configMap: + name: flyte-admin-config + name: config-volume + - configMap: + name: clusterresource-template + name: resource-templates + - name: auth + secret: + secretName: flyte-admin-auth + affinity: + podAntiAffinity: + requiredDuringSchedulingIgnoredDuringExecution: + - labelSelector: + matchLabels: + app.kubernetes.io/name: flyteadmin + topologyKey: kubernetes.io/hostname +--- +# Source: flyte/templates/console/deployment.yaml +apiVersion: apps/v1 +kind: Deployment +metadata: + name: flyteconsole + namespace: default + labels: + app.kubernetes.io/name: flyteconsole + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm +spec: + replicas: 2 + selector: + matchLabels: + app.kubernetes.io/name: flyteconsole + app.kubernetes.io/instance: flyte + template: + metadata: + annotations: + configChecksum: "a14c2477f002d1392494a730c37ba7109e61df5cd417d431efa36841ee56bb9" + labels: + app.kubernetes.io/name: flyteconsole + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm + spec: + containers: + - image: "ghcr.io/flyteorg/flyteconsole:v0.20.0" + imagePullPolicy: "IfNotPresent" + name: flyteconsole + envFrom: + - configMapRef: + name: flyte-console-config + ports: + - containerPort: 8080 + resources: + limits: + cpu: 250m + memory: 250Mi + requests: + cpu: 10m + memory: 50Mi + volumeMounts: + - mountPath: /srv/flyte + name: shared-data + volumes: + - emptyDir: {} + name: shared-data + affinity: + podAntiAffinity: + requiredDuringSchedulingIgnoredDuringExecution: + - labelSelector: + matchLabels: + app.kubernetes.io/name: flyteconsole + topologyKey: kubernetes.io/hostname +--- +# Source: flyte/templates/datacatalog/deployment.yaml +apiVersion: apps/v1 +kind: Deployment +metadata: + name: datacatalog + namespace: default + labels: + app.kubernetes.io/name: datacatalog + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm +spec: + replicas: 2 + selector: + matchLabels: + app.kubernetes.io/name: datacatalog + app.kubernetes.io/instance: flyte + template: + metadata: + annotations: + configChecksum: "735c43c585eda7d13e8654c91f0cada88063c80535274f5778bc3d7c1bc5545" + labels: + app.kubernetes.io/name: datacatalog + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm + spec: + initContainers: + - command: + - datacatalog + - --config + - /etc/datacatalog/config/*.yaml + - migrate + - run + image: "ghcr.io/flyteorg/datacatalog:v0.3.4" + imagePullPolicy: "IfNotPresent" + name: run-migrations + volumeMounts: + - mountPath: /etc/db + name: db-pass + - mountPath: /etc/datacatalog/config + name: config-volume + containers: + - command: + - datacatalog + - --config + - /etc/datacatalog/config/*.yaml + - serve + image: "ghcr.io/flyteorg/datacatalog:v0.3.4" + imagePullPolicy: "IfNotPresent" + name: datacatalog + ports: + - containerPort: 8088 + - containerPort: 8089 + resources: + limits: + cpu: 500m + ephemeral-storage: 200Mi + memory: 500Mi + requests: + cpu: 50m + ephemeral-storage: 200Mi + memory: 200Mi + volumeMounts: + - mountPath: /etc/db + name: db-pass + - mountPath: /etc/datacatalog/config + name: config-volume + serviceAccountName: datacatalog + volumes: + - name: db-pass + secret: + secretName: db-pass + - emptyDir: {} + name: shared-data + - configMap: + name: datacatalog-config + name: config-volume + affinity: + podAntiAffinity: + requiredDuringSchedulingIgnoredDuringExecution: + - labelSelector: + matchLabels: + app.kubernetes.io/name: datacatalog + topologyKey: kubernetes.io/hostname +--- +# Source: flyte/templates/propeller/deployment.yaml +apiVersion: apps/v1 +kind: Deployment +metadata: + name: flytepropeller + namespace: default + labels: + app.kubernetes.io/name: flytepropeller + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm +spec: + replicas: 2 + selector: + matchLabels: + app.kubernetes.io/name: flytepropeller + app.kubernetes.io/instance: flyte + template: + metadata: + annotations: + configChecksum: "73d82b14cc9e0ed31b4c5731396ec0a43727ceb7cf3c247c0e3c9023ce1cda2" + labels: + app.kubernetes.io/name: flytepropeller + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm + spec: + containers: + - command: + - flytepropeller + - --config + - /etc/flyte/config/*.yaml + env: + - name: POD_NAME + valueFrom: + fieldRef: + fieldPath: metadata.name + image: "ghcr.io/flyteorg/flytepropeller:v0.10.10" + imagePullPolicy: "IfNotPresent" + name: flytepropeller + ports: + - containerPort: 10254 + resources: + limits: + cpu: 1 + ephemeral-storage: 1Gi + memory: 2Gi + requests: + cpu: 1 + ephemeral-storage: 1Gi + memory: 2Gi + volumeMounts: + - name: config-volume + mountPath: /etc/flyte/config + - name: auth + mountPath: /etc/secrets/ + serviceAccountName: flytepropeller + volumes: + - configMap: + name: flyte-propeller-config + name: config-volume + - name: auth + secret: + secretName: flyte-propeller-auth + affinity: + podAntiAffinity: + requiredDuringSchedulingIgnoredDuringExecution: + - labelSelector: + matchLabels: + app.kubernetes.io/name: flytepropeller + topologyKey: kubernetes.io/hostname +--- +# Source: flyte/templates/propeller/webhook.yaml +# Create the actual deployment +apiVersion: apps/v1 +kind: Deployment +metadata: + name: flyte-pod-webhook + namespace: default + labels: + app: flyte-pod-webhook +spec: + selector: + matchLabels: + app: flyte-pod-webhook + template: + metadata: + labels: + app: flyte-pod-webhook + app.kubernetes.io/name: flyte-pod-webhook + app.kubernetes.io/version: v0.10.10 + annotations: + configChecksum: "73d82b14cc9e0ed31b4c5731396ec0a43727ceb7cf3c247c0e3c9023ce1cda2" + spec: + serviceAccountName: flyte-pod-webhook + initContainers: + - name: generate-secrets + image: "ghcr.io/flyteorg/flytepropeller:v0.10.10" + imagePullPolicy: "IfNotPresent" + command: + - flytepropeller + args: + - webhook + - init-certs + - --config + - /etc/flyte/config/*.yaml + env: + - name: POD_NAME + valueFrom: + fieldRef: + fieldPath: metadata.name + - name: POD_NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace + volumeMounts: + - name: config-volume + mountPath: /etc/flyte/config + containers: + - name: webhook + image: "ghcr.io/flyteorg/flytepropeller:v0.10.10" + imagePullPolicy: "IfNotPresent" + command: + - flytepropeller + args: + - webhook + - --config + - /etc/flyte/config/*.yaml + env: + - name: POD_NAME + valueFrom: + fieldRef: + fieldPath: metadata.name + - name: POD_NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace + volumeMounts: + - name: config-volume + mountPath: /etc/flyte/config + readOnly: true + - name: webhook-certs + mountPath: /etc/webhook/certs + readOnly: true + volumes: + - name: config-volume + configMap: + name: flyte-propeller-config + - name: webhook-certs + secret: + secretName: flyte-pod-webhook +--- +# Source: flyte/templates/pytorch-operator/deployment.yaml +apiVersion: apps/v1 +kind: Deployment +metadata: + name: pytorch-operator + namespace: pytorch-operator + labels: + app.kubernetes.io/name: pytorch-operator + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm +spec: + replicas: 1 + selector: + matchLabels: + app.kubernetes.io/name: pytorch-operator + app.kubernetes.io/instance: flyte + template: + metadata: + labels: + app.kubernetes.io/name: pytorch-operator + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm + spec: + containers: + - image: "gcr.io/kubeflow-images-public/pytorch-operator:v1.0.0-g047cf0f" + imagePullPolicy: "IfNotPresent" + name: pytorch-operator + command: + - /pytorch-operator.v1 + - --alsologtostderr + - -v=1 + - --monitoring-port=8443 + env: + - name: MY_POD_NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace + - name: MY_POD_NAME + valueFrom: + fieldRef: + fieldPath: metadata.name + resources: + limits: + cpu: 1000m + memory: 1000Mi + requests: + cpu: 50m + memory: 250Mi + serviceAccountName: pytorch-operator +--- +# Source: flyte/templates/redis/statefulset.yaml +apiVersion: apps/v1 +kind: StatefulSet +metadata: + name: redis + namespace: default + labels: + app.kubernetes.io/name: redis + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm +spec: + replicas: 1 + selector: + matchLabels: + app.kubernetes.io/name: redis + app.kubernetes.io/instance: flyte + serviceName: redis-resource-manager + template: + metadata: + labels: + app.kubernetes.io/name: redis + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm + spec: + containers: + - image: "docker.io/bitnami/redis:4.0.2-r1" + imagePullPolicy: "IfNotPresent" + name: redis + env: + - name: REDIS_PASSWORD + value: mypassword + livenessProbe: + exec: + command: + - redis-cli + - ping + failureThreshold: 3 + initialDelaySeconds: 30 + periodSeconds: 10 + successThreshold: 1 + timeoutSeconds: 5 + readinessProbe: + exec: + command: + - redis-cli + - ping + failureThreshold: 3 + initialDelaySeconds: 5 + periodSeconds: 10 + successThreshold: 1 + timeoutSeconds: 1 + ports: + - containerPort: 6379 + name: redis + protocol: TCP + resources: + limits: + cpu: 1000m + memory: 1Gi + requests: + cpu: 100m + memory: 250Mi + volumeMounts: + - mountPath: /bitnami + name: redis-data + + dnsPolicy: ClusterFirst + restartPolicy: Always + volumes: + - emptyDir: {} + name: redis-data +--- +# Source: flyte/templates/admin/cronjob.yaml +apiVersion: batch/v1beta1 +kind: CronJob +metadata: + name: syncresources + namespace: default + labels: + app.kubernetes.io/name: flyteadmin + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm +spec: + schedule: '*/1 * * * *' + jobTemplate: + spec: + template: + spec: + containers: + - command: + - flyteadmin + - --config + - /etc/flyte/config/*.yaml + - clusterresource + - sync + image: "ghcr.io/flyteorg/flyteadmin:v0.5.7" + imagePullPolicy: "IfNotPresent" + name: sync-cluster-resources + volumeMounts: + - mountPath: /etc/db + name: db-pass + - mountPath: /etc/flyte/clusterresource/templates + name: resource-templates + - mountPath: /etc/flyte/config + name: config-volume + restartPolicy: OnFailure + serviceAccountName: flyteadmin + volumes: + - name: db-pass + secret: + secretName: db-pass + - configMap: + name: clusterresource-template + name: resource-templates + - configMap: + name: flyte-admin-config + name: config-volume +--- +# Source: flyte/templates/common/ingress.yaml +apiVersion: networking.k8s.io/v1beta1 +kind: Ingress +metadata: + name: flyte + namespace: default + annotations: + alb.ingress.kubernetes.io/actions.ssl-redirect: '{"Type": "redirect", "RedirectConfig": { "Protocol": "HTTPS", "Port": "443", "StatusCode": "HTTP_301"}}' + alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:us-east-2::certificate/e2f04275-2dff-4118-a493-ed3ec8f41605 + alb.ingress.kubernetes.io/group.name: flyte + alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}, {"HTTPS":443}]' + alb.ingress.kubernetes.io/scheme: internet-facing + alb.ingress.kubernetes.io/tags: service_instance=production + kubernetes.io/ingress.class: alb +spec: + rules: + - http: + paths: + - path: /* + pathType: ImplementationSpecific + backend: + serviceName: ssl-redirect + servicePort: use-annotation + # This is useful only for frontend development + # Port 87 in FlyteAdmin maps to the redoc container. + - path: /openapi + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 87 + # NOTE: If you change this, you must update the BASE_URL value in flyteconsole.yaml + - path: /console + pathType: ImplementationSpecific + backend: + serviceName: flyteconsole + servicePort: 80 + - path: /console/* + pathType: ImplementationSpecific + backend: + serviceName: flyteconsole + servicePort: 80 + - path: /api + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 80 + - path: /api/* + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 80 + - path: /healthcheck + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 80 + - path: /v1/* + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 80 + # Port 87 in FlyteAdmin maps to the redoc container. + - path: /openapi/* + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 80 + - path: /.well-known + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 80 + - path: /.well-known/* + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 80 + - path: /login + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 80 + - path: /login/* + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 80 + - path: /logout + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 80 + - path: /logout/* + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 80 + - path: /callback + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 80 + - path: /callback/* + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 80 + - path: /me + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 80 + - path: /config + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 80 + - path: /config/* + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 80 + host: flyte.example.com +# Certain ingress controllers like nginx cannot serve HTTP 1 and GRPC with a single ingress because GRPC can only +# enabled on the ingress object, not on backend services (GRPC annotation is set on the ingress, not on the services). +--- +# Source: flyte/templates/common/ingress.yaml +apiVersion: networking.k8s.io/v1beta1 +kind: Ingress +metadata: + name: flyte-grpc + namespace: default + annotations: + alb.ingress.kubernetes.io/actions.ssl-redirect: '{"Type": "redirect", "RedirectConfig": { "Protocol": "HTTPS", "Port": "443", "StatusCode": "HTTP_301"}}' + alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:us-east-2::certificate/e2f04275-2dff-4118-a493-ed3ec8f41605 + alb.ingress.kubernetes.io/group.name: flyte + alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}, {"HTTPS":443}]' + alb.ingress.kubernetes.io/scheme: internet-facing + alb.ingress.kubernetes.io/tags: service_instance=production + kubernetes.io/ingress.class: alb + nginx.ingress.kubernetes.io/backend-protocol: GRPC +spec: + rules: + - http: + paths: + # + # - backend: + # serviceName: ssl-redirect + # servicePort: use-annotation + # path: /* + # pathType: ImplementationSpecific + # + # NOTE: Port 81 in flyteadmin is the GRPC server port for FlyteAdmin. + - path: /flyteidl.service.AdminService + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 81 + - path: /flyteidl.service.AdminService/* + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 81 + - path: /flyteidl.service.AuthMetadataService + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 81 + - path: /flyteidl.service.AuthMetadataService/* + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 81 + - path: /flyteidl.service.IdentityService + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 81 + - path: /flyteidl.service.IdentityService/* + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 81 + host: flyte.example.com diff --git a/deployment/gcp/flyte_helm_generated.yaml b/deployment/gcp/flyte_helm_generated.yaml new file mode 100644 index 0000000000..10d17c80ec --- /dev/null +++ b/deployment/gcp/flyte_helm_generated.yaml @@ -0,0 +1,4066 @@ +--- +# Source: flyte/templates/pytorch-operator/namespace.yaml +apiVersion: v1 +kind: Namespace +metadata: + name: pytorch-operator +--- +# Source: flyte/charts/contour/templates/contour/serviceaccount.yaml +apiVersion: v1 +kind: ServiceAccount +metadata: + name: flyte-contour-contour + namespace: default + labels: + app.kubernetes.io/name: contour + helm.sh/chart: contour-4.1.2 + app.kubernetes.io/instance: flyte + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/component: contour +--- +# Source: flyte/charts/contour/templates/envoy/serviceaccount.yaml +apiVersion: v1 +kind: ServiceAccount +metadata: + name: flyte-contour-envoy + namespace: default + labels: + app.kubernetes.io/name: contour + helm.sh/chart: contour-4.1.2 + app.kubernetes.io/instance: flyte + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/component: envoy +--- +# Source: flyte/charts/sparkoperator/templates/serviceaccount.yaml +apiVersion: v1 +kind: ServiceAccount +metadata: + name: flyte-sparkoperator + labels: + helm.sh/chart: sparkoperator-1.0.6 + app.kubernetes.io/name: sparkoperator + app.kubernetes.io/instance: flyte + app.kubernetes.io/version: "v1beta2-1.2.0-3.0.0" + app.kubernetes.io/managed-by: Helm +--- +# Source: flyte/charts/sparkoperator/templates/spark-serviceaccount.yaml +apiVersion: v1 +kind: ServiceAccount +metadata: + name: flyte-spark + namespace: default + labels: + helm.sh/chart: sparkoperator-1.0.6 + app.kubernetes.io/name: sparkoperator + app.kubernetes.io/instance: flyte + app.kubernetes.io/version: "v1beta2-1.2.0-3.0.0" + app.kubernetes.io/managed-by: Helm +--- +# Source: flyte/templates/admin/rbac.yaml +apiVersion: v1 +kind: ServiceAccount +metadata: + name: flyteadmin + namespace: default + labels: + app.kubernetes.io/name: flyteadmin + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm +--- +# Source: flyte/templates/datacatalog/rbac.yaml +apiVersion: v1 +kind: ServiceAccount +metadata: + name: datacatalog + namespace: default + labels: + app.kubernetes.io/name: datacatalog + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm +--- +# Source: flyte/templates/propeller/rbac.yaml +apiVersion: v1 +kind: ServiceAccount +metadata: + name: flytepropeller + namespace: default + labels: + app.kubernetes.io/name: flytepropeller + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm +--- +# Source: flyte/templates/propeller/webhook.yaml +# Create a Service Account for webhook +apiVersion: v1 +kind: ServiceAccount +metadata: + name: flyte-pod-webhook + namespace: default +--- +# Source: flyte/templates/pytorch-operator/rbac.yaml +apiVersion: v1 +kind: ServiceAccount +metadata: + name: pytorch-operator + namespace: pytorch-operator + labels: + app.kubernetes.io/name: pytorch-operator + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm +--- +# Source: flyte/templates/admin/secret-auth.yaml +apiVersion: v1 +kind: Secret +metadata: + name: flyte-admin-auth + namespace: default +type: Opaque +--- +# Source: flyte/templates/propeller/secret-auth.yaml +apiVersion: v1 +kind: Secret +metadata: + name: flyte-propeller-auth + namespace: default +type: Opaque +stringData: + client_secret: foobar +--- +# Source: flyte/templates/propeller/webhook.yaml +# Create an empty secret that the first propeller pod will populate +apiVersion: v1 +kind: Secret +metadata: + name: flyte-pod-webhook + namespace: default +type: Opaque +--- +# Source: flyte/charts/contour/templates/contour/configmap.yaml +apiVersion: v1 +kind: ConfigMap +metadata: + name: flyte-contour + namespace: default + labels: + app.kubernetes.io/name: contour + helm.sh/chart: contour-4.1.2 + app.kubernetes.io/instance: flyte + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/component: contour +data: + contour.yaml: | + accesslog-format: envoy + disablePermitInsecure: false + envoy-service-name: 'flyte-contour-envoy' + leaderelection: + configmap-namespace: 'default' + tls: {} +--- +# Source: flyte/templates/admin/cluster_resource_configmap.yaml +apiVersion: v1 +kind: ConfigMap +metadata: + name: clusterresource-template + namespace: default + labels: + app.kubernetes.io/name: flyteadmin + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm +data: + aa_namespace.yaml: | + apiVersion: v1 + kind: Namespace + metadata: + name: {{ namespace }} + spec: + finalizers: + - kubernetes + + ab_project_resource_quota.yaml: | + apiVersion: v1 + kind: ResourceQuota + metadata: + name: project-quota + namespace: {{ namespace }} + spec: + hard: + limits.cpu: {{ projectQuotaCpu }} + limits.memory: {{ projectQuotaMemory }} + + zz_copilot_config.yaml: | + kind: ConfigMap + apiVersion: v1 + metadata: + name: flyte-data-config + namespace: {{ namespace }} + data: + config.yaml: | + storage: + type: stow + stow: + kind: google + config: + json: "" + project_id: + scopes: https://www.googleapis.com/auth/devstorage.read_write + container: "" + enable-multicontainer: true +--- +# Source: flyte/templates/admin/configmap.yaml +apiVersion: v1 +kind: ConfigMap +metadata: + name: flyte-admin-config + namespace: default + labels: + app.kubernetes.io/name: flyteadmin + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm +data: + db.yaml: | + database: {} + domain.yaml: | + domains: + - id: development + name: development + - id: staging + name: staging + - id: production + name: production + logger.yaml: | + logger: + level: 4 + show-source: true + server.yaml: | + auth: + appAuth: + thirdPartyConfig: + flyteClient: + clientId: flytectl + redirectUri: https://localhost:53593/callback + scopes: + - offline + - all + authorizedUris: + - https://localhost:30081 + - http://flyteadmin:80 + - http://flyteadmin.flyte.svc.cluster.local:80 + userAuth: + openId: + baseUrl: https://accounts.google.com + clientId: 657465813211-6eog7ek7li5k7i7fvgv2921075063hpe.apps.googleusercontent.com + scopes: + - profile + - openid + flyteadmin: + eventVersion: 1 + metadataStoragePrefix: + - metadata + - admin + metricsScope: 'flyte:' + profilerPort: 10254 + roleNameKey: iam.amazonaws.com/role + testing: + host: http://flyteadmin + server: + grpcPort: 8089 + httpPort: 8088 + security: + allowCors: true + allowedHeaders: + - Content-Type + - flyte-authorization + allowedOrigins: + - '*' + secure: false + useAuth: false + remoteData.yaml: | + remoteData: + region: us-east-1 + scheme: gcp + signedUrls: + durationMinutes: 3 + storage.yaml: | + storage: + type: stow + stow: + kind: google + config: + json: "" + project_id: + scopes: https://www.googleapis.com/auth/devstorage.read_write + container: "" + limits: + maxDownloadMBs: 10 + task_resource_defaults.yaml: | + task_resources: + defaults: + cpu: 100m + memory: 100Mi + storage: 5Mi + limits: + cpu: 2 + gpu: 1 + memory: 8Gi + storage: 20Mi + cluster_resources.yaml: | + cluster_resources: + customData: + - production: + - projectQuotaCpu: + value: "5" + - projectQuotaMemory: + value: 4000Mi + - staging: + - projectQuotaCpu: + value: "2" + - projectQuotaMemory: + value: 3000Mi + - development: + - projectQuotaCpu: + value: "4" + - projectQuotaMemory: + value: 3000Mi + refresh: 5m + templatePath: /etc/flyte/clusterresource/templates +--- +# Source: flyte/templates/console/configmap.yaml +apiVersion: v1 +kind: ConfigMap +metadata: + name: flyte-console-config + namespace: default + labels: + app.kubernetes.io/name: flyteconsole + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm +data: + BASE_URL: /console + CONFIG_DIR: /etc/flyte/config + DISABLE_AUTH: "1" +--- +# Source: flyte/templates/datacatalog/configmap.yaml +apiVersion: v1 +kind: ConfigMap +metadata: + name: datacatalog-config + namespace: default + labels: + app.kubernetes.io/name: datacatalog + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm +data: + db.yaml: | + database: {} + logger.yaml: | + logger: + level: 4 + show-source: true + server.yaml: | + application: + grpcPort: 8089 + grpcServerReflection: true + httpPort: 8080 + datacatalog: + metrics-scope: datacatalog + profiler-port: 10254 + storage-prefix: metadata/datacatalog + storage.yaml: | + storage: + type: stow + stow: + kind: google + config: + json: "" + project_id: + scopes: https://www.googleapis.com/auth/devstorage.read_write + container: "" + limits: + maxDownloadMBs: 10 +--- +# Source: flyte/templates/propeller/configmap.yaml +apiVersion: v1 +kind: ConfigMap +metadata: + name: flyte-propeller-config + namespace: default + labels: + app.kubernetes.io/name: flyteadmin + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm +data: + admin.yaml: | + admin: + clientId: flytepropeller + clientSecretLocation: /etc/secrets/client_secret + endpoint: flyteadmin:81 + insecure: true + event: + capacity: 1000 + rate: 500 + type: admin + catalog.yaml: | + catalog-cache: + endpoint: datacatalog:89 + insecure: true + type: datacatalog + copilot.yaml: | + plugins: + k8s: + co-pilot: + image: ghcr.io/lyft/flyteplugins/flytecopilot:dc4bdbd61cac88a39a5ff43e40f026bdbc2c78a2 + name: flyte-copilot- + start-timeout: 30s + core.yaml: | + propeller: + downstream-eval-duration: 30s + enable-admin-launcher: true + leader-election: + enabled: true + lease-duration: 15s + lock-config-map: + name: propeller-leader + namespace: flyte + renew-deadline: 10s + retry-period: 2s + limit-namespace: all + max-workflow-retries: 30 + metadata-prefix: metadata/propeller + metrics-prefix: flyte + prof-port: 10254 + queue: + batch-size: -1 + batching-interval: 2s + queue: + base-delay: 5s + capacity: 1000 + max-delay: 120s + rate: 100 + type: maxof + sub-queue: + capacity: 100 + rate: 10 + type: bucket + type: batch + rawoutput-prefix: s3://my-s3-bucket/ + workers: 4 + workflow-reeval-duration: 30s + webhook: + certDir: /etc/webhook/certs + serviceName: flyte-pod-webhook + enabled_plugins.yaml: | + tasks: + task-plugins: + default-for-task-types: + container: container + container_array: k8s-array + sidecar: sidecar + enabled-plugins: + - container + - sidecar + - k8s-array + k8s.yaml: | + plugins: + k8s: + default-cpus: 100m + default-env-vars: [] + default-memory: 100Mi + logger.yaml: | + logger: + level: 4 + show-source: true + resource_manager.yaml: | + propeller: + resourcemanager: + redis: + hostKey: mypassword + hostPath: redis-resource-manager:6379 + resourceMaxQuota: 10000 + type: redis + spark.yaml: | + plugins: + spark: + spark-config-default: + - spark.hadoop.mapreduce.fileoutputcommitter.algorithm.version: "2" + - spark.kubernetes.allocation.batch.size: "50" + - spark.hadoop.fs.s3a.acl.default: BucketOwnerFullControl + - spark.hadoop.fs.s3n.impl: org.apache.hadoop.fs.s3a.S3AFileSystem + - spark.hadoop.fs.AbstractFileSystem.s3n.impl: org.apache.hadoop.fs.s3a.S3A + - spark.hadoop.fs.s3.impl: org.apache.hadoop.fs.s3a.S3AFileSystem + - spark.hadoop.fs.AbstractFileSystem.s3.impl: org.apache.hadoop.fs.s3a.S3A + - spark.hadoop.fs.s3a.impl: org.apache.hadoop.fs.s3a.S3AFileSystem + - spark.hadoop.fs.AbstractFileSystem.s3a.impl: org.apache.hadoop.fs.s3a.S3A + - spark.hadoop.fs.s3a.multipart.threshold: "536870912" + - spark.blacklist.enabled: "true" + - spark.blacklist.timeout: 5m + - spark.task.maxfailures: "8" + storage.yaml: | + storage: + type: stow + stow: + kind: google + config: + json: "" + project_id: + scopes: https://www.googleapis.com/auth/devstorage.read_write + container: "" + limits: + maxDownloadMBs: 10 + cache.yaml: | + cache: + max_size_mbs: 0 + target_gc_percent: 70 + task_logs.yaml: | + plugins: + logs: + cloudwatch-enabled: false + kubernetes-enabled: false +--- +# Source: flyte/charts/contour/templates/00-crds.yaml +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.2.9 + creationTimestamp: null + name: extensionservices.projectcontour.io +spec: + preserveUnknownFields: false + group: projectcontour.io + names: + kind: ExtensionService + listKind: ExtensionServiceList + plural: extensionservices + shortNames: + - extensionservice + - extensionservices + singular: extensionservice + scope: Namespaced + versions: + - name: v1alpha1 + schema: + openAPIV3Schema: + description: ExtensionService is the schema for the Contour extension services API. An ExtensionService resource binds a network service to the Contour API so that Contour API features can be implemented by collaborating components. + properties: + apiVersion: + description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' + type: string + kind: + description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + type: string + metadata: + type: object + spec: + description: ExtensionServiceSpec defines the desired state of an ExtensionService resource. + properties: + loadBalancerPolicy: + description: The policy for load balancing GRPC service requests. Note that the `Cookie` load balancing strategy cannot be used here. + properties: + strategy: + description: Strategy specifies the policy used to balance requests across the pool of backend pods. Valid policy names are `Random`, `RoundRobin`, `WeightedLeastRequest`, `Random` and `Cookie`. If an unknown strategy name is specified or no policy is supplied, the default `RoundRobin` policy is used. + type: string + type: object + protocol: + description: Protocol may be used to specify (or override) the protocol used to reach this Service. Values may be tls, h2, h2c. If omitted, protocol-selection falls back on Service annotations. + enum: + - h2 + - h2c + type: string + protocolVersion: + description: This field sets the version of the GRPC protocol that Envoy uses to send requests to the extension service. Since Contour always uses the v3 Envoy API, this is currently fixed at "v3". However, other protocol options will be available in future. + enum: + - v3 + type: string + services: + description: Services specifies the set of Kubernetes Service resources that receive GRPC extension API requests. If no weights are specified for any of the entries in this array, traffic will be spread evenly across all the services. Otherwise, traffic is balanced proportionally to the Weight field in each entry. + items: + description: ExtensionServiceTarget defines an Kubernetes Service to target with extension service traffic. + properties: + name: + description: Name is the name of Kubernetes service that will accept service traffic. + type: string + port: + description: Port (defined as Integer) to proxy traffic to since a service can have multiple defined. + exclusiveMaximum: true + maximum: 65536 + minimum: 1 + type: integer + weight: + description: Weight defines proportion of traffic to balance to the Kubernetes Service. + format: int32 + type: integer + required: + - name + - port + type: object + minItems: 1 + type: array + timeoutPolicy: + description: The timeout policy for requests to the services. + properties: + idle: + description: Timeout after which, if there are no active requests for this route, the connection between Envoy and the backend or Envoy and the external client will be closed. If not specified, there is no per-route idle timeout, though a connection manager-wide stream_idle_timeout default of 5m still applies. + pattern: ^(((\d*(\.\d*)?h)|(\d*(\.\d*)?m)|(\d*(\.\d*)?s)|(\d*(\.\d*)?ms)|(\d*(\.\d*)?us)|(\d*(\.\d*)?µs)|(\d*(\.\d*)?ns))+|infinity|infinite)$ + type: string + response: + description: Timeout for receiving a response from the server after processing a request from client. If not supplied, Envoy's default value of 15s applies. + pattern: ^(((\d*(\.\d*)?h)|(\d*(\.\d*)?m)|(\d*(\.\d*)?s)|(\d*(\.\d*)?ms)|(\d*(\.\d*)?us)|(\d*(\.\d*)?µs)|(\d*(\.\d*)?ns))+|infinity|infinite)$ + type: string + type: object + validation: + description: UpstreamValidation defines how to verify the backend service's certificate + properties: + caSecret: + description: Name of the Kubernetes secret be used to validate the certificate presented by the backend + type: string + subjectName: + description: Key which is expected to be present in the 'subjectAltName' of the presented certificate + type: string + required: + - caSecret + - subjectName + type: object + required: + - services + type: object + status: + description: ExtensionServiceStatus defines the observed state of an ExtensionService resource. + properties: + conditions: + description: "Conditions contains the current status of the ExtensionService resource. \n Contour will update a single condition, `Valid`, that is in normal-true polarity. \n Contour will not modify any other Conditions set in this block, in case some other controller wants to add a Condition." + items: + description: "DetailedCondition is an extension of the normal Kubernetes conditions, with two extra fields to hold sub-conditions, which provide more detailed reasons for the state (True or False) of the condition. \n `errors` holds information about sub-conditions which are fatal to that condition and render its state False. \n `warnings` holds information about sub-conditions which are not fatal to that condition and do not force the state to be False. \n Remember that Conditions have a type, a status, and a reason. \n The type is the type of the condition, the most important one in this CRD set is `Valid`. `Valid` is a positive-polarity condition: when it is `status: true` there are no problems. \n In more detail, `status: true` means that the object is has been ingested into Contour with no errors. `warnings` may still be present, and will be indicated in the Reason field. There must be zero entries in the `errors` slice in this case. \n `Valid`, `status: false` means that the object has had one or more fatal errors during processing into Contour. The details of the errors will be present under the `errors` field. There must be at least one error in the `errors` slice if `status` is `false`. \n For DetailedConditions of types other than `Valid`, the Condition must be in the negative polarity. When they have `status` `true`, there is an error. There must be at least one entry in the `errors` Subcondition slice. When they have `status` `false`, there are no serious errors, and there must be zero entries in the `errors` slice. In either case, there may be entries in the `warnings` slice. \n Regardless of the polarity, the `reason` and `message` fields must be updated with either the detail of the reason (if there is one and only one entry in total across both the `errors` and `warnings` slices), or `MultipleReasons` if there is more than one entry." + properties: + errors: + description: "Errors contains a slice of relevant error subconditions for this object. \n Subconditions are expected to appear when relevant (when there is a error), and disappear when not relevant. An empty slice here indicates no errors." + items: + description: "SubCondition is a Condition-like type intended for use as a subcondition inside a DetailedCondition. \n It contains a subset of the Condition fields. \n It is intended for warnings and errors, so `type` names should use abnormal-true polarity, that is, they should be of the form \"ErrorPresent: true\". \n The expected lifecycle for these errors is that they should only be present when the error or warning is, and should be removed when they are not relevant." + properties: + message: + description: "Message is a human readable message indicating details about the transition. \n This may be an empty string." + maxLength: 32768 + type: string + reason: + description: "Reason contains a programmatic identifier indicating the reason for the condition's last transition. Producers of specific condition types may define expected values and meanings for this field, and whether the values are considered a guaranteed API. \n The value should be a CamelCase string. \n This field may not be empty." + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + description: Status of the condition, one of True, False, Unknown. + enum: + - "True" + - "False" + - Unknown + type: string + type: + description: "Type of condition in `CamelCase` or in `foo.example.com/CamelCase`. \n This must be in abnormal-true polarity, that is, `ErrorFound` or `controller.io/ErrorFound`. \n The regex it matches is (dns1123SubdomainFmt/)?(qualifiedNameFmt)" + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + required: + - message + - reason + - status + - type + type: object + type: array + lastTransitionTime: + description: lastTransitionTime is the last time the condition transitioned from one status to another. This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable. + format: date-time + type: string + message: + description: message is a human readable message indicating details about the transition. This may be an empty string. + maxLength: 32768 + type: string + observedGeneration: + description: observedGeneration represents the .metadata.generation that the condition was set based upon. For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date with respect to the current state of the instance. + format: int64 + minimum: 0 + type: integer + reason: + description: reason contains a programmatic identifier indicating the reason for the condition's last transition. Producers of specific condition types may define expected values and meanings for this field, and whether the values are considered a guaranteed API. The value should be a CamelCase string. This field may not be empty. + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + description: status of the condition, one of True, False, Unknown. + enum: + - "True" + - "False" + - Unknown + type: string + type: + description: type of condition in CamelCase or in foo.example.com/CamelCase. --- Many .condition.type values are consistent across resources like Available, but because arbitrary conditions can be useful (see .node.status.conditions), the ability to deconflict is important. The regex it matches is (dns1123SubdomainFmt/)?(qualifiedNameFmt) + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + warnings: + description: "Warnings contains a slice of relevant warning subconditions for this object. \n Subconditions are expected to appear when relevant (when there is a warning), and disappear when not relevant. An empty slice here indicates no warnings." + items: + description: "SubCondition is a Condition-like type intended for use as a subcondition inside a DetailedCondition. \n It contains a subset of the Condition fields. \n It is intended for warnings and errors, so `type` names should use abnormal-true polarity, that is, they should be of the form \"ErrorPresent: true\". \n The expected lifecycle for these errors is that they should only be present when the error or warning is, and should be removed when they are not relevant." + properties: + message: + description: "Message is a human readable message indicating details about the transition. \n This may be an empty string." + maxLength: 32768 + type: string + reason: + description: "Reason contains a programmatic identifier indicating the reason for the condition's last transition. Producers of specific condition types may define expected values and meanings for this field, and whether the values are considered a guaranteed API. \n The value should be a CamelCase string. \n This field may not be empty." + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + description: Status of the condition, one of True, False, Unknown. + enum: + - "True" + - "False" + - Unknown + type: string + type: + description: "Type of condition in `CamelCase` or in `foo.example.com/CamelCase`. \n This must be in abnormal-true polarity, that is, `ErrorFound` or `controller.io/ErrorFound`. \n The regex it matches is (dns1123SubdomainFmt/)?(qualifiedNameFmt)" + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + required: + - message + - reason + - status + - type + type: object + type: array + required: + - lastTransitionTime + - message + - reason + - status + - type + type: object + type: array + x-kubernetes-list-map-keys: + - type + x-kubernetes-list-type: map + type: object + type: object + served: true + storage: true + subresources: + status: {} +status: + acceptedNames: + kind: "" + plural: "" + conditions: [] + storedVersions: [] +--- +# Source: flyte/charts/contour/templates/00-crds.yaml +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.2.9 + creationTimestamp: null + name: httpproxies.projectcontour.io +spec: + preserveUnknownFields: false + group: projectcontour.io + names: + kind: HTTPProxy + listKind: HTTPProxyList + plural: httpproxies + shortNames: + - proxy + - proxies + singular: httpproxy + scope: Namespaced + versions: + - additionalPrinterColumns: + - description: Fully qualified domain name + jsonPath: .spec.virtualhost.fqdn + name: FQDN + type: string + - description: Secret with TLS credentials + jsonPath: .spec.virtualhost.tls.secretName + name: TLS Secret + type: string + - description: The current status of the HTTPProxy + jsonPath: .status.currentStatus + name: Status + type: string + - description: Description of the current status + jsonPath: .status.description + name: Status Description + type: string + name: v1 + schema: + openAPIV3Schema: + description: HTTPProxy is an Ingress CRD specification. + properties: + apiVersion: + description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' + type: string + kind: + description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + type: string + metadata: + type: object + spec: + description: HTTPProxySpec defines the spec of the CRD. + properties: + ggvar: + description: Testing new param for CRD + type: string + includes: + description: Includes allow for specific routing configuration to be included from another HTTPProxy, possibly in another namespace. + items: + description: Include describes a set of policies that can be applied to an HTTPProxy in a namespace. + properties: + conditions: + description: 'Conditions are a set of rules that are applied to included HTTPProxies. In effect, they are added onto the Conditions of included HTTPProxy Route structs. When applied, they are merged using AND, with one exception: There can be only one Prefix MatchCondition per Conditions slice. More than one Prefix, or contradictory Conditions, will make the include invalid.' + items: + description: MatchCondition are a general holder for matching rules for HTTPProxies. One of Prefix or Header must be provided. + properties: + header: + description: Header specifies the header condition to match. + properties: + contains: + description: Contains specifies a substring that must be present in the header value. + type: string + exact: + description: Exact specifies a string that the header value must be equal to. + type: string + name: + description: Name is the name of the header to match against. Name is required. Header names are case insensitive. + type: string + notcontains: + description: NotContains specifies a substring that must not be present in the header value. + type: string + notexact: + description: NoExact specifies a string that the header value must not be equal to. The condition is true if the header has any other value. + type: string + present: + description: Present specifies that condition is true when the named header is present, regardless of its value. Note that setting Present to false does not make the condition true if the named header is absent. + type: boolean + required: + - name + type: object + prefix: + description: Prefix defines a prefix match for a request. + type: string + type: object + type: array + name: + description: Name of the HTTPProxy + type: string + namespace: + description: Namespace of the HTTPProxy to include. Defaults to the current namespace if not supplied. + type: string + required: + - name + type: object + type: array + routes: + description: Routes are the ingress routes. If TCPProxy is present, Routes is ignored. + items: + description: Route contains the set of routes for a virtual host. + properties: + authPolicy: + description: AuthPolicy updates the authorization policy that was set on the root HTTPProxy object for client requests that match this route. + properties: + context: + additionalProperties: + type: string + description: Context is a set of key/value pairs that are sent to the authentication server in the check request. If a context is provided at an enclosing scope, the entries are merged such that the inner scope overrides matching keys from the outer scope. + type: object + disabled: + description: When true, this field disables client request authentication for the scope of the policy. + type: boolean + type: object + conditions: + description: 'Conditions are a set of rules that are applied to a Route. When applied, they are merged using AND, with one exception: There can be only one Prefix MatchCondition per Conditions slice. More than one Prefix, or contradictory Conditions, will make the route invalid.' + items: + description: MatchCondition are a general holder for matching rules for HTTPProxies. One of Prefix or Header must be provided. + properties: + header: + description: Header specifies the header condition to match. + properties: + contains: + description: Contains specifies a substring that must be present in the header value. + type: string + exact: + description: Exact specifies a string that the header value must be equal to. + type: string + name: + description: Name is the name of the header to match against. Name is required. Header names are case insensitive. + type: string + notcontains: + description: NotContains specifies a substring that must not be present in the header value. + type: string + notexact: + description: NoExact specifies a string that the header value must not be equal to. The condition is true if the header has any other value. + type: string + present: + description: Present specifies that condition is true when the named header is present, regardless of its value. Note that setting Present to false does not make the condition true if the named header is absent. + type: boolean + required: + - name + type: object + prefix: + description: Prefix defines a prefix match for a request. + type: string + type: object + type: array + enableWebsockets: + description: Enables websocket support for the route. + type: boolean + healthCheckPolicy: + description: The health check policy for this route. + properties: + healthyThresholdCount: + description: The number of healthy health checks required before a host is marked healthy + format: int64 + minimum: 0 + type: integer + host: + description: The value of the host header in the HTTP health check request. If left empty (default value), the name "contour-envoy-healthcheck" will be used. + type: string + intervalSeconds: + description: The interval (seconds) between health checks + format: int64 + type: integer + path: + description: HTTP endpoint used to perform health checks on upstream service + type: string + timeoutSeconds: + description: The time to wait (seconds) for a health check response + format: int64 + type: integer + unhealthyThresholdCount: + description: The number of unhealthy health checks required before a host is marked unhealthy + format: int64 + minimum: 0 + type: integer + required: + - path + type: object + loadBalancerPolicy: + description: The load balancing policy for this route. + properties: + strategy: + description: Strategy specifies the policy used to balance requests across the pool of backend pods. Valid policy names are `Random`, `RoundRobin`, `WeightedLeastRequest`, `Random` and `Cookie`. If an unknown strategy name is specified or no policy is supplied, the default `RoundRobin` policy is used. + type: string + type: object + pathRewritePolicy: + description: The policy for rewriting the path of the request URL after the request has been routed to a Service. + properties: + replacePrefix: + description: ReplacePrefix describes how the path prefix should be replaced. + items: + description: ReplacePrefix describes a path prefix replacement. + properties: + prefix: + description: "Prefix specifies the URL path prefix to be replaced. \n If Prefix is specified, it must exactly match the MatchCondition prefix that is rendered by the chain of including HTTPProxies and only that path prefix will be replaced by Replacement. This allows HTTPProxies that are included through multiple roots to only replace specific path prefixes, leaving others unmodified. \n If Prefix is not specified, all routing prefixes rendered by the include chain will be replaced." + minLength: 1 + type: string + replacement: + description: Replacement is the string that the routing path prefix will be replaced with. This must not be empty. + minLength: 1 + type: string + required: + - replacement + type: object + type: array + type: object + permitInsecure: + description: Allow this path to respond to insecure requests over HTTP which are normally not permitted when a `virtualhost.tls` block is present. + type: boolean + requestHeadersPolicy: + description: The policy for managing request headers during proxying. + properties: + remove: + description: Remove specifies a list of HTTP header names to remove. + items: + type: string + type: array + set: + description: Set specifies a list of HTTP header values that will be set in the HTTP header. If the header does not exist it will be added, otherwise it will be overwritten with the new value. + items: + description: HeaderValue represents a header name/value pair + properties: + name: + description: Name represents a key of a header + minLength: 1 + type: string + value: + description: Value represents the value of a header specified by a key + minLength: 1 + type: string + required: + - name + - value + type: object + type: array + type: object + responseHeadersPolicy: + description: The policy for managing response headers during proxying. Rewriting the 'Host' header is not supported. + properties: + remove: + description: Remove specifies a list of HTTP header names to remove. + items: + type: string + type: array + set: + description: Set specifies a list of HTTP header values that will be set in the HTTP header. If the header does not exist it will be added, otherwise it will be overwritten with the new value. + items: + description: HeaderValue represents a header name/value pair + properties: + name: + description: Name represents a key of a header + minLength: 1 + type: string + value: + description: Value represents the value of a header specified by a key + minLength: 1 + type: string + required: + - name + - value + type: object + type: array + type: object + retryPolicy: + description: The retry policy for this route. + properties: + count: + description: NumRetries is maximum allowed number of retries. If not supplied, the number of retries is one. + format: int64 + minimum: 0 + type: integer + perTryTimeout: + description: PerTryTimeout specifies the timeout per retry attempt. Ignored if NumRetries is not supplied. + pattern: ^(((\d*(\.\d*)?h)|(\d*(\.\d*)?m)|(\d*(\.\d*)?s)|(\d*(\.\d*)?ms)|(\d*(\.\d*)?us)|(\d*(\.\d*)?µs)|(\d*(\.\d*)?ns))+|infinity|infinite)$ + type: string + retriableStatusCodes: + description: "RetriableStatusCodes specifies the HTTP status codes that should be retried. \n This field is only respected when you include `retriable-status-codes` in the `RetryOn` field." + items: + format: int32 + type: integer + type: array + retryOn: + description: "RetryOn specifies the conditions on which to retry a request. \n Supported [HTTP conditions](https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_filters/router_filter#x-envoy-retry-on): \n - `5xx` - `gateway-error` - `reset` - `connect-failure` - `retriable-4xx` - `refused-stream` - `retriable-status-codes` - `retriable-headers` \n Supported [gRPC conditions](https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_filters/router_filter#x-envoy-retry-grpc-on): \n - `cancelled` - `deadline-exceeded` - `internal` - `resource-exhausted` - `unavailable`" + items: + description: RetryOn is a string type alias with validation to ensure that the value is valid. + enum: + - 5xx + - gateway-error + - reset + - connect-failure + - retriable-4xx + - refused-stream + - retriable-status-codes + - retriable-headers + - cancelled + - deadline-exceeded + - internal + - resource-exhausted + - unavailable + type: string + type: array + type: object + services: + description: Services are the services to proxy traffic. + items: + description: Service defines an Kubernetes Service to proxy traffic. + properties: + mirror: + description: If Mirror is true the Service will receive a read only mirror of the traffic for this route. + type: boolean + name: + description: Name is the name of Kubernetes service to proxy traffic. Names defined here will be used to look up corresponding endpoints which contain the ips to route. + type: string + port: + description: Port (defined as Integer) to proxy traffic to since a service can have multiple defined. + exclusiveMaximum: true + maximum: 65536 + minimum: 1 + type: integer + protocol: + description: Protocol may be used to specify (or override) the protocol used to reach this Service. Values may be tls, h2, h2c. If omitted, protocol-selection falls back on Service annotations. + enum: + - h2 + - h2c + - tls + type: string + requestHeadersPolicy: + description: The policy for managing request headers during proxying. Rewriting the 'Host' header is not supported. + properties: + remove: + description: Remove specifies a list of HTTP header names to remove. + items: + type: string + type: array + set: + description: Set specifies a list of HTTP header values that will be set in the HTTP header. If the header does not exist it will be added, otherwise it will be overwritten with the new value. + items: + description: HeaderValue represents a header name/value pair + properties: + name: + description: Name represents a key of a header + minLength: 1 + type: string + value: + description: Value represents the value of a header specified by a key + minLength: 1 + type: string + required: + - name + - value + type: object + type: array + type: object + responseHeadersPolicy: + description: The policy for managing response headers during proxying. Rewriting the 'Host' header is not supported. + properties: + remove: + description: Remove specifies a list of HTTP header names to remove. + items: + type: string + type: array + set: + description: Set specifies a list of HTTP header values that will be set in the HTTP header. If the header does not exist it will be added, otherwise it will be overwritten with the new value. + items: + description: HeaderValue represents a header name/value pair + properties: + name: + description: Name represents a key of a header + minLength: 1 + type: string + value: + description: Value represents the value of a header specified by a key + minLength: 1 + type: string + required: + - name + - value + type: object + type: array + type: object + validation: + description: UpstreamValidation defines how to verify the backend service's certificate + properties: + caSecret: + description: Name of the Kubernetes secret be used to validate the certificate presented by the backend + type: string + subjectName: + description: Key which is expected to be present in the 'subjectAltName' of the presented certificate + type: string + required: + - caSecret + - subjectName + type: object + weight: + description: Weight defines percentage of traffic to balance traffic + format: int64 + minimum: 0 + type: integer + required: + - name + - port + type: object + minItems: 1 + type: array + timeoutPolicy: + description: The timeout policy for this route. + properties: + idle: + description: Timeout after which, if there are no active requests for this route, the connection between Envoy and the backend or Envoy and the external client will be closed. If not specified, there is no per-route idle timeout, though a connection manager-wide stream_idle_timeout default of 5m still applies. + pattern: ^(((\d*(\.\d*)?h)|(\d*(\.\d*)?m)|(\d*(\.\d*)?s)|(\d*(\.\d*)?ms)|(\d*(\.\d*)?us)|(\d*(\.\d*)?µs)|(\d*(\.\d*)?ns))+|infinity|infinite)$ + type: string + response: + description: Timeout for receiving a response from the server after processing a request from client. If not supplied, Envoy's default value of 15s applies. + pattern: ^(((\d*(\.\d*)?h)|(\d*(\.\d*)?m)|(\d*(\.\d*)?s)|(\d*(\.\d*)?ms)|(\d*(\.\d*)?us)|(\d*(\.\d*)?µs)|(\d*(\.\d*)?ns))+|infinity|infinite)$ + type: string + type: object + required: + - services + type: object + type: array + tcpproxy: + description: TCPProxy holds TCP proxy information. + properties: + healthCheckPolicy: + description: The health check policy for this tcp proxy + properties: + healthyThresholdCount: + description: The number of healthy health checks required before a host is marked healthy + format: int32 + type: integer + intervalSeconds: + description: The interval (seconds) between health checks + format: int64 + type: integer + timeoutSeconds: + description: The time to wait (seconds) for a health check response + format: int64 + type: integer + unhealthyThresholdCount: + description: The number of unhealthy health checks required before a host is marked unhealthy + format: int32 + type: integer + type: object + include: + description: Include specifies that this tcpproxy should be delegated to another HTTPProxy. + properties: + name: + description: Name of the child HTTPProxy + type: string + namespace: + description: Namespace of the HTTPProxy to include. Defaults to the current namespace if not supplied. + type: string + required: + - name + type: object + includes: + description: "IncludesDeprecated allow for specific routing configuration to be appended to another HTTPProxy in another namespace. \n Exists due to a mistake when developing HTTPProxy and the field was marked plural when it should have been singular. This field should stay to not break backwards compatibility to v1 users." + properties: + name: + description: Name of the child HTTPProxy + type: string + namespace: + description: Namespace of the HTTPProxy to include. Defaults to the current namespace if not supplied. + type: string + required: + - name + type: object + loadBalancerPolicy: + description: The load balancing policy for the backend services. + properties: + strategy: + description: Strategy specifies the policy used to balance requests across the pool of backend pods. Valid policy names are `Random`, `RoundRobin`, `WeightedLeastRequest`, `Random` and `Cookie`. If an unknown strategy name is specified or no policy is supplied, the default `RoundRobin` policy is used. + type: string + type: object + services: + description: Services are the services to proxy traffic + items: + description: Service defines an Kubernetes Service to proxy traffic. + properties: + mirror: + description: If Mirror is true the Service will receive a read only mirror of the traffic for this route. + type: boolean + name: + description: Name is the name of Kubernetes service to proxy traffic. Names defined here will be used to look up corresponding endpoints which contain the ips to route. + type: string + port: + description: Port (defined as Integer) to proxy traffic to since a service can have multiple defined. + exclusiveMaximum: true + maximum: 65536 + minimum: 1 + type: integer + protocol: + description: Protocol may be used to specify (or override) the protocol used to reach this Service. Values may be tls, h2, h2c. If omitted, protocol-selection falls back on Service annotations. + enum: + - h2 + - h2c + - tls + type: string + requestHeadersPolicy: + description: The policy for managing request headers during proxying. Rewriting the 'Host' header is not supported. + properties: + remove: + description: Remove specifies a list of HTTP header names to remove. + items: + type: string + type: array + set: + description: Set specifies a list of HTTP header values that will be set in the HTTP header. If the header does not exist it will be added, otherwise it will be overwritten with the new value. + items: + description: HeaderValue represents a header name/value pair + properties: + name: + description: Name represents a key of a header + minLength: 1 + type: string + value: + description: Value represents the value of a header specified by a key + minLength: 1 + type: string + required: + - name + - value + type: object + type: array + type: object + responseHeadersPolicy: + description: The policy for managing response headers during proxying. Rewriting the 'Host' header is not supported. + properties: + remove: + description: Remove specifies a list of HTTP header names to remove. + items: + type: string + type: array + set: + description: Set specifies a list of HTTP header values that will be set in the HTTP header. If the header does not exist it will be added, otherwise it will be overwritten with the new value. + items: + description: HeaderValue represents a header name/value pair + properties: + name: + description: Name represents a key of a header + minLength: 1 + type: string + value: + description: Value represents the value of a header specified by a key + minLength: 1 + type: string + required: + - name + - value + type: object + type: array + type: object + validation: + description: UpstreamValidation defines how to verify the backend service's certificate + properties: + caSecret: + description: Name of the Kubernetes secret be used to validate the certificate presented by the backend + type: string + subjectName: + description: Key which is expected to be present in the 'subjectAltName' of the presented certificate + type: string + required: + - caSecret + - subjectName + type: object + weight: + description: Weight defines percentage of traffic to balance traffic + format: int64 + minimum: 0 + type: integer + required: + - name + - port + type: object + type: array + type: object + virtualhost: + description: Virtualhost appears at most once. If it is present, the object is considered to be a "root" HTTPProxy. + properties: + authorization: + description: This field configures an extension service to perform authorization for this virtual host. Authorization can only be configured on virtual hosts that have TLS enabled. If the TLS configuration requires client certificate /validation, the client certificate is always included in the authentication check request. + properties: + authPolicy: + description: AuthPolicy sets a default authorization policy for client requests. This policy will be used unless overridden by individual routes. + properties: + context: + additionalProperties: + type: string + description: Context is a set of key/value pairs that are sent to the authentication server in the check request. If a context is provided at an enclosing scope, the entries are merged such that the inner scope overrides matching keys from the outer scope. + type: object + disabled: + description: When true, this field disables client request authentication for the scope of the policy. + type: boolean + type: object + extensionRef: + description: ExtensionServiceRef specifies the extension resource that will authorize client requests. + properties: + apiVersion: + description: API version of the referent. If this field is not specified, the default "projectcontour.io/v1alpha1" will be used + minLength: 1 + type: string + name: + description: "Name of the referent. \n More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names" + minLength: 1 + type: string + namespace: + description: "Namespace of the referent. If this field is not specifies, the namespace of the resource that targets the referent will be used. \n More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/" + minLength: 1 + type: string + type: object + failOpen: + description: If FailOpen is true, the client request is forwarded to the upstream service even if the authorization server fails to respond. This field should not be set in most cases. It is intended for use only while migrating applications from internal authorization to Contour external authorization. + type: boolean + responseTimeout: + description: ResponseTimeout configures maximum time to wait for a check response from the authorization server. Timeout durations are expressed in the Go [Duration format](https://godoc.org/time#ParseDuration). Valid time units are "ns", "us" (or "µs"), "ms", "s", "m", "h". The string "infinity" is also a valid input and specifies no timeout. + pattern: ^(((\d*(\.\d*)?h)|(\d*(\.\d*)?m)|(\d*(\.\d*)?s)|(\d*(\.\d*)?ms)|(\d*(\.\d*)?us)|(\d*(\.\d*)?µs)|(\d*(\.\d*)?ns))+|infinity|infinite)$ + type: string + required: + - extensionRef + type: object + corsPolicy: + description: Specifies the cross-origin policy to apply to the VirtualHost. + properties: + allowCredentials: + description: Specifies whether the resource allows credentials. + type: boolean + allowHeaders: + description: AllowHeaders specifies the content for the *access-control-allow-headers* header. + items: + description: CORSHeaderValue specifies the value of the string headers returned by a cross-domain request. + pattern: ^[a-zA-Z0-9!#$%&'*+.^_`|~-]+$ + type: string + type: array + allowMethods: + description: AllowMethods specifies the content for the *access-control-allow-methods* header. + items: + description: CORSHeaderValue specifies the value of the string headers returned by a cross-domain request. + pattern: ^[a-zA-Z0-9!#$%&'*+.^_`|~-]+$ + type: string + type: array + allowOrigin: + description: AllowOrigin specifies the origins that will be allowed to do CORS requests. "*" means allow any origin. + items: + type: string + type: array + exposeHeaders: + description: ExposeHeaders Specifies the content for the *access-control-expose-headers* header. + items: + description: CORSHeaderValue specifies the value of the string headers returned by a cross-domain request. + pattern: ^[a-zA-Z0-9!#$%&'*+.^_`|~-]+$ + type: string + type: array + maxAge: + description: MaxAge indicates for how long the results of a preflight request can be cached. MaxAge durations are expressed in the Go [Duration format](https://godoc.org/time#ParseDuration). Valid time units are "ns", "us" (or "µs"), "ms", "s", "m", "h". Only positive values are allowed while 0 disables the cache requiring a preflight OPTIONS check for all cross-origin requests. + type: string + required: + - allowMethods + - allowOrigin + type: object + fqdn: + description: The fully qualified domain name of the root of the ingress tree all leaves of the DAG rooted at this object relate to the fqdn. + type: string + tls: + description: If present the fields describes TLS properties of the virtual host. The SNI names that will be matched on are described in fqdn, the tls.secretName secret must contain a certificate that itself contains a name that matches the FQDN. + properties: + clientValidation: + description: "ClientValidation defines how to verify the client certificate when an external client establishes a TLS connection to Envoy. \n This setting: \n 1. Enables TLS client certificate validation. 2. Requires clients to present a TLS certificate (i.e. not optional validation). 3. Specifies how the client certificate will be validated." + properties: + caSecret: + description: Name of a Kubernetes secret that contains a CA certificate bundle. The client certificate must validate against the certificates in the bundle. + minLength: 1 + type: string + required: + - caSecret + type: object + enableFallbackCertificate: + description: EnableFallbackCertificate defines if the vhost should allow a default certificate to be applied which handles all requests which don't match the SNI defined in this vhost. + type: boolean + minimumProtocolVersion: + description: Minimum TLS version this vhost should negotiate + type: string + passthrough: + description: Passthrough defines whether the encrypted TLS handshake will be passed through to the backing cluster. Either Passthrough or SecretName must be specified, but not both. + type: boolean + secretName: + description: SecretName is the name of a TLS secret in the current namespace. Either SecretName or Passthrough must be specified, but not both. If specified, the named secret must contain a matching certificate for the virtual host's FQDN. + type: string + type: object + required: + - fqdn + type: object + type: object + status: + description: Status is a container for computed information about the HTTPProxy. + properties: + conditions: + description: "Conditions contains information about the current status of the HTTPProxy, in an upstream-friendly container. \n Contour will update a single condition, `Valid`, that is in normal-true polarity. That is, when `currentStatus` is `valid`, the `Valid` condition will be `status: true`, and vice versa. \n Contour will leave untouched any other Conditions set in this block, in case some other controller wants to add a Condition. \n If you are another controller owner and wish to add a condition, you *should* namespace your condition with a label, like `controller.domain.com/ConditionName`." + items: + description: "DetailedCondition is an extension of the normal Kubernetes conditions, with two extra fields to hold sub-conditions, which provide more detailed reasons for the state (True or False) of the condition. \n `errors` holds information about sub-conditions which are fatal to that condition and render its state False. \n `warnings` holds information about sub-conditions which are not fatal to that condition and do not force the state to be False. \n Remember that Conditions have a type, a status, and a reason. \n The type is the type of the condition, the most important one in this CRD set is `Valid`. `Valid` is a positive-polarity condition: when it is `status: true` there are no problems. \n In more detail, `status: true` means that the object is has been ingested into Contour with no errors. `warnings` may still be present, and will be indicated in the Reason field. There must be zero entries in the `errors` slice in this case. \n `Valid`, `status: false` means that the object has had one or more fatal errors during processing into Contour. The details of the errors will be present under the `errors` field. There must be at least one error in the `errors` slice if `status` is `false`. \n For DetailedConditions of types other than `Valid`, the Condition must be in the negative polarity. When they have `status` `true`, there is an error. There must be at least one entry in the `errors` Subcondition slice. When they have `status` `false`, there are no serious errors, and there must be zero entries in the `errors` slice. In either case, there may be entries in the `warnings` slice. \n Regardless of the polarity, the `reason` and `message` fields must be updated with either the detail of the reason (if there is one and only one entry in total across both the `errors` and `warnings` slices), or `MultipleReasons` if there is more than one entry." + properties: + errors: + description: "Errors contains a slice of relevant error subconditions for this object. \n Subconditions are expected to appear when relevant (when there is a error), and disappear when not relevant. An empty slice here indicates no errors." + items: + description: "SubCondition is a Condition-like type intended for use as a subcondition inside a DetailedCondition. \n It contains a subset of the Condition fields. \n It is intended for warnings and errors, so `type` names should use abnormal-true polarity, that is, they should be of the form \"ErrorPresent: true\". \n The expected lifecycle for these errors is that they should only be present when the error or warning is, and should be removed when they are not relevant." + properties: + message: + description: "Message is a human readable message indicating details about the transition. \n This may be an empty string." + maxLength: 32768 + type: string + reason: + description: "Reason contains a programmatic identifier indicating the reason for the condition's last transition. Producers of specific condition types may define expected values and meanings for this field, and whether the values are considered a guaranteed API. \n The value should be a CamelCase string. \n This field may not be empty." + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + description: Status of the condition, one of True, False, Unknown. + enum: + - "True" + - "False" + - Unknown + type: string + type: + description: "Type of condition in `CamelCase` or in `foo.example.com/CamelCase`. \n This must be in abnormal-true polarity, that is, `ErrorFound` or `controller.io/ErrorFound`. \n The regex it matches is (dns1123SubdomainFmt/)?(qualifiedNameFmt)" + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + required: + - message + - reason + - status + - type + type: object + type: array + lastTransitionTime: + description: lastTransitionTime is the last time the condition transitioned from one status to another. This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable. + format: date-time + type: string + message: + description: message is a human readable message indicating details about the transition. This may be an empty string. + maxLength: 32768 + type: string + observedGeneration: + description: observedGeneration represents the .metadata.generation that the condition was set based upon. For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date with respect to the current state of the instance. + format: int64 + minimum: 0 + type: integer + reason: + description: reason contains a programmatic identifier indicating the reason for the condition's last transition. Producers of specific condition types may define expected values and meanings for this field, and whether the values are considered a guaranteed API. The value should be a CamelCase string. This field may not be empty. + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + description: status of the condition, one of True, False, Unknown. + enum: + - "True" + - "False" + - Unknown + type: string + type: + description: type of condition in CamelCase or in foo.example.com/CamelCase. --- Many .condition.type values are consistent across resources like Available, but because arbitrary conditions can be useful (see .node.status.conditions), the ability to deconflict is important. The regex it matches is (dns1123SubdomainFmt/)?(qualifiedNameFmt) + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + warnings: + description: "Warnings contains a slice of relevant warning subconditions for this object. \n Subconditions are expected to appear when relevant (when there is a warning), and disappear when not relevant. An empty slice here indicates no warnings." + items: + description: "SubCondition is a Condition-like type intended for use as a subcondition inside a DetailedCondition. \n It contains a subset of the Condition fields. \n It is intended for warnings and errors, so `type` names should use abnormal-true polarity, that is, they should be of the form \"ErrorPresent: true\". \n The expected lifecycle for these errors is that they should only be present when the error or warning is, and should be removed when they are not relevant." + properties: + message: + description: "Message is a human readable message indicating details about the transition. \n This may be an empty string." + maxLength: 32768 + type: string + reason: + description: "Reason contains a programmatic identifier indicating the reason for the condition's last transition. Producers of specific condition types may define expected values and meanings for this field, and whether the values are considered a guaranteed API. \n The value should be a CamelCase string. \n This field may not be empty." + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + description: Status of the condition, one of True, False, Unknown. + enum: + - "True" + - "False" + - Unknown + type: string + type: + description: "Type of condition in `CamelCase` or in `foo.example.com/CamelCase`. \n This must be in abnormal-true polarity, that is, `ErrorFound` or `controller.io/ErrorFound`. \n The regex it matches is (dns1123SubdomainFmt/)?(qualifiedNameFmt)" + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + required: + - message + - reason + - status + - type + type: object + type: array + required: + - lastTransitionTime + - message + - reason + - status + - type + type: object + type: array + x-kubernetes-list-map-keys: + - type + x-kubernetes-list-type: map + currentStatus: + type: string + description: + type: string + loadBalancer: + description: LoadBalancer contains the current status of the load balancer. + properties: + ingress: + description: Ingress is a list containing ingress points for the load-balancer. Traffic intended for the service should be sent to these ingress points. + items: + description: 'LoadBalancerIngress represents the status of a load-balancer ingress point: traffic intended for the service should be sent to an ingress point.' + properties: + hostname: + description: Hostname is set for load-balancer ingress points that are DNS based (typically AWS load-balancers) + type: string + ip: + description: IP is set for load-balancer ingress points that are IP based (typically GCE or OpenStack load-balancers) + type: string + type: object + type: array + type: object + type: object + required: + - metadata + - spec + type: object + served: true + storage: true + subresources: + status: {} +status: + acceptedNames: + kind: "" + plural: "" + conditions: [] + storedVersions: [] +--- +# Source: flyte/charts/contour/templates/00-crds.yaml +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.2.9 + creationTimestamp: null + name: tlscertificatedelegations.projectcontour.io +spec: + preserveUnknownFields: false + group: projectcontour.io + names: + kind: TLSCertificateDelegation + listKind: TLSCertificateDelegationList + plural: tlscertificatedelegations + shortNames: + - tlscerts + singular: tlscertificatedelegation + scope: Namespaced + versions: + - name: v1 + schema: + openAPIV3Schema: + description: TLSCertificateDelegation is an TLS Certificate Delegation CRD specification. See design/tls-certificate-delegation.md for details. + properties: + apiVersion: + description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' + type: string + kind: + description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + type: string + metadata: + type: object + spec: + description: TLSCertificateDelegationSpec defines the spec of the CRD + properties: + delegations: + items: + description: CertificateDelegation maps the authority to reference a secret in the current namespace to a set of namespaces. + properties: + secretName: + description: required, the name of a secret in the current namespace. + type: string + targetNamespaces: + description: required, the namespaces the authority to reference the the secret will be delegated to. If TargetNamespaces is nil or empty, the CertificateDelegation' is ignored. If the TargetNamespace list contains the character, "*" the secret will be delegated to all namespaces. + items: + type: string + type: array + required: + - secretName + - targetNamespaces + type: object + type: array + required: + - delegations + type: object + status: + description: TLSCertificateDelegationStatus allows for the status of the delegation to be presented to the user. + properties: + conditions: + description: "Conditions contains information about the current status of the HTTPProxy, in an upstream-friendly container. \n Contour will update a single condition, `Valid`, that is in normal-true polarity. That is, when `currentStatus` is `valid`, the `Valid` condition will be `status: true`, and vice versa. \n Contour will leave untouched any other Conditions set in this block, in case some other controller wants to add a Condition. \n If you are another controller owner and wish to add a condition, you *should* namespace your condition with a label, like `controller.domain.com\\ConditionName`." + items: + description: "DetailedCondition is an extension of the normal Kubernetes conditions, with two extra fields to hold sub-conditions, which provide more detailed reasons for the state (True or False) of the condition. \n `errors` holds information about sub-conditions which are fatal to that condition and render its state False. \n `warnings` holds information about sub-conditions which are not fatal to that condition and do not force the state to be False. \n Remember that Conditions have a type, a status, and a reason. \n The type is the type of the condition, the most important one in this CRD set is `Valid`. `Valid` is a positive-polarity condition: when it is `status: true` there are no problems. \n In more detail, `status: true` means that the object is has been ingested into Contour with no errors. `warnings` may still be present, and will be indicated in the Reason field. There must be zero entries in the `errors` slice in this case. \n `Valid`, `status: false` means that the object has had one or more fatal errors during processing into Contour. The details of the errors will be present under the `errors` field. There must be at least one error in the `errors` slice if `status` is `false`. \n For DetailedConditions of types other than `Valid`, the Condition must be in the negative polarity. When they have `status` `true`, there is an error. There must be at least one entry in the `errors` Subcondition slice. When they have `status` `false`, there are no serious errors, and there must be zero entries in the `errors` slice. In either case, there may be entries in the `warnings` slice. \n Regardless of the polarity, the `reason` and `message` fields must be updated with either the detail of the reason (if there is one and only one entry in total across both the `errors` and `warnings` slices), or `MultipleReasons` if there is more than one entry." + properties: + errors: + description: "Errors contains a slice of relevant error subconditions for this object. \n Subconditions are expected to appear when relevant (when there is a error), and disappear when not relevant. An empty slice here indicates no errors." + items: + description: "SubCondition is a Condition-like type intended for use as a subcondition inside a DetailedCondition. \n It contains a subset of the Condition fields. \n It is intended for warnings and errors, so `type` names should use abnormal-true polarity, that is, they should be of the form \"ErrorPresent: true\". \n The expected lifecycle for these errors is that they should only be present when the error or warning is, and should be removed when they are not relevant." + properties: + message: + description: "Message is a human readable message indicating details about the transition. \n This may be an empty string." + maxLength: 32768 + type: string + reason: + description: "Reason contains a programmatic identifier indicating the reason for the condition's last transition. Producers of specific condition types may define expected values and meanings for this field, and whether the values are considered a guaranteed API. \n The value should be a CamelCase string. \n This field may not be empty." + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + description: Status of the condition, one of True, False, Unknown. + enum: + - "True" + - "False" + - Unknown + type: string + type: + description: "Type of condition in `CamelCase` or in `foo.example.com/CamelCase`. \n This must be in abnormal-true polarity, that is, `ErrorFound` or `controller.io/ErrorFound`. \n The regex it matches is (dns1123SubdomainFmt/)?(qualifiedNameFmt)" + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + required: + - message + - reason + - status + - type + type: object + type: array + lastTransitionTime: + description: lastTransitionTime is the last time the condition transitioned from one status to another. This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable. + format: date-time + type: string + message: + description: message is a human readable message indicating details about the transition. This may be an empty string. + maxLength: 32768 + type: string + observedGeneration: + description: observedGeneration represents the .metadata.generation that the condition was set based upon. For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date with respect to the current state of the instance. + format: int64 + minimum: 0 + type: integer + reason: + description: reason contains a programmatic identifier indicating the reason for the condition's last transition. Producers of specific condition types may define expected values and meanings for this field, and whether the values are considered a guaranteed API. The value should be a CamelCase string. This field may not be empty. + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + description: status of the condition, one of True, False, Unknown. + enum: + - "True" + - "False" + - Unknown + type: string + type: + description: type of condition in CamelCase or in foo.example.com/CamelCase. --- Many .condition.type values are consistent across resources like Available, but because arbitrary conditions can be useful (see .node.status.conditions), the ability to deconflict is important. The regex it matches is (dns1123SubdomainFmt/)?(qualifiedNameFmt) + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + warnings: + description: "Warnings contains a slice of relevant warning subconditions for this object. \n Subconditions are expected to appear when relevant (when there is a warning), and disappear when not relevant. An empty slice here indicates no warnings." + items: + description: "SubCondition is a Condition-like type intended for use as a subcondition inside a DetailedCondition. \n It contains a subset of the Condition fields. \n It is intended for warnings and errors, so `type` names should use abnormal-true polarity, that is, they should be of the form \"ErrorPresent: true\". \n The expected lifecycle for these errors is that they should only be present when the error or warning is, and should be removed when they are not relevant." + properties: + message: + description: "Message is a human readable message indicating details about the transition. \n This may be an empty string." + maxLength: 32768 + type: string + reason: + description: "Reason contains a programmatic identifier indicating the reason for the condition's last transition. Producers of specific condition types may define expected values and meanings for this field, and whether the values are considered a guaranteed API. \n The value should be a CamelCase string. \n This field may not be empty." + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + description: Status of the condition, one of True, False, Unknown. + enum: + - "True" + - "False" + - Unknown + type: string + type: + description: "Type of condition in `CamelCase` or in `foo.example.com/CamelCase`. \n This must be in abnormal-true polarity, that is, `ErrorFound` or `controller.io/ErrorFound`. \n The regex it matches is (dns1123SubdomainFmt/)?(qualifiedNameFmt)" + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + required: + - message + - reason + - status + - type + type: object + type: array + required: + - lastTransitionTime + - message + - reason + - status + - type + type: object + type: array + x-kubernetes-list-map-keys: + - type + x-kubernetes-list-type: map + type: object + required: + - metadata + - spec + type: object + served: true + storage: true + subresources: + status: {} +status: + acceptedNames: + kind: "" + plural: "" + conditions: [] + storedVersions: [] +--- +# Source: flyte/templates/propeller/crds/flyteworkflow.yaml +apiVersion: apiextensions.k8s.io/v1beta1 +kind: CustomResourceDefinition +metadata: + name: flyteworkflows.flyte.lyft.com +spec: + group: flyte.lyft.com + names: + kind: FlyteWorkflow + plural: flyteworkflows + shortNames: + - fly + singular: flyteworkflow + scope: Namespaced + version: v1alpha1 +--- +# Source: flyte/templates/pytorch-operator/crds/pytorchjobs.yaml +apiVersion: apiextensions.k8s.io/v1beta1 +kind: CustomResourceDefinition +metadata: + name: pytorchjobs.kubeflow.org +spec: + additionalPrinterColumns: + - JSONPath: .status.conditions[-1:].type + name: State + type: string + - JSONPath: .metadata.creationTimestamp + name: Age + type: date + group: kubeflow.org + names: + kind: PyTorchJob + plural: pytorchjobs + singular: pytorchjob + scope: Namespaced + subresources: + status: {} + validation: + openAPIV3Schema: + properties: + spec: + properties: + pytorchReplicaSpecs: + properties: + Master: + properties: + replicas: + maximum: 1 + minimum: 1 + type: integer + Worker: + properties: + replicas: + minimum: 1 + type: integer + versions: + - name: v1 + served: true + storage: true +--- +# Source: flyte/charts/contour/templates/contour/rbac.yaml +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: flyte-contour-contour + labels: + app.kubernetes.io/name: contour + helm.sh/chart: contour-4.1.2 + app.kubernetes.io/instance: flyte + app.kubernetes.io/managed-by: Helm +rules: + - apiGroups: + - "" + resources: + - configmaps + verbs: + - create + - get + - update + - apiGroups: + - "" + resources: + - endpoints + verbs: + - get + - list + - watch + - apiGroups: + - "" + resources: + - secrets + verbs: + - get + - list + - watch + - apiGroups: + - "" + resources: + - services + verbs: + - get + - list + - watch + - apiGroups: + - apiextensions.k8s.io + resources: + - customresourcedefinitions + verbs: + - list + - apiGroups: + - networking.k8s.io + resources: + - gatewayclasses + - gateways + - httproutes + - tcproutes + verbs: + - get + - list + - watch + - apiGroups: + - networking.k8s.io + resources: + - ingresses + verbs: + - get + - list + - watch + - apiGroups: + - networking.k8s.io + resources: + - ingresses/status + verbs: + - create + - get + - update + - apiGroups: + - projectcontour.io + resources: + - extensionservices + verbs: + - get + - list + - watch + - apiGroups: + - projectcontour.io + resources: + - extensionservices/status + verbs: + - create + - get + - update + - apiGroups: + - projectcontour.io + resources: + - httpproxies + - tlscertificatedelegations + verbs: + - get + - list + - watch + - apiGroups: + - projectcontour.io + resources: + - httpproxies/status + verbs: + - create + - get + - update +--- +# Source: flyte/charts/sparkoperator/templates/rbac.yaml +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: flyte-sparkoperator + labels: + helm.sh/chart: sparkoperator-1.0.6 + app.kubernetes.io/name: sparkoperator + app.kubernetes.io/instance: flyte + app.kubernetes.io/version: "v1beta2-1.2.0-3.0.0" + app.kubernetes.io/managed-by: Helm +rules: +- apiGroups: + - "" + resources: + - pods + verbs: + - "*" +- apiGroups: + - "" + resources: + - services + - configmaps + - secrets + verbs: + - create + - get + - delete + - update +- apiGroups: + - extensions + - networking.k8s.io + resources: + - ingresses + verbs: + - create + - get + - delete +- apiGroups: + - "" + resources: + - nodes + verbs: + - get +- apiGroups: + - "" + resources: + - events + verbs: + - create + - update + - patch +- apiGroups: + - "" + resources: + - resourcequotas + verbs: + - get + - list + - watch +- apiGroups: + - apiextensions.k8s.io + resources: + - customresourcedefinitions + verbs: + - create + - get + - update + - delete +- apiGroups: + - admissionregistration.k8s.io + resources: + - mutatingwebhookconfigurations + - validatingwebhookconfigurations + verbs: + - create + - get + - update + - delete +- apiGroups: + - sparkoperator.k8s.io + resources: + - sparkapplications + - sparkapplications/status + - scheduledsparkapplications + - scheduledsparkapplications/status + verbs: + - "*" +--- +# Source: flyte/templates/admin/rbac.yaml +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: flyteadmin + labels: + app.kubernetes.io/name: flyteadmin + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm +rules: +- apiGroups: + - "" + - flyte.lyft.com + - rbac.authorization.k8s.io + resources: + - configmaps + - flyteworkflows + - namespaces + - pods + - resourcequotas + - roles + - rolebindings + - secrets + - services + - serviceaccounts + - spark-role + verbs: + - '*' +--- +# Source: flyte/templates/propeller/rbac.yaml +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: flytepropeller + labels: + app.kubernetes.io/name: flytepropeller + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm +rules: +# Allow RO access to PODS +- apiGroups: + - "" + resources: + - pods + verbs: + - get + - list + - watch +# Allow Event recording access +- apiGroups: + - "" + resources: + - events + verbs: + - create + - update + - delete + - patch +# Allow Access All plugin objects +- apiGroups: + - '*' + resources: + - '*' + verbs: + - get + - list + - watch + - create + - update + - delete + - patch +# Allow Access to CRD +- apiGroups: + - apiextensions.k8s.io + resources: + - customresourcedefinitions + verbs: + - get + - list + - watch + - create + - delete + - update +# Allow Access to all resources under flyte.lyft.com +- apiGroups: + - flyte.lyft.com + resources: + - flyteworkflows + - flyteworkflows/finalizers + verbs: + - get + - list + - watch + - create + - update + - delete + - patch + - post + - deletecollection +--- +# Source: flyte/templates/propeller/webhook.yaml +# Create a ClusterRole for the webhook +# https://kubernetes.io/docs/admin/authorization/rbac/ +kind: ClusterRole +apiVersion: rbac.authorization.k8s.io/v1 +metadata: + name: flyte-pod-webhook + namespace: default +rules: + - apiGroups: + - "*" + resources: + - mutatingwebhookconfigurations + - secrets + - pods + verbs: + - get + - create + - update + - patch +--- +# Source: flyte/templates/pytorch-operator/rbac.yaml +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: pytorch-operator-admin + labels: + app.kubernetes.io/name: pytorch-operator + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm +rules: [] +aggregationRule: + clusterRoleSelectors: + - matchLabels: + rbac.authorization.kubeflow.org/aggregate-to-kubeflow-pytorchjobs-admin: "true" +--- +# Source: flyte/templates/pytorch-operator/rbac.yaml +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: pytorch-operator-edit + labels: + app.kubernetes.io/name: pytorch-operator + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm +rules: +- apiGroups: + - kubeflow.org + resources: + - pytorchjobs + - pytorchjobs/status + verbs: + - get + - list + - watch + - create + - delete + - deletecollection + - patch + - update +--- +# Source: flyte/templates/pytorch-operator/rbac.yaml +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: pytorch-operator-view + labels: + app.kubernetes.io/name: pytorch-operator + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm +rules: +- apiGroups: + - kubeflow.org + resources: + - pytorchjobs + - pytorchjobs/status + verbs: + - get + - list + - watch +--- +# Source: flyte/templates/pytorch-operator/rbac.yaml +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: pytorch-operator + labels: + app.kubernetes.io/name: pytorch-operator + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm +rules: +- apiGroups: + - kubeflow.org + resources: + - pytorchjobs + - pytorchjobs/status + verbs: + - '*' +- apiGroups: + - apiextensions.k8s.io + resources: + - customresourcedefinitions + verbs: + - '*' +- apiGroups: + - "" + resources: + - pods + - services + - endpoints + - events + verbs: + - '*' +--- +# Source: flyte/charts/contour/templates/contour/rbac.yaml +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding +metadata: + name: flyte-contour-contour + labels: + app.kubernetes.io/name: contour + helm.sh/chart: contour-4.1.2 + app.kubernetes.io/instance: flyte + app.kubernetes.io/managed-by: Helm +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: flyte-contour-contour +subjects: + - kind: ServiceAccount + name: flyte-contour-contour + namespace: default +--- +# Source: flyte/charts/sparkoperator/templates/rbac.yaml +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding +metadata: + name: flyte-sparkoperator + labels: + helm.sh/chart: sparkoperator-1.0.6 + app.kubernetes.io/name: sparkoperator + app.kubernetes.io/instance: flyte + app.kubernetes.io/version: "v1beta2-1.2.0-3.0.0" + app.kubernetes.io/managed-by: Helm +subjects: + - kind: ServiceAccount + name: flyte-sparkoperator + namespace: default +roleRef: + kind: ClusterRole + name: flyte-sparkoperator + apiGroup: rbac.authorization.k8s.io +--- +# Source: flyte/templates/admin/rbac.yaml +apiVersion: rbac.authorization.k8s.io/v1beta1 +kind: ClusterRoleBinding +metadata: + name: flyteadmin-binding + labels: + app.kubernetes.io/name: flyteadmin + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: flyteadmin +subjects: +- kind: ServiceAccount + name: flyteadmin + namespace: default +--- +# Source: flyte/templates/propeller/rbac.yaml +apiVersion: rbac.authorization.k8s.io/v1beta1 +kind: ClusterRoleBinding +metadata: + name: flytepropeller + labels: + app.kubernetes.io/name: flytepropeller + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: flytepropeller +subjects: +- kind: ServiceAccount + name: flytepropeller + namespace: default +--- +# Source: flyte/templates/propeller/webhook.yaml +# Create a binding from Role -> ServiceAccount +kind: ClusterRoleBinding +apiVersion: rbac.authorization.k8s.io/v1 +metadata: + name: flyte-pod-webhook + namespace: default +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: flyte-pod-webhook +subjects: + - kind: ServiceAccount + name: flyte-pod-webhook + namespace: default +--- +# Source: flyte/templates/pytorch-operator/rbac.yaml +apiVersion: rbac.authorization.k8s.io/v1beta1 +kind: ClusterRoleBinding +metadata: + name: pytorch-operator + labels: + app.kubernetes.io/name: pytorch-operator + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: pytorch-operator +subjects: +- kind: ServiceAccount + name: pytorch-operator + namespace: pytorch-operator +--- +# Source: flyte/charts/sparkoperator/templates/spark-rbac.yaml +apiVersion: rbac.authorization.k8s.io/v1 +kind: Role +metadata: + name: spark-role + namespace: default + labels: + helm.sh/chart: sparkoperator-1.0.6 + app.kubernetes.io/name: sparkoperator + app.kubernetes.io/instance: flyte + app.kubernetes.io/version: "v1beta2-1.2.0-3.0.0" + app.kubernetes.io/managed-by: Helm +rules: +- apiGroups: + - "" + resources: + - pods + verbs: + - "*" +- apiGroups: + - "" + resources: + - services + verbs: + - "*" +--- +# Source: flyte/charts/sparkoperator/templates/spark-rbac.yaml +apiVersion: rbac.authorization.k8s.io/v1 +kind: RoleBinding +metadata: + name: spark + namespace: default + labels: + helm.sh/chart: sparkoperator-1.0.6 + app.kubernetes.io/name: sparkoperator + app.kubernetes.io/instance: flyte + app.kubernetes.io/version: "v1beta2-1.2.0-3.0.0" + app.kubernetes.io/managed-by: Helm +subjects: +- kind: ServiceAccount + name: flyte-spark + namespace: default +roleRef: + kind: Role + name: spark-role + apiGroup: rbac.authorization.k8s.io +--- +# Source: flyte/charts/contour/templates/contour/service.yaml +apiVersion: v1 +kind: Service +metadata: + name: flyte-contour + namespace: default + labels: + app.kubernetes.io/name: contour + helm.sh/chart: contour-4.1.2 + app.kubernetes.io/instance: flyte + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/component: contour +spec: + ports: + - port: 8001 + name: xds + protocol: TCP + targetPort: 8001 + selector: + app.kubernetes.io/name: contour + app.kubernetes.io/instance: flyte + app.kubernetes.io/component: contour + type: ClusterIP +--- +# Source: flyte/charts/contour/templates/envoy/service.yaml +apiVersion: v1 +kind: Service +metadata: + name: flyte-contour-envoy + namespace: default + labels: + app.kubernetes.io/name: contour + helm.sh/chart: contour-4.1.2 + app.kubernetes.io/instance: flyte + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/component: envoy + annotations: + # This annotation puts the AWS ELB into "TCP" mode so that it does not + # do HTTP negotiation for HTTPS connections at the ELB edge. + # The downside of this is the remote IP address of all connections will + # appear to be the internal address of the ELB. See docs/proxy-proto.md + # for information about enabling the PROXY protocol on the ELB to recover + # the original remote IP address. + # We don't set this for nlb, per the contour docs. + service.beta.kubernetes.io/aws-load-balancer-backend-protocol: tcp +spec: + externalTrafficPolicy: "Local" + ports: + - name: http + port: 80 + protocol: TCP + targetPort: http + - name: https + port: 443 + protocol: TCP + targetPort: https + selector: + app.kubernetes.io/name: contour + app.kubernetes.io/instance: flyte + app.kubernetes.io/component: envoy + type: LoadBalancer +--- +# Source: flyte/templates/admin/service.yaml +apiVersion: v1 +kind: Service +metadata: + name: flyteadmin + namespace: default + labels: + app.kubernetes.io/name: flyteadmin + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm + annotations: + projectcontour.io/upstream-protocol.h2c: grpc +spec: + type: ClusterIP + ports: + - name: http + port: 80 + protocol: TCP + targetPort: 8088 + - name: grpc + port: 81 + protocol: TCP + targetPort: 8089 + - name: redoc + protocol: TCP + port: 87 + targetPort: 8087 + - name: http-metrics + protocol: TCP + port: 10254 + selector: + app.kubernetes.io/name: flyteadmin + app.kubernetes.io/instance: flyte +--- +# Source: flyte/templates/console/service.yaml +apiVersion: v1 +kind: Service +metadata: + name: flyteconsole + namespace: default + labels: + app.kubernetes.io/name: flyteconsole + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm +spec: + type: ClusterIP + ports: + - port: 80 + protocol: TCP + targetPort: 8080 + selector: + app.kubernetes.io/name: flyteconsole + app.kubernetes.io/instance: flyte +--- +# Source: flyte/templates/datacatalog/service.yaml +apiVersion: v1 +kind: Service +metadata: + name: datacatalog + namespace: default + labels: + app.kubernetes.io/name: datacatalog + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm + annotations: + projectcontour.io/upstream-protocol.h2c: grpc +spec: + type: NodePort + ports: + - name: grpc-2 + port: 8089 + protocol: TCP + targetPort: 8089 + - name: http + port: 88 + protocol: TCP + targetPort: 8088 + - name: grpc + port: 89 + protocol: TCP + targetPort: 8089 + selector: + app.kubernetes.io/name: datacatalog + app.kubernetes.io/instance: flyte +--- +# Source: flyte/templates/minio/service.yaml +apiVersion: v1 +kind: Service +metadata: + name: minio + namespace: default + labels: + app.kubernetes.io/name: minio + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm +spec: + type: ClusterIP + ports: + - name: minio + port: 9000 + protocol: TCP + targetPort: minio + selector: + app.kubernetes.io/name: minio + app.kubernetes.io/instance: flyte +--- +# Source: flyte/templates/postgres/service.yaml +apiVersion: v1 +kind: Service +metadata: + name: postgres + namespace: default + labels: + app.kubernetes.io/name: postgres + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm +spec: + type: ClusterIP + ports: + - name: postgres + port: 5432 + protocol: TCP + targetPort: postgres + selector: + app.kubernetes.io/name: postgres + app.kubernetes.io/instance: flyte +--- +# Source: flyte/templates/propeller/webhook.yaml +# Service +apiVersion: v1 +kind: Service +metadata: + name: flyte-pod-webhook + namespace: default + annotations: + projectcontour.io/upstream-protocol.h2c: grpc +spec: + selector: + app: flyte-pod-webhook + ports: + - name: https + protocol: TCP + port: 443 + targetPort: 9443 +--- +# Source: flyte/templates/pytorch-operator/service.yaml +apiVersion: v1 +kind: Service +metadata: + name: pytorch-operator + namespace: pytorch-operator + labels: + app.kubernetes.io/name: pytorch-operator + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm +spec: + type: ClusterIP + ports: + - name: monitoring-port + port: 8443 + targetPort: 8443 + selector: + app.kubernetes.io/name: pytorch-operator + app.kubernetes.io/instance: flyte +--- +# Source: flyte/templates/redis/service.yaml +apiVersion: v1 +kind: Service +metadata: + name: redis-resource-manager + namespace: default + labels: + app.kubernetes.io/name: redis + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm +spec: + type: ClusterIP + ports: + - name: redis + port: 6379 + protocol: TCP + targetPort: redis + selector: + app.kubernetes.io/name: redis + app.kubernetes.io/instance: flyte +--- +# Source: flyte/charts/contour/templates/envoy/daemonset.yaml +apiVersion: apps/v1 +kind: DaemonSet +metadata: + name: flyte-contour-envoy + namespace: default + labels: + app.kubernetes.io/name: contour + helm.sh/chart: contour-4.1.2 + app.kubernetes.io/instance: flyte + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/component: envoy +spec: + updateStrategy: + type: RollingUpdate + rollingUpdate: + maxUnavailable: 10% + selector: + matchLabels: + app.kubernetes.io/name: contour + app.kubernetes.io/instance: flyte + app.kubernetes.io/component: envoy + template: + metadata: + labels: + app.kubernetes.io/name: contour + helm.sh/chart: contour-4.1.2 + app.kubernetes.io/instance: flyte + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/component: envoy + spec: + + affinity: + podAffinity: + + podAntiAffinity: + + nodeAffinity: + + terminationGracePeriodSeconds: 300 + hostNetwork: false + dnsPolicy: ClusterFirst + containers: + - command: + - contour + args: + - envoy + - shutdown-manager + image: docker.io/bitnami/contour:1.12.0-debian-10-r0 + imagePullPolicy: IfNotPresent + lifecycle: + preStop: + exec: + command: + - contour + - envoy + - shutdown + livenessProbe: + httpGet: + path: /healthz + port: 8090 + initialDelaySeconds: 120 + periodSeconds: 20 + timeoutSeconds: 5 + successThreshold: 1 + failureThreshold: 6 + name: shutdown-manager + - command: + - envoy + args: + - -c + - /config/envoy.json + - --service-cluster $(CONTOUR_NAMESPACE) + - --service-node $(ENVOY_POD_NAME) + - --log-level info + image: docker.io/bitnami/envoy:1.16.2-debian-10-r54 + imagePullPolicy: IfNotPresent + name: envoy + securityContext: + runAsUser: 0 + env: + - name: CONTOUR_NAMESPACE + valueFrom: + fieldRef: + apiVersion: v1 + fieldPath: metadata.namespace + - name: ENVOY_POD_NAME + valueFrom: + fieldRef: + apiVersion: v1 + fieldPath: metadata.name + ports: + - containerPort: 80 + hostPort: 80 + name: http + protocol: TCP + - containerPort: 443 + hostPort: 443 + name: https + protocol: TCP + - containerPort: 8002 + name: metrics + protocol: TCP + readinessProbe: + httpGet: + path: /ready + port: 8002 + initialDelaySeconds: 10 + periodSeconds: 3 + timeoutSeconds: 1 + successThreshold: 1 + failureThreshold: 3 + livenessProbe: + httpGet: + path: /ready + port: 8002 + initialDelaySeconds: 120 + periodSeconds: 20 + timeoutSeconds: 5 + successThreshold: 1 + failureThreshold: 6 + resources: + limits: + cpu: 100m + memory: 100Mi + requests: + cpu: 10m + memory: 50Mi + volumeMounts: + - name: envoy-config + mountPath: /config + - name: envoycert + mountPath: /certs + lifecycle: + preStop: + httpGet: + path: /shutdown + port: 8090 + scheme: HTTP + initContainers: + - command: + - contour + args: + - bootstrap + - /config/envoy.json + - --xds-address=flyte-contour + - --xds-port=8001 + - --resources-dir=/config/resources + - --envoy-cafile=/certs/ca.crt + - --envoy-cert-file=/certs/tls.crt + - --envoy-key-file=/certs/tls.key + image: docker.io/bitnami/contour:1.12.0-debian-10-r0 + imagePullPolicy: IfNotPresent + name: envoy-initconfig + resources: + limits: + cpu: 100m + memory: 100Mi + requests: + cpu: 10m + memory: 50Mi + volumeMounts: + - name: envoy-config + mountPath: /config + - name: envoycert + mountPath: /certs + readOnly: true + env: + - name: CONTOUR_NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace + automountServiceAccountToken: false + serviceAccountName: flyte-contour-envoy + volumes: + - name: envoy-config + emptyDir: {} + - name: envoycert + secret: + secretName: envoycert + restartPolicy: Always +--- +# Source: flyte/charts/contour/templates/contour/deployment.yaml +apiVersion: apps/v1 +kind: Deployment +metadata: + name: flyte-contour-contour + namespace: default + labels: + app.kubernetes.io/name: contour + helm.sh/chart: contour-4.1.2 + app.kubernetes.io/instance: flyte + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/component: contour +spec: + replicas: 1 + strategy: + type: RollingUpdate + rollingUpdate: + # This value of maxSurge means that during a rolling update + # the new ReplicaSet will be created first. + maxSurge: 50% + selector: + matchLabels: + app.kubernetes.io/name: contour + app.kubernetes.io/instance: flyte + app.kubernetes.io/component: contour + template: + metadata: + annotations: + checksum/config: 1c6f180e2479c6aff9159e6eea066d5238a8c9ee1f38f19a6de2f070a88e5ad9 + labels: + app.kubernetes.io/name: contour + helm.sh/chart: contour-4.1.2 + app.kubernetes.io/instance: flyte + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/component: contour + spec: + + affinity: + podAffinity: + + podAntiAffinity: + preferredDuringSchedulingIgnoredDuringExecution: + - podAffinityTerm: + labelSelector: + matchLabels: + app.kubernetes.io/name: contour + app.kubernetes.io/instance: flyte + app.kubernetes.io/component: contour + namespaces: + - "default" + topologyKey: kubernetes.io/hostname + weight: 1 + nodeAffinity: + + containers: + - command: + - contour + args: + - serve + - --incluster + - --xds-address=0.0.0.0 + - --xds-port=8001 + - --envoy-service-http-port=80 + - --envoy-service-https-port=443 + - --contour-cafile=/certs/ca.crt + - --contour-cert-file=/certs/tls.crt + - --contour-key-file=/certs/tls.key + - --config-path=/config/contour.yaml + image: docker.io/bitnami/contour:1.12.0-debian-10-r0 + imagePullPolicy: IfNotPresent + name: contour + ports: + - containerPort: 8001 + name: xds + protocol: TCP + - containerPort: 8000 + name: metrics + protocol: TCP + livenessProbe: + httpGet: + path: /healthz + port: 8000 + initialDelaySeconds: 120 + periodSeconds: 20 + timeoutSeconds: 5 + successThreshold: 1 + failureThreshold: 6 + readinessProbe: + tcpSocket: + port: 8001 + initialDelaySeconds: 15 + periodSeconds: 10 + timeoutSeconds: 5 + successThreshold: 1 + failureThreshold: 3 + resources: + limits: + cpu: 100m + memory: 100Mi + requests: + cpu: 10m + memory: 50Mi + volumeMounts: + - name: contourcert + mountPath: /certs + readOnly: true + - name: contour-config + mountPath: /config + readOnly: true + env: + - name: CONTOUR_NAMESPACE + valueFrom: + fieldRef: + apiVersion: v1 + fieldPath: metadata.namespace + - name: POD_NAME + valueFrom: + fieldRef: + apiVersion: v1 + fieldPath: metadata.name + dnsPolicy: ClusterFirst + serviceAccountName: flyte-contour-contour + securityContext: + runAsUser: 1001 + runAsGroup: 1001 + fsGroup: + runAsNonRoot: true + volumes: + - name: contourcert + secret: + secretName: contourcert + - name: contour-config + configMap: + name: flyte-contour + defaultMode: 0644 + items: + - key: contour.yaml + path: contour.yaml +--- +# Source: flyte/charts/sparkoperator/templates/deployment.yaml +# If the admission webhook is enabled, then a post-install step is required +# to generate and install the secret in the operator namespace. + +# In the post-install hook, the token corresponding to the operator service account +# is used to authenticate with the Kubernetes API server to install the secret bundle. + +apiVersion: apps/v1 +kind: Deployment +metadata: + name: flyte-sparkoperator + labels: + helm.sh/chart: sparkoperator-1.0.6 + app.kubernetes.io/name: sparkoperator + app.kubernetes.io/instance: flyte + app.kubernetes.io/version: "v1beta2-1.2.0-3.0.0" + app.kubernetes.io/managed-by: Helm +spec: + replicas: 1 + selector: + matchLabels: + app.kubernetes.io/name: sparkoperator + app.kubernetes.io/instance: flyte + strategy: + type: Recreate + template: + metadata: + annotations: + prometheus.io/scrape: "true" + prometheus.io/port: "10254" + prometheus.io/path: /metrics + labels: + app.kubernetes.io/name: sparkoperator + app.kubernetes.io/instance: flyte + spec: + serviceAccountName: flyte-sparkoperator + securityContext: + {} + containers: + - name: sparkoperator + image: gcr.io/spark-operator/spark-operator:v1beta2-1.2.0-3.0.0 + imagePullPolicy: IfNotPresent + securityContext: + {} + ports: + - name: "metrics" + containerPort: 10254 + + args: + - -v=2 + - -logtostderr + - -namespace= + - -ingress-url-format= + - -controller-threads=10 + - -resync-interval=30 + - -enable-batch-scheduler=false + - -enable-metrics=true + - -metrics-labels=app_type + - -metrics-port=10254 + - -metrics-endpoint=/metrics + - -metrics-prefix= + - -enable-resource-quota-enforcement=false + resources: + limits: + cpu: 1000m + memory: 500M + requests: + cpu: 10m + memory: 50M +--- +# Source: flyte/templates/admin/deployment.yaml +apiVersion: apps/v1 +kind: Deployment +metadata: + name: flyteadmin + namespace: default + labels: + app.kubernetes.io/name: flyteadmin + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm +spec: + replicas: 1 + selector: + matchLabels: + app.kubernetes.io/name: flyteadmin + app.kubernetes.io/instance: flyte + template: + metadata: + annotations: + configChecksum: "8b55c48b5efdac613e43466b270be585430a0585ad2865c1ac6c6358fa5643e" + labels: + app.kubernetes.io/name: flyteadmin + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm + spec: + initContainers: + - name: check-db-ready + image: postgres:10.16-alpine + command: + - sh + - -c + - until pg_isready -h postgres -p 5432; do echo waiting for database; sleep 2; done; + - command: + - flyteadmin + - --config + - /etc/flyte/config/*.yaml + - migrate + - run + image: "ghcr.io/flyteorg/flyteadmin:v0.5.7" + imagePullPolicy: "IfNotPresent" + name: run-migrations + volumeMounts: + + - mountPath: /etc/flyte/config + name: config-volume + - command: + - flyteadmin + - --config + - /etc/flyte/config/*.yaml + - migrate + - seed-projects + - flytesnacks + - flytetester + - flyteexamples + image: "ghcr.io/flyteorg/flyteadmin:v0.5.7" + imagePullPolicy: "IfNotPresent" + name: seed-projects + volumeMounts: + + - mountPath: /etc/flyte/config + name: config-volume + - command: + - flyteadmin + - --config + - /etc/flyte/config/*.yaml + - clusterresource + - sync + image: "ghcr.io/flyteorg/flyteadmin:v0.5.7" + imagePullPolicy: "IfNotPresent" + name: sync-cluster-resources + volumeMounts: + + - mountPath: /etc/flyte/clusterresource/templates + name: resource-templates + - mountPath: /etc/flyte/config + name: config-volume + - name: generate-secrets + image: "ghcr.io/flyteorg/flyteadmin:v0.5.7" + imagePullPolicy: "IfNotPresent" + command: ["/bin/sh", "-c"] + args: + [ + "flyteadmin --config=/etc/flyte/config/*.yaml secrets init --localPath /etc/secrets/auth && flyteadmin --config=/etc/flyte/config/*.yaml secrets create --fromPath /etc/secrets/auth", + ] + volumeMounts: + - name: config-volume + mountPath: /etc/flyte/config + env: + - name: POD_NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace + containers: + - command: + - flyteadmin + - --config + - /etc/flyte/config/*.yaml + - serve + image: "ghcr.io/flyteorg/flyteadmin:v0.5.7" + imagePullPolicy: "IfNotPresent" + name: flyteadmin + ports: + - containerPort: 8088 + - containerPort: 8089 + resources: + limits: + cpu: 250m + ephemeral-storage: 100Mi + memory: 500Mi + requests: + cpu: 10m + ephemeral-storage: 50Mi + memory: 50Mi + volumeMounts: + + - mountPath: /srv/flyte + name: shared-data + - mountPath: /etc/flyte/config + name: config-volume + - name: auth + mountPath: /etc/secrets/ + - command: + - sh + - -c + - ln -s /usr/share/nginx/html /usr/share/nginx/html/openapi && sh /usr/local/bin/docker-run.sh + env: + - name: PAGE_TITLE + value: Flyte Admin OpenAPI + - name: SPEC_URL + value: /api/v1/openapi + - name: PORT + value: "8087" + image: docker.io/redocly/redoc + imagePullPolicy: IfNotPresent + name: redoc + ports: + - containerPort: 8087 + resources: + limits: + cpu: "0.1" + memory: 200Mi + serviceAccountName: flyteadmin + volumes: + + - emptyDir: {} + name: shared-data + - configMap: + name: flyte-admin-config + name: config-volume + - configMap: + name: clusterresource-template + name: resource-templates + - name: auth + secret: + secretName: flyte-admin-auth +--- +# Source: flyte/templates/console/deployment.yaml +apiVersion: apps/v1 +kind: Deployment +metadata: + name: flyteconsole + namespace: default + labels: + app.kubernetes.io/name: flyteconsole + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm +spec: + replicas: 1 + selector: + matchLabels: + app.kubernetes.io/name: flyteconsole + app.kubernetes.io/instance: flyte + template: + metadata: + annotations: + configChecksum: "a14c2477f002d1392494a730c37ba7109e61df5cd417d431efa36841ee56bb9" + labels: + app.kubernetes.io/name: flyteconsole + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm + spec: + containers: + - image: "ghcr.io/flyteorg/flyteconsole:v0.20.0" + imagePullPolicy: "IfNotPresent" + name: flyteconsole + envFrom: + - configMapRef: + name: flyte-console-config + ports: + - containerPort: 8080 + resources: + limits: + cpu: 500m + memory: 250Mi + requests: + cpu: 10m + memory: 50Mi + volumeMounts: + - mountPath: /srv/flyte + name: shared-data + volumes: + - emptyDir: {} + name: shared-data +--- +# Source: flyte/templates/datacatalog/deployment.yaml +apiVersion: apps/v1 +kind: Deployment +metadata: + name: datacatalog + namespace: default + labels: + app.kubernetes.io/name: datacatalog + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm +spec: + replicas: 1 + selector: + matchLabels: + app.kubernetes.io/name: datacatalog + app.kubernetes.io/instance: flyte + template: + metadata: + annotations: + configChecksum: "d63a2c1822443788cd796cf771ba9e4c722d7a93729e9f8f153b03e23e918da" + labels: + app.kubernetes.io/name: datacatalog + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm + spec: + initContainers: + - command: + - datacatalog + - --config + - /etc/datacatalog/config/*.yaml + - migrate + - run + image: "ghcr.io/flyteorg/datacatalog:v0.3.4" + imagePullPolicy: "IfNotPresent" + name: run-migrations + volumeMounts: + + - mountPath: /etc/datacatalog/config + name: config-volume + containers: + - command: + - datacatalog + - --config + - /etc/datacatalog/config/*.yaml + - serve + image: "ghcr.io/flyteorg/datacatalog:v0.3.4" + imagePullPolicy: "IfNotPresent" + name: datacatalog + ports: + - containerPort: 8088 + - containerPort: 8089 + resources: + limits: + cpu: 500m + ephemeral-storage: 100Mi + memory: 500Mi + requests: + cpu: 10m + ephemeral-storage: 50Mi + memory: 50Mi + volumeMounts: + + - mountPath: /etc/datacatalog/config + name: config-volume + serviceAccountName: datacatalog + volumes: + + - emptyDir: {} + name: shared-data + - configMap: + name: datacatalog-config + name: config-volume +--- +# Source: flyte/templates/minio/deployment.yaml +apiVersion: apps/v1 +kind: Deployment +metadata: + name: minio + namespace: default + labels: + app.kubernetes.io/name: minio + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm +spec: + replicas: 1 + selector: + matchLabels: + app.kubernetes.io/name: minio + app.kubernetes.io/instance: flyte + template: + metadata: + labels: + app.kubernetes.io/name: minio + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm + spec: + containers: + - image: "minio/minio:RELEASE.2020-12-16T05-05-17Z" + imagePullPolicy: "IfNotPresent" + name: minio + args: + - server + - /data + env: + - name: MINIO_ACCESS_KEY + value: minio + - name: MINIO_SECRET_KEY + value: miniostorage + ports: + - containerPort: 9000 + name: minio + resources: + limits: + cpu: 200m + memory: 512Mi + requests: + cpu: 10m + memory: 128Mi + volumeMounts: + - name: minio-storage + mountPath: /var/lib/minioql/data + volumes: + - name: minio-storage + emptyDir: {} +--- +# Source: flyte/templates/postgres/deployment.yaml +apiVersion: apps/v1 +kind: Deployment +metadata: + name: postgres + namespace: default + labels: + app.kubernetes.io/name: postgres + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm +spec: + replicas: 1 + selector: + matchLabels: + app.kubernetes.io/name: postgres + app.kubernetes.io/instance: flyte + template: + metadata: + labels: + app.kubernetes.io/name: postgres + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm + spec: + containers: + - image: "postgres:10.16" + imagePullPolicy: "IfNotPresent" + name: postgres + env: + - name: POSTGRES_HOST_AUTH_METHOD + value: trust + ports: + - containerPort: 5432 + name: postgres + resources: + limits: + cpu: 1000m + memory: 512Mi + requests: + cpu: 10m + memory: 128Mi + volumeMounts: + - name: postgres-storage + mountPath: /var/lib/postgresql/data + volumes: + - name: postgres-storage + emptyDir: {} +--- +# Source: flyte/templates/propeller/deployment.yaml +apiVersion: apps/v1 +kind: Deployment +metadata: + name: flytepropeller + namespace: default + labels: + app.kubernetes.io/name: flytepropeller + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm +spec: + replicas: 1 + selector: + matchLabels: + app.kubernetes.io/name: flytepropeller + app.kubernetes.io/instance: flyte + template: + metadata: + annotations: + configChecksum: "7ce67c66483a6263afa5bed97db5cfb2bf546b52baf95e6880ea3f2d3a11f42" + labels: + app.kubernetes.io/name: flytepropeller + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm + spec: + containers: + - command: + - flytepropeller + - --config + - /etc/flyte/config/*.yaml + env: + - name: POD_NAME + valueFrom: + fieldRef: + fieldPath: metadata.name + image: "ghcr.io/flyteorg/flytepropeller:v0.10.10" + imagePullPolicy: "IfNotPresent" + name: flytepropeller + ports: + - containerPort: 10254 + resources: + limits: + cpu: 200m + ephemeral-storage: 100Mi + memory: 200Mi + requests: + cpu: 10m + ephemeral-storage: 50Mi + memory: 50Mi + volumeMounts: + - name: config-volume + mountPath: /etc/flyte/config + - name: auth + mountPath: /etc/secrets/ + serviceAccountName: flytepropeller + volumes: + - configMap: + name: flyte-propeller-config + name: config-volume + - name: auth + secret: + secretName: flyte-propeller-auth +--- +# Source: flyte/templates/propeller/webhook.yaml +# Create the actual deployment +apiVersion: apps/v1 +kind: Deployment +metadata: + name: flyte-pod-webhook + namespace: default + labels: + app: flyte-pod-webhook +spec: + selector: + matchLabels: + app: flyte-pod-webhook + template: + metadata: + labels: + app: flyte-pod-webhook + app.kubernetes.io/name: flyte-pod-webhook + app.kubernetes.io/version: v0.10.10 + annotations: + configChecksum: "7ce67c66483a6263afa5bed97db5cfb2bf546b52baf95e6880ea3f2d3a11f42" + spec: + serviceAccountName: flyte-pod-webhook + initContainers: + - name: generate-secrets + image: "ghcr.io/flyteorg/flytepropeller:v0.10.10" + imagePullPolicy: "IfNotPresent" + command: + - flytepropeller + args: + - webhook + - init-certs + - --config + - /etc/flyte/config/*.yaml + env: + - name: POD_NAME + valueFrom: + fieldRef: + fieldPath: metadata.name + - name: POD_NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace + volumeMounts: + - name: config-volume + mountPath: /etc/flyte/config + containers: + - name: webhook + image: "ghcr.io/flyteorg/flytepropeller:v0.10.10" + imagePullPolicy: "IfNotPresent" + command: + - flytepropeller + args: + - webhook + - --config + - /etc/flyte/config/*.yaml + env: + - name: POD_NAME + valueFrom: + fieldRef: + fieldPath: metadata.name + - name: POD_NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace + volumeMounts: + - name: config-volume + mountPath: /etc/flyte/config + readOnly: true + - name: webhook-certs + mountPath: /etc/webhook/certs + readOnly: true + volumes: + - name: config-volume + configMap: + name: flyte-propeller-config + - name: webhook-certs + secret: + secretName: flyte-pod-webhook +--- +# Source: flyte/templates/pytorch-operator/deployment.yaml +apiVersion: apps/v1 +kind: Deployment +metadata: + name: pytorch-operator + namespace: pytorch-operator + labels: + app.kubernetes.io/name: pytorch-operator + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm +spec: + replicas: 1 + selector: + matchLabels: + app.kubernetes.io/name: pytorch-operator + app.kubernetes.io/instance: flyte + template: + metadata: + labels: + app.kubernetes.io/name: pytorch-operator + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm + spec: + containers: + - image: "gcr.io/kubeflow-images-public/pytorch-operator:v1.0.0-g047cf0f" + imagePullPolicy: "IfNotPresent" + name: pytorch-operator + command: + - /pytorch-operator.v1 + - --alsologtostderr + - -v=1 + - --monitoring-port=8443 + env: + - name: MY_POD_NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace + - name: MY_POD_NAME + valueFrom: + fieldRef: + fieldPath: metadata.name + resources: + limits: + cpu: 500m + memory: 1000M + requests: + cpu: 10m + memory: 50M + serviceAccountName: pytorch-operator +--- +# Source: flyte/templates/redis/statefulset.yaml +apiVersion: apps/v1 +kind: StatefulSet +metadata: + name: redis + namespace: default + labels: + app.kubernetes.io/name: redis + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm +spec: + replicas: 1 + selector: + matchLabels: + app.kubernetes.io/name: redis + app.kubernetes.io/instance: flyte + serviceName: redis-resource-manager + template: + metadata: + labels: + app.kubernetes.io/name: redis + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm + spec: + containers: + - image: "docker.io/bitnami/redis:4.0.2-r1" + imagePullPolicy: "IfNotPresent" + name: redis + env: + - name: REDIS_PASSWORD + value: mypassword + livenessProbe: + exec: + command: + - redis-cli + - ping + failureThreshold: 3 + initialDelaySeconds: 30 + periodSeconds: 10 + successThreshold: 1 + timeoutSeconds: 5 + readinessProbe: + exec: + command: + - redis-cli + - ping + failureThreshold: 3 + initialDelaySeconds: 5 + periodSeconds: 10 + successThreshold: 1 + timeoutSeconds: 1 + ports: + - containerPort: 6379 + name: redis + protocol: TCP + resources: + limits: + cpu: 1000m + memory: 1Gi + requests: + cpu: 10m + memory: 50Mi + volumeMounts: + - mountPath: /bitnami + name: redis-data + + dnsPolicy: ClusterFirst + restartPolicy: Always + volumes: + - emptyDir: {} + name: redis-data +--- +# Source: flyte/templates/admin/cronjob.yaml +apiVersion: batch/v1beta1 +kind: CronJob +metadata: + name: syncresources + namespace: default + labels: + app.kubernetes.io/name: flyteadmin + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm +spec: + schedule: '*/1 * * * *' + jobTemplate: + spec: + template: + spec: + containers: + - command: + - flyteadmin + - --config + - /etc/flyte/config/*.yaml + - clusterresource + - sync + image: "ghcr.io/flyteorg/flyteadmin:v0.5.7" + imagePullPolicy: "IfNotPresent" + name: sync-cluster-resources + volumeMounts: + + - mountPath: /etc/flyte/clusterresource/templates + name: resource-templates + - mountPath: /etc/flyte/config + name: config-volume + restartPolicy: OnFailure + serviceAccountName: flyteadmin + volumes: + + - configMap: + name: clusterresource-template + name: resource-templates + - configMap: + name: flyte-admin-config + name: config-volume +--- +# Source: flyte/templates/common/ingress.yaml +apiVersion: networking.k8s.io/v1beta1 +kind: Ingress +metadata: + name: flyte + namespace: default +spec: + rules: + - http: + paths: + # This is useful only for frontend development + # Port 87 in FlyteAdmin maps to the redoc container. + - path: /openapi + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 87 + # NOTE: If you change this, you must update the BASE_URL value in flyteconsole.yaml + - path: /console + pathType: ImplementationSpecific + backend: + serviceName: flyteconsole + servicePort: 80 + - path: /console/* + pathType: ImplementationSpecific + backend: + serviceName: flyteconsole + servicePort: 80 + - path: /api + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 80 + - path: /api/* + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 80 + - path: /healthcheck + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 80 + - path: /v1/* + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 80 + # Port 87 in FlyteAdmin maps to the redoc container. + - path: /openapi/* + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 80 + - path: /.well-known + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 80 + - path: /.well-known/* + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 80 + - path: /login + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 80 + - path: /login/* + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 80 + - path: /logout + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 80 + - path: /logout/* + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 80 + - path: /callback + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 80 + - path: /callback/* + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 80 + - path: /me + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 80 + - path: /config + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 80 + - path: /config/* + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 80 + # NOTE: Port 81 in flyteadmin is the GRPC server port for FlyteAdmin. + - path: /flyteidl.service.AdminService + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 81 + - path: /flyteidl.service.AdminService/* + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 81 + - path: /flyteidl.service.AuthMetadataService + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 81 + - path: /flyteidl.service.AuthMetadataService/* + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 81 + - path: /flyteidl.service.IdentityService + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 81 + - path: /flyteidl.service.IdentityService/* + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 81 +--- +# Source: flyte/charts/contour/templates/certgen/serviceaccount.yaml +apiVersion: v1 +kind: ServiceAccount +metadata: + name: flyte-contour-contour-certgen + namespace: default + annotations: + "helm.sh/hook": "pre-install,pre-upgrade" + "helm.sh/hook-delete-policy": before-hook-creation,hook-succeeded + labels: + app.kubernetes.io/name: contour + helm.sh/chart: contour-4.1.2 + app.kubernetes.io/instance: flyte + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/component: contour-certgen +--- +# Source: flyte/charts/contour/templates/certgen/rbac.yaml +apiVersion: rbac.authorization.k8s.io/v1 +kind: Role +metadata: + name: flyte-contour-contour-certgen + namespace: default + annotations: + "helm.sh/hook": "pre-install,pre-upgrade" + "helm.sh/hook-delete-policy": before-hook-creation,hook-succeeded + labels: + app.kubernetes.io/name: contour + helm.sh/chart: contour-4.1.2 + app.kubernetes.io/instance: flyte + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/component: contour-certgen +rules: + - apiGroups: + - "" + resources: + - secrets + verbs: + - create + - update +--- +# Source: flyte/charts/contour/templates/certgen/rbac.yaml +apiVersion: rbac.authorization.k8s.io/v1 +kind: RoleBinding +metadata: + name: flyte-contour-contour-certgen + namespace: default + annotations: + "helm.sh/hook": "pre-install,pre-upgrade" + "helm.sh/hook-delete-policy": before-hook-creation,hook-succeeded + labels: + app.kubernetes.io/name: contour + helm.sh/chart: contour-4.1.2 + app.kubernetes.io/instance: flyte + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/component: contour-certgen +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: Role + name: flyte-contour-contour-certgen +subjects: + - kind: ServiceAccount + name: flyte-contour-contour-certgen +--- +# Source: flyte/charts/contour/templates/certgen/job.yaml +apiVersion: batch/v1 +kind: Job +metadata: + name: flyte-contour-contour-certgen + namespace: default + annotations: + "helm.sh/hook": "pre-install,pre-upgrade" + "helm.sh/hook-weight": "1" + "helm.sh/hook-delete-policy": before-hook-creation,hook-succeeded + labels: + app.kubernetes.io/name: contour + helm.sh/chart: contour-4.1.2 + app.kubernetes.io/instance: flyte + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/component: contour-certgen +spec: + ttlSecondsAfterFinished: 0 + template: + metadata: + labels: + app.kubernetes.io/name: contour + helm.sh/chart: contour-4.1.2 + app.kubernetes.io/instance: flyte + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/component: contour-certgen + spec: + + containers: + - name: contour + image: docker.io/bitnami/contour:1.12.0-debian-10-r0 + imagePullPolicy: IfNotPresent + command: + - contour + args: + - certgen + - --kube + - --incluster + - --overwrite + - --secrets-format=compact + - --namespace=$(CONTOUR_NAMESPACE) + env: + - name: CONTOUR_NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace + resources: + limits: + cpu: 100m + memory: 100Mi + requests: + cpu: 10m + memory: 50Mi + restartPolicy: Never + serviceAccountName: flyte-contour-contour-certgen + securityContext: + runAsUser: 1001 + runAsGroup: 1001 + fsGroup: + runAsNonRoot: true + parallelism: 1 + completions: 1 + backoffLimit: 1 diff --git a/deployment/sandbox/flyte_helm_generated.yaml b/deployment/sandbox/flyte_helm_generated.yaml new file mode 100644 index 0000000000..01a082e2cc --- /dev/null +++ b/deployment/sandbox/flyte_helm_generated.yaml @@ -0,0 +1,4200 @@ +--- +# Source: flyte/charts/contour/templates/contour/serviceaccount.yaml +apiVersion: v1 +kind: ServiceAccount +metadata: + name: flyte-contour-contour + namespace: default + labels: + app.kubernetes.io/name: contour + helm.sh/chart: contour-4.1.2 + app.kubernetes.io/instance: flyte + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/component: contour +--- +# Source: flyte/charts/contour/templates/envoy/serviceaccount.yaml +apiVersion: v1 +kind: ServiceAccount +metadata: + name: flyte-contour-envoy + namespace: default + labels: + app.kubernetes.io/name: contour + helm.sh/chart: contour-4.1.2 + app.kubernetes.io/instance: flyte + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/component: envoy +--- +# Source: flyte/charts/kubernetes-dashboard/templates/serviceaccount.yaml +# Copyright 2017 The Kubernetes Authors. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +apiVersion: v1 +kind: ServiceAccount +metadata: + labels: + app.kubernetes.io/name: kubernetes-dashboard + helm.sh/chart: kubernetes-dashboard-4.0.2 + app.kubernetes.io/instance: flyte + app.kubernetes.io/version: "2.2.0" + app.kubernetes.io/managed-by: Helm + name: flyte-kubernetes-dashboard +--- +# Source: flyte/charts/sparkoperator/templates/serviceaccount.yaml +apiVersion: v1 +kind: ServiceAccount +metadata: + name: flyte-sparkoperator + labels: + helm.sh/chart: sparkoperator-1.0.6 + app.kubernetes.io/name: sparkoperator + app.kubernetes.io/instance: flyte + app.kubernetes.io/version: "v1beta2-1.2.0-3.0.0" + app.kubernetes.io/managed-by: Helm +--- +# Source: flyte/charts/sparkoperator/templates/spark-serviceaccount.yaml +apiVersion: v1 +kind: ServiceAccount +metadata: + name: flyte-spark + namespace: default + labels: + helm.sh/chart: sparkoperator-1.0.6 + app.kubernetes.io/name: sparkoperator + app.kubernetes.io/instance: flyte + app.kubernetes.io/version: "v1beta2-1.2.0-3.0.0" + app.kubernetes.io/managed-by: Helm +--- +# Source: flyte/templates/admin/rbac.yaml +apiVersion: v1 +kind: ServiceAccount +metadata: + name: flyteadmin + namespace: default + labels: + app.kubernetes.io/name: flyteadmin + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm +--- +# Source: flyte/templates/datacatalog/rbac.yaml +apiVersion: v1 +kind: ServiceAccount +metadata: + name: datacatalog + namespace: default + labels: + app.kubernetes.io/name: datacatalog + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm +--- +# Source: flyte/templates/propeller/rbac.yaml +apiVersion: v1 +kind: ServiceAccount +metadata: + name: flytepropeller + namespace: default + labels: + app.kubernetes.io/name: flytepropeller + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm +--- +# Source: flyte/templates/propeller/webhook.yaml +# Create a Service Account for webhook +apiVersion: v1 +kind: ServiceAccount +metadata: + name: flyte-pod-webhook + namespace: default +--- +# Source: flyte/charts/kubernetes-dashboard/templates/secret.yaml +# Copyright 2017 The Kubernetes Authors. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# kubernetes-dashboard-certs +apiVersion: v1 +kind: Secret +metadata: + labels: + app.kubernetes.io/name: kubernetes-dashboard + helm.sh/chart: kubernetes-dashboard-4.0.2 + app.kubernetes.io/instance: flyte + app.kubernetes.io/version: "2.2.0" + app.kubernetes.io/managed-by: Helm + name: flyte-kubernetes-dashboard-certs +type: Opaque +--- +# Source: flyte/charts/kubernetes-dashboard/templates/secret.yaml +# kubernetes-dashboard-csrf +apiVersion: v1 +kind: Secret +metadata: + labels: + app.kubernetes.io/name: kubernetes-dashboard + helm.sh/chart: kubernetes-dashboard-4.0.2 + app.kubernetes.io/instance: flyte + app.kubernetes.io/version: "2.2.0" + app.kubernetes.io/managed-by: Helm + name: kubernetes-dashboard-csrf +type: Opaque +--- +# Source: flyte/charts/kubernetes-dashboard/templates/secret.yaml +# kubernetes-dashboard-key-holder +apiVersion: v1 +kind: Secret +metadata: + labels: + app.kubernetes.io/name: kubernetes-dashboard + helm.sh/chart: kubernetes-dashboard-4.0.2 + app.kubernetes.io/instance: flyte + app.kubernetes.io/version: "2.2.0" + app.kubernetes.io/managed-by: Helm + name: kubernetes-dashboard-key-holder +type: Opaque +--- +# Source: flyte/templates/admin/secret-auth.yaml +apiVersion: v1 +kind: Secret +metadata: + name: flyte-admin-auth + namespace: default +type: Opaque +--- +# Source: flyte/templates/propeller/secret-auth.yaml +apiVersion: v1 +kind: Secret +metadata: + name: flyte-propeller-auth + namespace: default +type: Opaque +stringData: + client_secret: foobar +--- +# Source: flyte/templates/propeller/webhook.yaml +# Create an empty secret that the first propeller pod will populate +apiVersion: v1 +kind: Secret +metadata: + name: flyte-pod-webhook + namespace: default +type: Opaque +--- +# Source: flyte/charts/contour/templates/contour/configmap.yaml +apiVersion: v1 +kind: ConfigMap +metadata: + name: flyte-contour + namespace: default + labels: + app.kubernetes.io/name: contour + helm.sh/chart: contour-4.1.2 + app.kubernetes.io/instance: flyte + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/component: contour +data: + contour.yaml: | + accesslog-format: envoy + disablePermitInsecure: false + envoy-service-name: 'flyte-contour-envoy' + leaderelection: + configmap-namespace: 'default' + tls: {} +--- +# Source: flyte/charts/kubernetes-dashboard/templates/configmap.yaml +# Copyright 2017 The Kubernetes Authors. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +apiVersion: v1 +kind: ConfigMap +metadata: + labels: + app.kubernetes.io/name: kubernetes-dashboard + helm.sh/chart: kubernetes-dashboard-4.0.2 + app.kubernetes.io/instance: flyte + app.kubernetes.io/version: "2.2.0" + app.kubernetes.io/managed-by: Helm + name: kubernetes-dashboard-settings +data: +--- +# Source: flyte/templates/admin/cluster_resource_configmap.yaml +apiVersion: v1 +kind: ConfigMap +metadata: + name: clusterresource-template + namespace: default + labels: + app.kubernetes.io/name: flyteadmin + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm +data: + aa_namespace.yaml: | + apiVersion: v1 + kind: Namespace + metadata: + name: {{ namespace }} + spec: + finalizers: + - kubernetes + + ab_project_resource_quota.yaml: | + apiVersion: v1 + kind: ResourceQuota + metadata: + name: project-quota + namespace: {{ namespace }} + spec: + hard: + limits.cpu: {{ projectQuotaCpu }} + limits.memory: {{ projectQuotaMemory }} + + zz_copilot_config.yaml: | + kind: ConfigMap + apiVersion: v1 + metadata: + name: flyte-data-config + namespace: {{ namespace }} + data: + config.yaml: | + storage: + type: minio + container: "my-s3-bucket" + connection: + access-key: minio + auth-type: accesskey + secret-key: miniostorage + disable-ssl: true + endpoint: http://minio.flyte.svc.cluster.local:9000 + region: us-east-1 + enable-multicontainer: true +--- +# Source: flyte/templates/admin/configmap.yaml +apiVersion: v1 +kind: ConfigMap +metadata: + name: flyte-admin-config + namespace: default + labels: + app.kubernetes.io/name: flyteadmin + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm +data: + db.yaml: | + database: + dbname: flyte_development + host: postgres + port: 5432 + username: postgres + domain.yaml: | + domains: + - id: development + name: development + - id: staging + name: staging + - id: production + name: production + logger.yaml: | + logger: + level: 4 + show-source: true + server.yaml: | + auth: + appAuth: + thirdPartyConfig: + flyteClient: + clientId: flytectl + redirectUri: https://localhost:53593/callback + scopes: + - offline + - all + authorizedUris: + - https://localhost:30081 + - http://flyteadmin:80 + - http://flyteadmin.flyte.svc.cluster.local:80 + userAuth: + openId: + baseUrl: https://accounts.google.com + clientId: 657465813211-6eog7ek7li5k7i7fvgv2921075063hpe.apps.googleusercontent.com + scopes: + - profile + - openid + flyteadmin: + eventVersion: 1 + metadataStoragePrefix: + - metadata + - admin + metricsScope: 'flyte:' + profilerPort: 10254 + roleNameKey: iam.amazonaws.com/role + testing: + host: http://flyteadmin + server: + grpcPort: 8089 + httpPort: 8088 + security: + allowCors: true + allowedHeaders: + - Content-Type + - flyte-authorization + allowedOrigins: + - '*' + secure: false + useAuth: false + remoteData.yaml: | + remoteData: + region: us-east-1 + scheme: local + signedUrls: + durationMinutes: 3 + storage.yaml: | + storage: + type: minio + container: "my-s3-bucket" + connection: + access-key: minio + auth-type: accesskey + secret-key: miniostorage + disable-ssl: true + endpoint: http://minio.flyte.svc.cluster.local:9000 + region: us-east-1 + limits: + maxDownloadMBs: 10 + task_resource_defaults.yaml: | + task_resources: + defaults: + cpu: 100m + memory: 200Mi + storage: 5Mi + limits: + cpu: 2 + gpu: 1 + memory: 8Gi + storage: 20Mi + cluster_resources.yaml: | + cluster_resources: + customData: + - production: + - projectQuotaCpu: + value: "5" + - projectQuotaMemory: + value: 4000Mi + - staging: + - projectQuotaCpu: + value: "2" + - projectQuotaMemory: + value: 3000Mi + - development: + - projectQuotaCpu: + value: "4" + - projectQuotaMemory: + value: 3000Mi + refresh: 5m + templatePath: /etc/flyte/clusterresource/templates +--- +# Source: flyte/templates/console/configmap.yaml +apiVersion: v1 +kind: ConfigMap +metadata: + name: flyte-console-config + namespace: default + labels: + app.kubernetes.io/name: flyteconsole + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm +data: + BASE_URL: /console + CONFIG_DIR: /etc/flyte/config + DISABLE_AUTH: "1" +--- +# Source: flyte/templates/datacatalog/configmap.yaml +apiVersion: v1 +kind: ConfigMap +metadata: + name: datacatalog-config + namespace: default + labels: + app.kubernetes.io/name: datacatalog + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm +data: + db.yaml: | + database: + dbname: flyte_development + host: postgres + port: 5432 + username: postgres + logger.yaml: | + logger: + level: 4 + show-source: true + server.yaml: | + application: + grpcPort: 8089 + grpcServerReflection: true + httpPort: 8080 + datacatalog: + metrics-scope: datacatalog + profiler-port: 10254 + storage-prefix: metadata/datacatalog + storage.yaml: | + storage: + type: minio + container: "my-s3-bucket" + connection: + access-key: minio + auth-type: accesskey + secret-key: miniostorage + disable-ssl: true + endpoint: http://minio.flyte.svc.cluster.local:9000 + region: us-east-1 + limits: + maxDownloadMBs: 10 +--- +# Source: flyte/templates/propeller/configmap.yaml +apiVersion: v1 +kind: ConfigMap +metadata: + name: flyte-propeller-config + namespace: default + labels: + app.kubernetes.io/name: flyteadmin + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm +data: + admin.yaml: | + admin: + clientId: flytepropeller + clientSecretLocation: /etc/secrets/client_secret + endpoint: flyteadmin:81 + insecure: true + event: + capacity: 1000 + rate: 500 + type: admin + catalog.yaml: | + catalog-cache: + endpoint: datacatalog:89 + insecure: true + type: datacatalog + copilot.yaml: | + plugins: + k8s: + co-pilot: + image: ghcr.io/lyft/flyteplugins/flytecopilot:dc4bdbd61cac88a39a5ff43e40f026bdbc2c78a2 + name: flyte-copilot- + start-timeout: 30s + core.yaml: | + propeller: + downstream-eval-duration: 30s + enable-admin-launcher: true + leader-election: + enabled: true + lease-duration: 15s + lock-config-map: + name: propeller-leader + namespace: flyte + renew-deadline: 10s + retry-period: 2s + limit-namespace: all + max-workflow-retries: 30 + metadata-prefix: metadata/propeller + metrics-prefix: flyte + prof-port: 10254 + queue: + batch-size: -1 + batching-interval: 2s + queue: + base-delay: 5s + capacity: 1000 + max-delay: 120s + rate: 100 + type: maxof + sub-queue: + capacity: 100 + rate: 10 + type: bucket + type: batch + rawoutput-prefix: s3://my-s3-bucket/ + workers: 20 + workflow-reeval-duration: 30s + webhook: + certDir: /etc/webhook/certs + serviceName: flyte-pod-webhook + enabled_plugins.yaml: | + tasks: + task-plugins: + default-for-task-types: + container: container + container_array: k8s-array + sidecar: sidecar + enabled-plugins: + - container + - sidecar + - k8s-array + k8s.yaml: | + plugins: + k8s: + default-cpus: 100m + default-env-vars: + - FLYTE_AWS_ENDPOINT: http://minio.flyte:9000 + - FLYTE_AWS_ACCESS_KEY_ID: minio + - FLYTE_AWS_SECRET_ACCESS_KEY: miniostorage + default-memory: 200Mi + logger.yaml: | + logger: + level: 4 + show-source: true + resource_manager.yaml: | + propeller: + resourcemanager: + resourceMaxQuota: 10000 + type: noop + storage.yaml: | + storage: + type: minio + container: "my-s3-bucket" + connection: + access-key: minio + auth-type: accesskey + secret-key: miniostorage + disable-ssl: true + endpoint: http://minio.flyte.svc.cluster.local:9000 + region: us-east-1 + limits: + maxDownloadMBs: 10 + cache.yaml: | + cache: + max_size_mbs: 0 + target_gc_percent: 70 + task_logs.yaml: | + plugins: + logs: + cloudwatch-enabled: false + kubernetes-enabled: true + kubernetes-template-uri: http://localhost:30082/#/log/{{ .namespace }}/{{ .podName }}/pod?namespace={{ .namespace }} +--- +# Source: flyte/charts/contour/templates/00-crds.yaml +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.2.9 + creationTimestamp: null + name: extensionservices.projectcontour.io +spec: + preserveUnknownFields: false + group: projectcontour.io + names: + kind: ExtensionService + listKind: ExtensionServiceList + plural: extensionservices + shortNames: + - extensionservice + - extensionservices + singular: extensionservice + scope: Namespaced + versions: + - name: v1alpha1 + schema: + openAPIV3Schema: + description: ExtensionService is the schema for the Contour extension services API. An ExtensionService resource binds a network service to the Contour API so that Contour API features can be implemented by collaborating components. + properties: + apiVersion: + description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' + type: string + kind: + description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + type: string + metadata: + type: object + spec: + description: ExtensionServiceSpec defines the desired state of an ExtensionService resource. + properties: + loadBalancerPolicy: + description: The policy for load balancing GRPC service requests. Note that the `Cookie` load balancing strategy cannot be used here. + properties: + strategy: + description: Strategy specifies the policy used to balance requests across the pool of backend pods. Valid policy names are `Random`, `RoundRobin`, `WeightedLeastRequest`, `Random` and `Cookie`. If an unknown strategy name is specified or no policy is supplied, the default `RoundRobin` policy is used. + type: string + type: object + protocol: + description: Protocol may be used to specify (or override) the protocol used to reach this Service. Values may be tls, h2, h2c. If omitted, protocol-selection falls back on Service annotations. + enum: + - h2 + - h2c + type: string + protocolVersion: + description: This field sets the version of the GRPC protocol that Envoy uses to send requests to the extension service. Since Contour always uses the v3 Envoy API, this is currently fixed at "v3". However, other protocol options will be available in future. + enum: + - v3 + type: string + services: + description: Services specifies the set of Kubernetes Service resources that receive GRPC extension API requests. If no weights are specified for any of the entries in this array, traffic will be spread evenly across all the services. Otherwise, traffic is balanced proportionally to the Weight field in each entry. + items: + description: ExtensionServiceTarget defines an Kubernetes Service to target with extension service traffic. + properties: + name: + description: Name is the name of Kubernetes service that will accept service traffic. + type: string + port: + description: Port (defined as Integer) to proxy traffic to since a service can have multiple defined. + exclusiveMaximum: true + maximum: 65536 + minimum: 1 + type: integer + weight: + description: Weight defines proportion of traffic to balance to the Kubernetes Service. + format: int32 + type: integer + required: + - name + - port + type: object + minItems: 1 + type: array + timeoutPolicy: + description: The timeout policy for requests to the services. + properties: + idle: + description: Timeout after which, if there are no active requests for this route, the connection between Envoy and the backend or Envoy and the external client will be closed. If not specified, there is no per-route idle timeout, though a connection manager-wide stream_idle_timeout default of 5m still applies. + pattern: ^(((\d*(\.\d*)?h)|(\d*(\.\d*)?m)|(\d*(\.\d*)?s)|(\d*(\.\d*)?ms)|(\d*(\.\d*)?us)|(\d*(\.\d*)?µs)|(\d*(\.\d*)?ns))+|infinity|infinite)$ + type: string + response: + description: Timeout for receiving a response from the server after processing a request from client. If not supplied, Envoy's default value of 15s applies. + pattern: ^(((\d*(\.\d*)?h)|(\d*(\.\d*)?m)|(\d*(\.\d*)?s)|(\d*(\.\d*)?ms)|(\d*(\.\d*)?us)|(\d*(\.\d*)?µs)|(\d*(\.\d*)?ns))+|infinity|infinite)$ + type: string + type: object + validation: + description: UpstreamValidation defines how to verify the backend service's certificate + properties: + caSecret: + description: Name of the Kubernetes secret be used to validate the certificate presented by the backend + type: string + subjectName: + description: Key which is expected to be present in the 'subjectAltName' of the presented certificate + type: string + required: + - caSecret + - subjectName + type: object + required: + - services + type: object + status: + description: ExtensionServiceStatus defines the observed state of an ExtensionService resource. + properties: + conditions: + description: "Conditions contains the current status of the ExtensionService resource. \n Contour will update a single condition, `Valid`, that is in normal-true polarity. \n Contour will not modify any other Conditions set in this block, in case some other controller wants to add a Condition." + items: + description: "DetailedCondition is an extension of the normal Kubernetes conditions, with two extra fields to hold sub-conditions, which provide more detailed reasons for the state (True or False) of the condition. \n `errors` holds information about sub-conditions which are fatal to that condition and render its state False. \n `warnings` holds information about sub-conditions which are not fatal to that condition and do not force the state to be False. \n Remember that Conditions have a type, a status, and a reason. \n The type is the type of the condition, the most important one in this CRD set is `Valid`. `Valid` is a positive-polarity condition: when it is `status: true` there are no problems. \n In more detail, `status: true` means that the object is has been ingested into Contour with no errors. `warnings` may still be present, and will be indicated in the Reason field. There must be zero entries in the `errors` slice in this case. \n `Valid`, `status: false` means that the object has had one or more fatal errors during processing into Contour. The details of the errors will be present under the `errors` field. There must be at least one error in the `errors` slice if `status` is `false`. \n For DetailedConditions of types other than `Valid`, the Condition must be in the negative polarity. When they have `status` `true`, there is an error. There must be at least one entry in the `errors` Subcondition slice. When they have `status` `false`, there are no serious errors, and there must be zero entries in the `errors` slice. In either case, there may be entries in the `warnings` slice. \n Regardless of the polarity, the `reason` and `message` fields must be updated with either the detail of the reason (if there is one and only one entry in total across both the `errors` and `warnings` slices), or `MultipleReasons` if there is more than one entry." + properties: + errors: + description: "Errors contains a slice of relevant error subconditions for this object. \n Subconditions are expected to appear when relevant (when there is a error), and disappear when not relevant. An empty slice here indicates no errors." + items: + description: "SubCondition is a Condition-like type intended for use as a subcondition inside a DetailedCondition. \n It contains a subset of the Condition fields. \n It is intended for warnings and errors, so `type` names should use abnormal-true polarity, that is, they should be of the form \"ErrorPresent: true\". \n The expected lifecycle for these errors is that they should only be present when the error or warning is, and should be removed when they are not relevant." + properties: + message: + description: "Message is a human readable message indicating details about the transition. \n This may be an empty string." + maxLength: 32768 + type: string + reason: + description: "Reason contains a programmatic identifier indicating the reason for the condition's last transition. Producers of specific condition types may define expected values and meanings for this field, and whether the values are considered a guaranteed API. \n The value should be a CamelCase string. \n This field may not be empty." + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + description: Status of the condition, one of True, False, Unknown. + enum: + - "True" + - "False" + - Unknown + type: string + type: + description: "Type of condition in `CamelCase` or in `foo.example.com/CamelCase`. \n This must be in abnormal-true polarity, that is, `ErrorFound` or `controller.io/ErrorFound`. \n The regex it matches is (dns1123SubdomainFmt/)?(qualifiedNameFmt)" + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + required: + - message + - reason + - status + - type + type: object + type: array + lastTransitionTime: + description: lastTransitionTime is the last time the condition transitioned from one status to another. This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable. + format: date-time + type: string + message: + description: message is a human readable message indicating details about the transition. This may be an empty string. + maxLength: 32768 + type: string + observedGeneration: + description: observedGeneration represents the .metadata.generation that the condition was set based upon. For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date with respect to the current state of the instance. + format: int64 + minimum: 0 + type: integer + reason: + description: reason contains a programmatic identifier indicating the reason for the condition's last transition. Producers of specific condition types may define expected values and meanings for this field, and whether the values are considered a guaranteed API. The value should be a CamelCase string. This field may not be empty. + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + description: status of the condition, one of True, False, Unknown. + enum: + - "True" + - "False" + - Unknown + type: string + type: + description: type of condition in CamelCase or in foo.example.com/CamelCase. --- Many .condition.type values are consistent across resources like Available, but because arbitrary conditions can be useful (see .node.status.conditions), the ability to deconflict is important. The regex it matches is (dns1123SubdomainFmt/)?(qualifiedNameFmt) + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + warnings: + description: "Warnings contains a slice of relevant warning subconditions for this object. \n Subconditions are expected to appear when relevant (when there is a warning), and disappear when not relevant. An empty slice here indicates no warnings." + items: + description: "SubCondition is a Condition-like type intended for use as a subcondition inside a DetailedCondition. \n It contains a subset of the Condition fields. \n It is intended for warnings and errors, so `type` names should use abnormal-true polarity, that is, they should be of the form \"ErrorPresent: true\". \n The expected lifecycle for these errors is that they should only be present when the error or warning is, and should be removed when they are not relevant." + properties: + message: + description: "Message is a human readable message indicating details about the transition. \n This may be an empty string." + maxLength: 32768 + type: string + reason: + description: "Reason contains a programmatic identifier indicating the reason for the condition's last transition. Producers of specific condition types may define expected values and meanings for this field, and whether the values are considered a guaranteed API. \n The value should be a CamelCase string. \n This field may not be empty." + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + description: Status of the condition, one of True, False, Unknown. + enum: + - "True" + - "False" + - Unknown + type: string + type: + description: "Type of condition in `CamelCase` or in `foo.example.com/CamelCase`. \n This must be in abnormal-true polarity, that is, `ErrorFound` or `controller.io/ErrorFound`. \n The regex it matches is (dns1123SubdomainFmt/)?(qualifiedNameFmt)" + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + required: + - message + - reason + - status + - type + type: object + type: array + required: + - lastTransitionTime + - message + - reason + - status + - type + type: object + type: array + x-kubernetes-list-map-keys: + - type + x-kubernetes-list-type: map + type: object + type: object + served: true + storage: true + subresources: + status: {} +status: + acceptedNames: + kind: "" + plural: "" + conditions: [] + storedVersions: [] +--- +# Source: flyte/charts/contour/templates/00-crds.yaml +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.2.9 + creationTimestamp: null + name: httpproxies.projectcontour.io +spec: + preserveUnknownFields: false + group: projectcontour.io + names: + kind: HTTPProxy + listKind: HTTPProxyList + plural: httpproxies + shortNames: + - proxy + - proxies + singular: httpproxy + scope: Namespaced + versions: + - additionalPrinterColumns: + - description: Fully qualified domain name + jsonPath: .spec.virtualhost.fqdn + name: FQDN + type: string + - description: Secret with TLS credentials + jsonPath: .spec.virtualhost.tls.secretName + name: TLS Secret + type: string + - description: The current status of the HTTPProxy + jsonPath: .status.currentStatus + name: Status + type: string + - description: Description of the current status + jsonPath: .status.description + name: Status Description + type: string + name: v1 + schema: + openAPIV3Schema: + description: HTTPProxy is an Ingress CRD specification. + properties: + apiVersion: + description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' + type: string + kind: + description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + type: string + metadata: + type: object + spec: + description: HTTPProxySpec defines the spec of the CRD. + properties: + ggvar: + description: Testing new param for CRD + type: string + includes: + description: Includes allow for specific routing configuration to be included from another HTTPProxy, possibly in another namespace. + items: + description: Include describes a set of policies that can be applied to an HTTPProxy in a namespace. + properties: + conditions: + description: 'Conditions are a set of rules that are applied to included HTTPProxies. In effect, they are added onto the Conditions of included HTTPProxy Route structs. When applied, they are merged using AND, with one exception: There can be only one Prefix MatchCondition per Conditions slice. More than one Prefix, or contradictory Conditions, will make the include invalid.' + items: + description: MatchCondition are a general holder for matching rules for HTTPProxies. One of Prefix or Header must be provided. + properties: + header: + description: Header specifies the header condition to match. + properties: + contains: + description: Contains specifies a substring that must be present in the header value. + type: string + exact: + description: Exact specifies a string that the header value must be equal to. + type: string + name: + description: Name is the name of the header to match against. Name is required. Header names are case insensitive. + type: string + notcontains: + description: NotContains specifies a substring that must not be present in the header value. + type: string + notexact: + description: NoExact specifies a string that the header value must not be equal to. The condition is true if the header has any other value. + type: string + present: + description: Present specifies that condition is true when the named header is present, regardless of its value. Note that setting Present to false does not make the condition true if the named header is absent. + type: boolean + required: + - name + type: object + prefix: + description: Prefix defines a prefix match for a request. + type: string + type: object + type: array + name: + description: Name of the HTTPProxy + type: string + namespace: + description: Namespace of the HTTPProxy to include. Defaults to the current namespace if not supplied. + type: string + required: + - name + type: object + type: array + routes: + description: Routes are the ingress routes. If TCPProxy is present, Routes is ignored. + items: + description: Route contains the set of routes for a virtual host. + properties: + authPolicy: + description: AuthPolicy updates the authorization policy that was set on the root HTTPProxy object for client requests that match this route. + properties: + context: + additionalProperties: + type: string + description: Context is a set of key/value pairs that are sent to the authentication server in the check request. If a context is provided at an enclosing scope, the entries are merged such that the inner scope overrides matching keys from the outer scope. + type: object + disabled: + description: When true, this field disables client request authentication for the scope of the policy. + type: boolean + type: object + conditions: + description: 'Conditions are a set of rules that are applied to a Route. When applied, they are merged using AND, with one exception: There can be only one Prefix MatchCondition per Conditions slice. More than one Prefix, or contradictory Conditions, will make the route invalid.' + items: + description: MatchCondition are a general holder for matching rules for HTTPProxies. One of Prefix or Header must be provided. + properties: + header: + description: Header specifies the header condition to match. + properties: + contains: + description: Contains specifies a substring that must be present in the header value. + type: string + exact: + description: Exact specifies a string that the header value must be equal to. + type: string + name: + description: Name is the name of the header to match against. Name is required. Header names are case insensitive. + type: string + notcontains: + description: NotContains specifies a substring that must not be present in the header value. + type: string + notexact: + description: NoExact specifies a string that the header value must not be equal to. The condition is true if the header has any other value. + type: string + present: + description: Present specifies that condition is true when the named header is present, regardless of its value. Note that setting Present to false does not make the condition true if the named header is absent. + type: boolean + required: + - name + type: object + prefix: + description: Prefix defines a prefix match for a request. + type: string + type: object + type: array + enableWebsockets: + description: Enables websocket support for the route. + type: boolean + healthCheckPolicy: + description: The health check policy for this route. + properties: + healthyThresholdCount: + description: The number of healthy health checks required before a host is marked healthy + format: int64 + minimum: 0 + type: integer + host: + description: The value of the host header in the HTTP health check request. If left empty (default value), the name "contour-envoy-healthcheck" will be used. + type: string + intervalSeconds: + description: The interval (seconds) between health checks + format: int64 + type: integer + path: + description: HTTP endpoint used to perform health checks on upstream service + type: string + timeoutSeconds: + description: The time to wait (seconds) for a health check response + format: int64 + type: integer + unhealthyThresholdCount: + description: The number of unhealthy health checks required before a host is marked unhealthy + format: int64 + minimum: 0 + type: integer + required: + - path + type: object + loadBalancerPolicy: + description: The load balancing policy for this route. + properties: + strategy: + description: Strategy specifies the policy used to balance requests across the pool of backend pods. Valid policy names are `Random`, `RoundRobin`, `WeightedLeastRequest`, `Random` and `Cookie`. If an unknown strategy name is specified or no policy is supplied, the default `RoundRobin` policy is used. + type: string + type: object + pathRewritePolicy: + description: The policy for rewriting the path of the request URL after the request has been routed to a Service. + properties: + replacePrefix: + description: ReplacePrefix describes how the path prefix should be replaced. + items: + description: ReplacePrefix describes a path prefix replacement. + properties: + prefix: + description: "Prefix specifies the URL path prefix to be replaced. \n If Prefix is specified, it must exactly match the MatchCondition prefix that is rendered by the chain of including HTTPProxies and only that path prefix will be replaced by Replacement. This allows HTTPProxies that are included through multiple roots to only replace specific path prefixes, leaving others unmodified. \n If Prefix is not specified, all routing prefixes rendered by the include chain will be replaced." + minLength: 1 + type: string + replacement: + description: Replacement is the string that the routing path prefix will be replaced with. This must not be empty. + minLength: 1 + type: string + required: + - replacement + type: object + type: array + type: object + permitInsecure: + description: Allow this path to respond to insecure requests over HTTP which are normally not permitted when a `virtualhost.tls` block is present. + type: boolean + requestHeadersPolicy: + description: The policy for managing request headers during proxying. + properties: + remove: + description: Remove specifies a list of HTTP header names to remove. + items: + type: string + type: array + set: + description: Set specifies a list of HTTP header values that will be set in the HTTP header. If the header does not exist it will be added, otherwise it will be overwritten with the new value. + items: + description: HeaderValue represents a header name/value pair + properties: + name: + description: Name represents a key of a header + minLength: 1 + type: string + value: + description: Value represents the value of a header specified by a key + minLength: 1 + type: string + required: + - name + - value + type: object + type: array + type: object + responseHeadersPolicy: + description: The policy for managing response headers during proxying. Rewriting the 'Host' header is not supported. + properties: + remove: + description: Remove specifies a list of HTTP header names to remove. + items: + type: string + type: array + set: + description: Set specifies a list of HTTP header values that will be set in the HTTP header. If the header does not exist it will be added, otherwise it will be overwritten with the new value. + items: + description: HeaderValue represents a header name/value pair + properties: + name: + description: Name represents a key of a header + minLength: 1 + type: string + value: + description: Value represents the value of a header specified by a key + minLength: 1 + type: string + required: + - name + - value + type: object + type: array + type: object + retryPolicy: + description: The retry policy for this route. + properties: + count: + description: NumRetries is maximum allowed number of retries. If not supplied, the number of retries is one. + format: int64 + minimum: 0 + type: integer + perTryTimeout: + description: PerTryTimeout specifies the timeout per retry attempt. Ignored if NumRetries is not supplied. + pattern: ^(((\d*(\.\d*)?h)|(\d*(\.\d*)?m)|(\d*(\.\d*)?s)|(\d*(\.\d*)?ms)|(\d*(\.\d*)?us)|(\d*(\.\d*)?µs)|(\d*(\.\d*)?ns))+|infinity|infinite)$ + type: string + retriableStatusCodes: + description: "RetriableStatusCodes specifies the HTTP status codes that should be retried. \n This field is only respected when you include `retriable-status-codes` in the `RetryOn` field." + items: + format: int32 + type: integer + type: array + retryOn: + description: "RetryOn specifies the conditions on which to retry a request. \n Supported [HTTP conditions](https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_filters/router_filter#x-envoy-retry-on): \n - `5xx` - `gateway-error` - `reset` - `connect-failure` - `retriable-4xx` - `refused-stream` - `retriable-status-codes` - `retriable-headers` \n Supported [gRPC conditions](https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_filters/router_filter#x-envoy-retry-grpc-on): \n - `cancelled` - `deadline-exceeded` - `internal` - `resource-exhausted` - `unavailable`" + items: + description: RetryOn is a string type alias with validation to ensure that the value is valid. + enum: + - 5xx + - gateway-error + - reset + - connect-failure + - retriable-4xx + - refused-stream + - retriable-status-codes + - retriable-headers + - cancelled + - deadline-exceeded + - internal + - resource-exhausted + - unavailable + type: string + type: array + type: object + services: + description: Services are the services to proxy traffic. + items: + description: Service defines an Kubernetes Service to proxy traffic. + properties: + mirror: + description: If Mirror is true the Service will receive a read only mirror of the traffic for this route. + type: boolean + name: + description: Name is the name of Kubernetes service to proxy traffic. Names defined here will be used to look up corresponding endpoints which contain the ips to route. + type: string + port: + description: Port (defined as Integer) to proxy traffic to since a service can have multiple defined. + exclusiveMaximum: true + maximum: 65536 + minimum: 1 + type: integer + protocol: + description: Protocol may be used to specify (or override) the protocol used to reach this Service. Values may be tls, h2, h2c. If omitted, protocol-selection falls back on Service annotations. + enum: + - h2 + - h2c + - tls + type: string + requestHeadersPolicy: + description: The policy for managing request headers during proxying. Rewriting the 'Host' header is not supported. + properties: + remove: + description: Remove specifies a list of HTTP header names to remove. + items: + type: string + type: array + set: + description: Set specifies a list of HTTP header values that will be set in the HTTP header. If the header does not exist it will be added, otherwise it will be overwritten with the new value. + items: + description: HeaderValue represents a header name/value pair + properties: + name: + description: Name represents a key of a header + minLength: 1 + type: string + value: + description: Value represents the value of a header specified by a key + minLength: 1 + type: string + required: + - name + - value + type: object + type: array + type: object + responseHeadersPolicy: + description: The policy for managing response headers during proxying. Rewriting the 'Host' header is not supported. + properties: + remove: + description: Remove specifies a list of HTTP header names to remove. + items: + type: string + type: array + set: + description: Set specifies a list of HTTP header values that will be set in the HTTP header. If the header does not exist it will be added, otherwise it will be overwritten with the new value. + items: + description: HeaderValue represents a header name/value pair + properties: + name: + description: Name represents a key of a header + minLength: 1 + type: string + value: + description: Value represents the value of a header specified by a key + minLength: 1 + type: string + required: + - name + - value + type: object + type: array + type: object + validation: + description: UpstreamValidation defines how to verify the backend service's certificate + properties: + caSecret: + description: Name of the Kubernetes secret be used to validate the certificate presented by the backend + type: string + subjectName: + description: Key which is expected to be present in the 'subjectAltName' of the presented certificate + type: string + required: + - caSecret + - subjectName + type: object + weight: + description: Weight defines percentage of traffic to balance traffic + format: int64 + minimum: 0 + type: integer + required: + - name + - port + type: object + minItems: 1 + type: array + timeoutPolicy: + description: The timeout policy for this route. + properties: + idle: + description: Timeout after which, if there are no active requests for this route, the connection between Envoy and the backend or Envoy and the external client will be closed. If not specified, there is no per-route idle timeout, though a connection manager-wide stream_idle_timeout default of 5m still applies. + pattern: ^(((\d*(\.\d*)?h)|(\d*(\.\d*)?m)|(\d*(\.\d*)?s)|(\d*(\.\d*)?ms)|(\d*(\.\d*)?us)|(\d*(\.\d*)?µs)|(\d*(\.\d*)?ns))+|infinity|infinite)$ + type: string + response: + description: Timeout for receiving a response from the server after processing a request from client. If not supplied, Envoy's default value of 15s applies. + pattern: ^(((\d*(\.\d*)?h)|(\d*(\.\d*)?m)|(\d*(\.\d*)?s)|(\d*(\.\d*)?ms)|(\d*(\.\d*)?us)|(\d*(\.\d*)?µs)|(\d*(\.\d*)?ns))+|infinity|infinite)$ + type: string + type: object + required: + - services + type: object + type: array + tcpproxy: + description: TCPProxy holds TCP proxy information. + properties: + healthCheckPolicy: + description: The health check policy for this tcp proxy + properties: + healthyThresholdCount: + description: The number of healthy health checks required before a host is marked healthy + format: int32 + type: integer + intervalSeconds: + description: The interval (seconds) between health checks + format: int64 + type: integer + timeoutSeconds: + description: The time to wait (seconds) for a health check response + format: int64 + type: integer + unhealthyThresholdCount: + description: The number of unhealthy health checks required before a host is marked unhealthy + format: int32 + type: integer + type: object + include: + description: Include specifies that this tcpproxy should be delegated to another HTTPProxy. + properties: + name: + description: Name of the child HTTPProxy + type: string + namespace: + description: Namespace of the HTTPProxy to include. Defaults to the current namespace if not supplied. + type: string + required: + - name + type: object + includes: + description: "IncludesDeprecated allow for specific routing configuration to be appended to another HTTPProxy in another namespace. \n Exists due to a mistake when developing HTTPProxy and the field was marked plural when it should have been singular. This field should stay to not break backwards compatibility to v1 users." + properties: + name: + description: Name of the child HTTPProxy + type: string + namespace: + description: Namespace of the HTTPProxy to include. Defaults to the current namespace if not supplied. + type: string + required: + - name + type: object + loadBalancerPolicy: + description: The load balancing policy for the backend services. + properties: + strategy: + description: Strategy specifies the policy used to balance requests across the pool of backend pods. Valid policy names are `Random`, `RoundRobin`, `WeightedLeastRequest`, `Random` and `Cookie`. If an unknown strategy name is specified or no policy is supplied, the default `RoundRobin` policy is used. + type: string + type: object + services: + description: Services are the services to proxy traffic + items: + description: Service defines an Kubernetes Service to proxy traffic. + properties: + mirror: + description: If Mirror is true the Service will receive a read only mirror of the traffic for this route. + type: boolean + name: + description: Name is the name of Kubernetes service to proxy traffic. Names defined here will be used to look up corresponding endpoints which contain the ips to route. + type: string + port: + description: Port (defined as Integer) to proxy traffic to since a service can have multiple defined. + exclusiveMaximum: true + maximum: 65536 + minimum: 1 + type: integer + protocol: + description: Protocol may be used to specify (or override) the protocol used to reach this Service. Values may be tls, h2, h2c. If omitted, protocol-selection falls back on Service annotations. + enum: + - h2 + - h2c + - tls + type: string + requestHeadersPolicy: + description: The policy for managing request headers during proxying. Rewriting the 'Host' header is not supported. + properties: + remove: + description: Remove specifies a list of HTTP header names to remove. + items: + type: string + type: array + set: + description: Set specifies a list of HTTP header values that will be set in the HTTP header. If the header does not exist it will be added, otherwise it will be overwritten with the new value. + items: + description: HeaderValue represents a header name/value pair + properties: + name: + description: Name represents a key of a header + minLength: 1 + type: string + value: + description: Value represents the value of a header specified by a key + minLength: 1 + type: string + required: + - name + - value + type: object + type: array + type: object + responseHeadersPolicy: + description: The policy for managing response headers during proxying. Rewriting the 'Host' header is not supported. + properties: + remove: + description: Remove specifies a list of HTTP header names to remove. + items: + type: string + type: array + set: + description: Set specifies a list of HTTP header values that will be set in the HTTP header. If the header does not exist it will be added, otherwise it will be overwritten with the new value. + items: + description: HeaderValue represents a header name/value pair + properties: + name: + description: Name represents a key of a header + minLength: 1 + type: string + value: + description: Value represents the value of a header specified by a key + minLength: 1 + type: string + required: + - name + - value + type: object + type: array + type: object + validation: + description: UpstreamValidation defines how to verify the backend service's certificate + properties: + caSecret: + description: Name of the Kubernetes secret be used to validate the certificate presented by the backend + type: string + subjectName: + description: Key which is expected to be present in the 'subjectAltName' of the presented certificate + type: string + required: + - caSecret + - subjectName + type: object + weight: + description: Weight defines percentage of traffic to balance traffic + format: int64 + minimum: 0 + type: integer + required: + - name + - port + type: object + type: array + type: object + virtualhost: + description: Virtualhost appears at most once. If it is present, the object is considered to be a "root" HTTPProxy. + properties: + authorization: + description: This field configures an extension service to perform authorization for this virtual host. Authorization can only be configured on virtual hosts that have TLS enabled. If the TLS configuration requires client certificate /validation, the client certificate is always included in the authentication check request. + properties: + authPolicy: + description: AuthPolicy sets a default authorization policy for client requests. This policy will be used unless overridden by individual routes. + properties: + context: + additionalProperties: + type: string + description: Context is a set of key/value pairs that are sent to the authentication server in the check request. If a context is provided at an enclosing scope, the entries are merged such that the inner scope overrides matching keys from the outer scope. + type: object + disabled: + description: When true, this field disables client request authentication for the scope of the policy. + type: boolean + type: object + extensionRef: + description: ExtensionServiceRef specifies the extension resource that will authorize client requests. + properties: + apiVersion: + description: API version of the referent. If this field is not specified, the default "projectcontour.io/v1alpha1" will be used + minLength: 1 + type: string + name: + description: "Name of the referent. \n More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names" + minLength: 1 + type: string + namespace: + description: "Namespace of the referent. If this field is not specifies, the namespace of the resource that targets the referent will be used. \n More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/" + minLength: 1 + type: string + type: object + failOpen: + description: If FailOpen is true, the client request is forwarded to the upstream service even if the authorization server fails to respond. This field should not be set in most cases. It is intended for use only while migrating applications from internal authorization to Contour external authorization. + type: boolean + responseTimeout: + description: ResponseTimeout configures maximum time to wait for a check response from the authorization server. Timeout durations are expressed in the Go [Duration format](https://godoc.org/time#ParseDuration). Valid time units are "ns", "us" (or "µs"), "ms", "s", "m", "h". The string "infinity" is also a valid input and specifies no timeout. + pattern: ^(((\d*(\.\d*)?h)|(\d*(\.\d*)?m)|(\d*(\.\d*)?s)|(\d*(\.\d*)?ms)|(\d*(\.\d*)?us)|(\d*(\.\d*)?µs)|(\d*(\.\d*)?ns))+|infinity|infinite)$ + type: string + required: + - extensionRef + type: object + corsPolicy: + description: Specifies the cross-origin policy to apply to the VirtualHost. + properties: + allowCredentials: + description: Specifies whether the resource allows credentials. + type: boolean + allowHeaders: + description: AllowHeaders specifies the content for the *access-control-allow-headers* header. + items: + description: CORSHeaderValue specifies the value of the string headers returned by a cross-domain request. + pattern: ^[a-zA-Z0-9!#$%&'*+.^_`|~-]+$ + type: string + type: array + allowMethods: + description: AllowMethods specifies the content for the *access-control-allow-methods* header. + items: + description: CORSHeaderValue specifies the value of the string headers returned by a cross-domain request. + pattern: ^[a-zA-Z0-9!#$%&'*+.^_`|~-]+$ + type: string + type: array + allowOrigin: + description: AllowOrigin specifies the origins that will be allowed to do CORS requests. "*" means allow any origin. + items: + type: string + type: array + exposeHeaders: + description: ExposeHeaders Specifies the content for the *access-control-expose-headers* header. + items: + description: CORSHeaderValue specifies the value of the string headers returned by a cross-domain request. + pattern: ^[a-zA-Z0-9!#$%&'*+.^_`|~-]+$ + type: string + type: array + maxAge: + description: MaxAge indicates for how long the results of a preflight request can be cached. MaxAge durations are expressed in the Go [Duration format](https://godoc.org/time#ParseDuration). Valid time units are "ns", "us" (or "µs"), "ms", "s", "m", "h". Only positive values are allowed while 0 disables the cache requiring a preflight OPTIONS check for all cross-origin requests. + type: string + required: + - allowMethods + - allowOrigin + type: object + fqdn: + description: The fully qualified domain name of the root of the ingress tree all leaves of the DAG rooted at this object relate to the fqdn. + type: string + tls: + description: If present the fields describes TLS properties of the virtual host. The SNI names that will be matched on are described in fqdn, the tls.secretName secret must contain a certificate that itself contains a name that matches the FQDN. + properties: + clientValidation: + description: "ClientValidation defines how to verify the client certificate when an external client establishes a TLS connection to Envoy. \n This setting: \n 1. Enables TLS client certificate validation. 2. Requires clients to present a TLS certificate (i.e. not optional validation). 3. Specifies how the client certificate will be validated." + properties: + caSecret: + description: Name of a Kubernetes secret that contains a CA certificate bundle. The client certificate must validate against the certificates in the bundle. + minLength: 1 + type: string + required: + - caSecret + type: object + enableFallbackCertificate: + description: EnableFallbackCertificate defines if the vhost should allow a default certificate to be applied which handles all requests which don't match the SNI defined in this vhost. + type: boolean + minimumProtocolVersion: + description: Minimum TLS version this vhost should negotiate + type: string + passthrough: + description: Passthrough defines whether the encrypted TLS handshake will be passed through to the backing cluster. Either Passthrough or SecretName must be specified, but not both. + type: boolean + secretName: + description: SecretName is the name of a TLS secret in the current namespace. Either SecretName or Passthrough must be specified, but not both. If specified, the named secret must contain a matching certificate for the virtual host's FQDN. + type: string + type: object + required: + - fqdn + type: object + type: object + status: + description: Status is a container for computed information about the HTTPProxy. + properties: + conditions: + description: "Conditions contains information about the current status of the HTTPProxy, in an upstream-friendly container. \n Contour will update a single condition, `Valid`, that is in normal-true polarity. That is, when `currentStatus` is `valid`, the `Valid` condition will be `status: true`, and vice versa. \n Contour will leave untouched any other Conditions set in this block, in case some other controller wants to add a Condition. \n If you are another controller owner and wish to add a condition, you *should* namespace your condition with a label, like `controller.domain.com/ConditionName`." + items: + description: "DetailedCondition is an extension of the normal Kubernetes conditions, with two extra fields to hold sub-conditions, which provide more detailed reasons for the state (True or False) of the condition. \n `errors` holds information about sub-conditions which are fatal to that condition and render its state False. \n `warnings` holds information about sub-conditions which are not fatal to that condition and do not force the state to be False. \n Remember that Conditions have a type, a status, and a reason. \n The type is the type of the condition, the most important one in this CRD set is `Valid`. `Valid` is a positive-polarity condition: when it is `status: true` there are no problems. \n In more detail, `status: true` means that the object is has been ingested into Contour with no errors. `warnings` may still be present, and will be indicated in the Reason field. There must be zero entries in the `errors` slice in this case. \n `Valid`, `status: false` means that the object has had one or more fatal errors during processing into Contour. The details of the errors will be present under the `errors` field. There must be at least one error in the `errors` slice if `status` is `false`. \n For DetailedConditions of types other than `Valid`, the Condition must be in the negative polarity. When they have `status` `true`, there is an error. There must be at least one entry in the `errors` Subcondition slice. When they have `status` `false`, there are no serious errors, and there must be zero entries in the `errors` slice. In either case, there may be entries in the `warnings` slice. \n Regardless of the polarity, the `reason` and `message` fields must be updated with either the detail of the reason (if there is one and only one entry in total across both the `errors` and `warnings` slices), or `MultipleReasons` if there is more than one entry." + properties: + errors: + description: "Errors contains a slice of relevant error subconditions for this object. \n Subconditions are expected to appear when relevant (when there is a error), and disappear when not relevant. An empty slice here indicates no errors." + items: + description: "SubCondition is a Condition-like type intended for use as a subcondition inside a DetailedCondition. \n It contains a subset of the Condition fields. \n It is intended for warnings and errors, so `type` names should use abnormal-true polarity, that is, they should be of the form \"ErrorPresent: true\". \n The expected lifecycle for these errors is that they should only be present when the error or warning is, and should be removed when they are not relevant." + properties: + message: + description: "Message is a human readable message indicating details about the transition. \n This may be an empty string." + maxLength: 32768 + type: string + reason: + description: "Reason contains a programmatic identifier indicating the reason for the condition's last transition. Producers of specific condition types may define expected values and meanings for this field, and whether the values are considered a guaranteed API. \n The value should be a CamelCase string. \n This field may not be empty." + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + description: Status of the condition, one of True, False, Unknown. + enum: + - "True" + - "False" + - Unknown + type: string + type: + description: "Type of condition in `CamelCase` or in `foo.example.com/CamelCase`. \n This must be in abnormal-true polarity, that is, `ErrorFound` or `controller.io/ErrorFound`. \n The regex it matches is (dns1123SubdomainFmt/)?(qualifiedNameFmt)" + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + required: + - message + - reason + - status + - type + type: object + type: array + lastTransitionTime: + description: lastTransitionTime is the last time the condition transitioned from one status to another. This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable. + format: date-time + type: string + message: + description: message is a human readable message indicating details about the transition. This may be an empty string. + maxLength: 32768 + type: string + observedGeneration: + description: observedGeneration represents the .metadata.generation that the condition was set based upon. For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date with respect to the current state of the instance. + format: int64 + minimum: 0 + type: integer + reason: + description: reason contains a programmatic identifier indicating the reason for the condition's last transition. Producers of specific condition types may define expected values and meanings for this field, and whether the values are considered a guaranteed API. The value should be a CamelCase string. This field may not be empty. + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + description: status of the condition, one of True, False, Unknown. + enum: + - "True" + - "False" + - Unknown + type: string + type: + description: type of condition in CamelCase or in foo.example.com/CamelCase. --- Many .condition.type values are consistent across resources like Available, but because arbitrary conditions can be useful (see .node.status.conditions), the ability to deconflict is important. The regex it matches is (dns1123SubdomainFmt/)?(qualifiedNameFmt) + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + warnings: + description: "Warnings contains a slice of relevant warning subconditions for this object. \n Subconditions are expected to appear when relevant (when there is a warning), and disappear when not relevant. An empty slice here indicates no warnings." + items: + description: "SubCondition is a Condition-like type intended for use as a subcondition inside a DetailedCondition. \n It contains a subset of the Condition fields. \n It is intended for warnings and errors, so `type` names should use abnormal-true polarity, that is, they should be of the form \"ErrorPresent: true\". \n The expected lifecycle for these errors is that they should only be present when the error or warning is, and should be removed when they are not relevant." + properties: + message: + description: "Message is a human readable message indicating details about the transition. \n This may be an empty string." + maxLength: 32768 + type: string + reason: + description: "Reason contains a programmatic identifier indicating the reason for the condition's last transition. Producers of specific condition types may define expected values and meanings for this field, and whether the values are considered a guaranteed API. \n The value should be a CamelCase string. \n This field may not be empty." + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + description: Status of the condition, one of True, False, Unknown. + enum: + - "True" + - "False" + - Unknown + type: string + type: + description: "Type of condition in `CamelCase` or in `foo.example.com/CamelCase`. \n This must be in abnormal-true polarity, that is, `ErrorFound` or `controller.io/ErrorFound`. \n The regex it matches is (dns1123SubdomainFmt/)?(qualifiedNameFmt)" + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + required: + - message + - reason + - status + - type + type: object + type: array + required: + - lastTransitionTime + - message + - reason + - status + - type + type: object + type: array + x-kubernetes-list-map-keys: + - type + x-kubernetes-list-type: map + currentStatus: + type: string + description: + type: string + loadBalancer: + description: LoadBalancer contains the current status of the load balancer. + properties: + ingress: + description: Ingress is a list containing ingress points for the load-balancer. Traffic intended for the service should be sent to these ingress points. + items: + description: 'LoadBalancerIngress represents the status of a load-balancer ingress point: traffic intended for the service should be sent to an ingress point.' + properties: + hostname: + description: Hostname is set for load-balancer ingress points that are DNS based (typically AWS load-balancers) + type: string + ip: + description: IP is set for load-balancer ingress points that are IP based (typically GCE or OpenStack load-balancers) + type: string + type: object + type: array + type: object + type: object + required: + - metadata + - spec + type: object + served: true + storage: true + subresources: + status: {} +status: + acceptedNames: + kind: "" + plural: "" + conditions: [] + storedVersions: [] +--- +# Source: flyte/charts/contour/templates/00-crds.yaml +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.2.9 + creationTimestamp: null + name: tlscertificatedelegations.projectcontour.io +spec: + preserveUnknownFields: false + group: projectcontour.io + names: + kind: TLSCertificateDelegation + listKind: TLSCertificateDelegationList + plural: tlscertificatedelegations + shortNames: + - tlscerts + singular: tlscertificatedelegation + scope: Namespaced + versions: + - name: v1 + schema: + openAPIV3Schema: + description: TLSCertificateDelegation is an TLS Certificate Delegation CRD specification. See design/tls-certificate-delegation.md for details. + properties: + apiVersion: + description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' + type: string + kind: + description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + type: string + metadata: + type: object + spec: + description: TLSCertificateDelegationSpec defines the spec of the CRD + properties: + delegations: + items: + description: CertificateDelegation maps the authority to reference a secret in the current namespace to a set of namespaces. + properties: + secretName: + description: required, the name of a secret in the current namespace. + type: string + targetNamespaces: + description: required, the namespaces the authority to reference the the secret will be delegated to. If TargetNamespaces is nil or empty, the CertificateDelegation' is ignored. If the TargetNamespace list contains the character, "*" the secret will be delegated to all namespaces. + items: + type: string + type: array + required: + - secretName + - targetNamespaces + type: object + type: array + required: + - delegations + type: object + status: + description: TLSCertificateDelegationStatus allows for the status of the delegation to be presented to the user. + properties: + conditions: + description: "Conditions contains information about the current status of the HTTPProxy, in an upstream-friendly container. \n Contour will update a single condition, `Valid`, that is in normal-true polarity. That is, when `currentStatus` is `valid`, the `Valid` condition will be `status: true`, and vice versa. \n Contour will leave untouched any other Conditions set in this block, in case some other controller wants to add a Condition. \n If you are another controller owner and wish to add a condition, you *should* namespace your condition with a label, like `controller.domain.com\\ConditionName`." + items: + description: "DetailedCondition is an extension of the normal Kubernetes conditions, with two extra fields to hold sub-conditions, which provide more detailed reasons for the state (True or False) of the condition. \n `errors` holds information about sub-conditions which are fatal to that condition and render its state False. \n `warnings` holds information about sub-conditions which are not fatal to that condition and do not force the state to be False. \n Remember that Conditions have a type, a status, and a reason. \n The type is the type of the condition, the most important one in this CRD set is `Valid`. `Valid` is a positive-polarity condition: when it is `status: true` there are no problems. \n In more detail, `status: true` means that the object is has been ingested into Contour with no errors. `warnings` may still be present, and will be indicated in the Reason field. There must be zero entries in the `errors` slice in this case. \n `Valid`, `status: false` means that the object has had one or more fatal errors during processing into Contour. The details of the errors will be present under the `errors` field. There must be at least one error in the `errors` slice if `status` is `false`. \n For DetailedConditions of types other than `Valid`, the Condition must be in the negative polarity. When they have `status` `true`, there is an error. There must be at least one entry in the `errors` Subcondition slice. When they have `status` `false`, there are no serious errors, and there must be zero entries in the `errors` slice. In either case, there may be entries in the `warnings` slice. \n Regardless of the polarity, the `reason` and `message` fields must be updated with either the detail of the reason (if there is one and only one entry in total across both the `errors` and `warnings` slices), or `MultipleReasons` if there is more than one entry." + properties: + errors: + description: "Errors contains a slice of relevant error subconditions for this object. \n Subconditions are expected to appear when relevant (when there is a error), and disappear when not relevant. An empty slice here indicates no errors." + items: + description: "SubCondition is a Condition-like type intended for use as a subcondition inside a DetailedCondition. \n It contains a subset of the Condition fields. \n It is intended for warnings and errors, so `type` names should use abnormal-true polarity, that is, they should be of the form \"ErrorPresent: true\". \n The expected lifecycle for these errors is that they should only be present when the error or warning is, and should be removed when they are not relevant." + properties: + message: + description: "Message is a human readable message indicating details about the transition. \n This may be an empty string." + maxLength: 32768 + type: string + reason: + description: "Reason contains a programmatic identifier indicating the reason for the condition's last transition. Producers of specific condition types may define expected values and meanings for this field, and whether the values are considered a guaranteed API. \n The value should be a CamelCase string. \n This field may not be empty." + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + description: Status of the condition, one of True, False, Unknown. + enum: + - "True" + - "False" + - Unknown + type: string + type: + description: "Type of condition in `CamelCase` or in `foo.example.com/CamelCase`. \n This must be in abnormal-true polarity, that is, `ErrorFound` or `controller.io/ErrorFound`. \n The regex it matches is (dns1123SubdomainFmt/)?(qualifiedNameFmt)" + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + required: + - message + - reason + - status + - type + type: object + type: array + lastTransitionTime: + description: lastTransitionTime is the last time the condition transitioned from one status to another. This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable. + format: date-time + type: string + message: + description: message is a human readable message indicating details about the transition. This may be an empty string. + maxLength: 32768 + type: string + observedGeneration: + description: observedGeneration represents the .metadata.generation that the condition was set based upon. For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date with respect to the current state of the instance. + format: int64 + minimum: 0 + type: integer + reason: + description: reason contains a programmatic identifier indicating the reason for the condition's last transition. Producers of specific condition types may define expected values and meanings for this field, and whether the values are considered a guaranteed API. The value should be a CamelCase string. This field may not be empty. + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + description: status of the condition, one of True, False, Unknown. + enum: + - "True" + - "False" + - Unknown + type: string + type: + description: type of condition in CamelCase or in foo.example.com/CamelCase. --- Many .condition.type values are consistent across resources like Available, but because arbitrary conditions can be useful (see .node.status.conditions), the ability to deconflict is important. The regex it matches is (dns1123SubdomainFmt/)?(qualifiedNameFmt) + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + warnings: + description: "Warnings contains a slice of relevant warning subconditions for this object. \n Subconditions are expected to appear when relevant (when there is a warning), and disappear when not relevant. An empty slice here indicates no warnings." + items: + description: "SubCondition is a Condition-like type intended for use as a subcondition inside a DetailedCondition. \n It contains a subset of the Condition fields. \n It is intended for warnings and errors, so `type` names should use abnormal-true polarity, that is, they should be of the form \"ErrorPresent: true\". \n The expected lifecycle for these errors is that they should only be present when the error or warning is, and should be removed when they are not relevant." + properties: + message: + description: "Message is a human readable message indicating details about the transition. \n This may be an empty string." + maxLength: 32768 + type: string + reason: + description: "Reason contains a programmatic identifier indicating the reason for the condition's last transition. Producers of specific condition types may define expected values and meanings for this field, and whether the values are considered a guaranteed API. \n The value should be a CamelCase string. \n This field may not be empty." + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + description: Status of the condition, one of True, False, Unknown. + enum: + - "True" + - "False" + - Unknown + type: string + type: + description: "Type of condition in `CamelCase` or in `foo.example.com/CamelCase`. \n This must be in abnormal-true polarity, that is, `ErrorFound` or `controller.io/ErrorFound`. \n The regex it matches is (dns1123SubdomainFmt/)?(qualifiedNameFmt)" + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + required: + - message + - reason + - status + - type + type: object + type: array + required: + - lastTransitionTime + - message + - reason + - status + - type + type: object + type: array + x-kubernetes-list-map-keys: + - type + x-kubernetes-list-type: map + type: object + required: + - metadata + - spec + type: object + served: true + storage: true + subresources: + status: {} +status: + acceptedNames: + kind: "" + plural: "" + conditions: [] + storedVersions: [] +--- +# Source: flyte/templates/propeller/crds/flyteworkflow.yaml +apiVersion: apiextensions.k8s.io/v1beta1 +kind: CustomResourceDefinition +metadata: + name: flyteworkflows.flyte.lyft.com +spec: + group: flyte.lyft.com + names: + kind: FlyteWorkflow + plural: flyteworkflows + shortNames: + - fly + singular: flyteworkflow + scope: Namespaced + version: v1alpha1 +--- +# Source: flyte/charts/contour/templates/contour/rbac.yaml +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: flyte-contour-contour + labels: + app.kubernetes.io/name: contour + helm.sh/chart: contour-4.1.2 + app.kubernetes.io/instance: flyte + app.kubernetes.io/managed-by: Helm +rules: + - apiGroups: + - "" + resources: + - configmaps + verbs: + - create + - get + - update + - apiGroups: + - "" + resources: + - endpoints + verbs: + - get + - list + - watch + - apiGroups: + - "" + resources: + - secrets + verbs: + - get + - list + - watch + - apiGroups: + - "" + resources: + - services + verbs: + - get + - list + - watch + - apiGroups: + - apiextensions.k8s.io + resources: + - customresourcedefinitions + verbs: + - list + - apiGroups: + - networking.k8s.io + resources: + - gatewayclasses + - gateways + - httproutes + - tcproutes + verbs: + - get + - list + - watch + - apiGroups: + - networking.k8s.io + resources: + - ingresses + verbs: + - get + - list + - watch + - apiGroups: + - networking.k8s.io + resources: + - ingresses/status + verbs: + - create + - get + - update + - apiGroups: + - projectcontour.io + resources: + - extensionservices + verbs: + - get + - list + - watch + - apiGroups: + - projectcontour.io + resources: + - extensionservices/status + verbs: + - create + - get + - update + - apiGroups: + - projectcontour.io + resources: + - httpproxies + - tlscertificatedelegations + verbs: + - get + - list + - watch + - apiGroups: + - projectcontour.io + resources: + - httpproxies/status + verbs: + - create + - get + - update +--- +# Source: flyte/charts/kubernetes-dashboard/templates/clusterrole-metrics.yaml +# Copyright 2017 The Kubernetes Authors. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +kind: ClusterRole +apiVersion: rbac.authorization.k8s.io/v1 +metadata: + name: "flyte-kubernetes-dashboard-metrics" + labels: + app.kubernetes.io/name: kubernetes-dashboard + helm.sh/chart: kubernetes-dashboard-4.0.2 + app.kubernetes.io/instance: flyte + app.kubernetes.io/version: "2.2.0" + app.kubernetes.io/managed-by: Helm +rules: + # Allow Metrics Scraper to get metrics from the Metrics server + - apiGroups: ["metrics.k8s.io"] + resources: ["pods", "nodes"] + verbs: ["get", "list", "watch"] +--- +# Source: flyte/charts/sparkoperator/templates/rbac.yaml +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: flyte-sparkoperator + labels: + helm.sh/chart: sparkoperator-1.0.6 + app.kubernetes.io/name: sparkoperator + app.kubernetes.io/instance: flyte + app.kubernetes.io/version: "v1beta2-1.2.0-3.0.0" + app.kubernetes.io/managed-by: Helm +rules: +- apiGroups: + - "" + resources: + - pods + verbs: + - "*" +- apiGroups: + - "" + resources: + - services + - configmaps + - secrets + verbs: + - create + - get + - delete + - update +- apiGroups: + - extensions + - networking.k8s.io + resources: + - ingresses + verbs: + - create + - get + - delete +- apiGroups: + - "" + resources: + - nodes + verbs: + - get +- apiGroups: + - "" + resources: + - events + verbs: + - create + - update + - patch +- apiGroups: + - "" + resources: + - resourcequotas + verbs: + - get + - list + - watch +- apiGroups: + - apiextensions.k8s.io + resources: + - customresourcedefinitions + verbs: + - create + - get + - update + - delete +- apiGroups: + - admissionregistration.k8s.io + resources: + - mutatingwebhookconfigurations + - validatingwebhookconfigurations + verbs: + - create + - get + - update + - delete +- apiGroups: + - sparkoperator.k8s.io + resources: + - sparkapplications + - sparkapplications/status + - scheduledsparkapplications + - scheduledsparkapplications/status + verbs: + - "*" +--- +# Source: flyte/templates/admin/rbac.yaml +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: flyteadmin + labels: + app.kubernetes.io/name: flyteadmin + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm +rules: +- apiGroups: + - "" + - flyte.lyft.com + - rbac.authorization.k8s.io + resources: + - configmaps + - flyteworkflows + - namespaces + - pods + - resourcequotas + - roles + - rolebindings + - secrets + - services + - serviceaccounts + - spark-role + verbs: + - '*' +--- +# Source: flyte/templates/propeller/rbac.yaml +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: flytepropeller + labels: + app.kubernetes.io/name: flytepropeller + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm +rules: +# Allow RO access to PODS +- apiGroups: + - "" + resources: + - pods + verbs: + - get + - list + - watch +# Allow Event recording access +- apiGroups: + - "" + resources: + - events + verbs: + - create + - update + - delete + - patch +# Allow Access All plugin objects +- apiGroups: + - '*' + resources: + - '*' + verbs: + - get + - list + - watch + - create + - update + - delete + - patch +# Allow Access to CRD +- apiGroups: + - apiextensions.k8s.io + resources: + - customresourcedefinitions + verbs: + - get + - list + - watch + - create + - delete + - update +# Allow Access to all resources under flyte.lyft.com +- apiGroups: + - flyte.lyft.com + resources: + - flyteworkflows + - flyteworkflows/finalizers + verbs: + - get + - list + - watch + - create + - update + - delete + - patch + - post + - deletecollection +--- +# Source: flyte/templates/propeller/webhook.yaml +# Create a ClusterRole for the webhook +# https://kubernetes.io/docs/admin/authorization/rbac/ +kind: ClusterRole +apiVersion: rbac.authorization.k8s.io/v1 +metadata: + name: flyte-pod-webhook + namespace: default +rules: + - apiGroups: + - "*" + resources: + - mutatingwebhookconfigurations + - secrets + - pods + verbs: + - get + - create + - update + - patch +--- +# Source: flyte/charts/contour/templates/contour/rbac.yaml +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding +metadata: + name: flyte-contour-contour + labels: + app.kubernetes.io/name: contour + helm.sh/chart: contour-4.1.2 + app.kubernetes.io/instance: flyte + app.kubernetes.io/managed-by: Helm +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: flyte-contour-contour +subjects: + - kind: ServiceAccount + name: flyte-contour-contour + namespace: default +--- +# Source: flyte/charts/kubernetes-dashboard/templates/clusterrolebinding-metrics.yaml +# Copyright 2017 The Kubernetes Authors. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding +metadata: + name: "flyte-kubernetes-dashboard-metrics" + labels: + app.kubernetes.io/name: kubernetes-dashboard + helm.sh/chart: kubernetes-dashboard-4.0.2 + app.kubernetes.io/instance: flyte + app.kubernetes.io/version: "2.2.0" + app.kubernetes.io/managed-by: Helm +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: flyte-kubernetes-dashboard-metrics +subjects: + - kind: ServiceAccount + name: flyte-kubernetes-dashboard + namespace: default +--- +# Source: flyte/charts/sparkoperator/templates/rbac.yaml +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding +metadata: + name: flyte-sparkoperator + labels: + helm.sh/chart: sparkoperator-1.0.6 + app.kubernetes.io/name: sparkoperator + app.kubernetes.io/instance: flyte + app.kubernetes.io/version: "v1beta2-1.2.0-3.0.0" + app.kubernetes.io/managed-by: Helm +subjects: + - kind: ServiceAccount + name: flyte-sparkoperator + namespace: default +roleRef: + kind: ClusterRole + name: flyte-sparkoperator + apiGroup: rbac.authorization.k8s.io +--- +# Source: flyte/templates/admin/rbac.yaml +apiVersion: rbac.authorization.k8s.io/v1beta1 +kind: ClusterRoleBinding +metadata: + name: flyteadmin-binding + labels: + app.kubernetes.io/name: flyteadmin + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: flyteadmin +subjects: +- kind: ServiceAccount + name: flyteadmin + namespace: default +--- +# Source: flyte/templates/propeller/rbac.yaml +apiVersion: rbac.authorization.k8s.io/v1beta1 +kind: ClusterRoleBinding +metadata: + name: flytepropeller + labels: + app.kubernetes.io/name: flytepropeller + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: flytepropeller +subjects: +- kind: ServiceAccount + name: flytepropeller + namespace: default +--- +# Source: flyte/templates/propeller/webhook.yaml +# Create a binding from Role -> ServiceAccount +kind: ClusterRoleBinding +apiVersion: rbac.authorization.k8s.io/v1 +metadata: + name: flyte-pod-webhook + namespace: default +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: flyte-pod-webhook +subjects: + - kind: ServiceAccount + name: flyte-pod-webhook + namespace: default +--- +# Source: flyte/charts/kubernetes-dashboard/templates/role.yaml +# Copyright 2017 The Kubernetes Authors. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +apiVersion: rbac.authorization.k8s.io/v1 +kind: Role +metadata: + name: flyte-kubernetes-dashboard + labels: + app.kubernetes.io/name: kubernetes-dashboard + helm.sh/chart: kubernetes-dashboard-4.0.2 + app.kubernetes.io/instance: flyte + app.kubernetes.io/version: "2.2.0" + app.kubernetes.io/managed-by: Helm +rules: + # Allow Dashboard to get, update and delete Dashboard exclusive secrets. + - apiGroups: [""] + resources: ["secrets"] + resourceNames: ["kubernetes-dashboard-key-holder", "kubernetes-dashboard-certs", "kubernetes-dashboard-csrf"] + verbs: ["get", "update", "delete"] + # Allow Dashboard to get and update 'kubernetes-dashboard-settings' config map. + - apiGroups: [""] + resources: ["configmaps"] + resourceNames: ["kubernetes-dashboard-settings"] + verbs: ["get", "update"] + # Allow Dashboard to get metrics. + - apiGroups: [""] + resources: ["services"] + resourceNames: ["heapster", "dashboard-metrics-scraper"] + verbs: ["proxy"] + - apiGroups: [""] + resources: ["services/proxy"] + resourceNames: ["heapster", "http:heapster:", "https:heapster:", "dashboard-metrics-scraper", "http:dashboard-metrics-scraper"] + verbs: ["get"] +--- +# Source: flyte/charts/sparkoperator/templates/spark-rbac.yaml +apiVersion: rbac.authorization.k8s.io/v1 +kind: Role +metadata: + name: spark-role + namespace: default + labels: + helm.sh/chart: sparkoperator-1.0.6 + app.kubernetes.io/name: sparkoperator + app.kubernetes.io/instance: flyte + app.kubernetes.io/version: "v1beta2-1.2.0-3.0.0" + app.kubernetes.io/managed-by: Helm +rules: +- apiGroups: + - "" + resources: + - pods + verbs: + - "*" +- apiGroups: + - "" + resources: + - services + verbs: + - "*" +--- +# Source: flyte/charts/kubernetes-dashboard/templates/rolebinding.yaml +# Copyright 2017 The Kubernetes Authors. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +apiVersion: rbac.authorization.k8s.io/v1 +kind: RoleBinding +metadata: + name: flyte-kubernetes-dashboard + labels: + app.kubernetes.io/name: kubernetes-dashboard + helm.sh/chart: kubernetes-dashboard-4.0.2 + app.kubernetes.io/instance: flyte + app.kubernetes.io/version: "2.2.0" + app.kubernetes.io/managed-by: Helm +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: Role + name: flyte-kubernetes-dashboard +subjects: + - kind: ServiceAccount + name: flyte-kubernetes-dashboard + namespace: default +--- +# Source: flyte/charts/sparkoperator/templates/spark-rbac.yaml +apiVersion: rbac.authorization.k8s.io/v1 +kind: RoleBinding +metadata: + name: spark + namespace: default + labels: + helm.sh/chart: sparkoperator-1.0.6 + app.kubernetes.io/name: sparkoperator + app.kubernetes.io/instance: flyte + app.kubernetes.io/version: "v1beta2-1.2.0-3.0.0" + app.kubernetes.io/managed-by: Helm +subjects: +- kind: ServiceAccount + name: flyte-spark + namespace: default +roleRef: + kind: Role + name: spark-role + apiGroup: rbac.authorization.k8s.io +--- +# Source: flyte/charts/contour/templates/contour/service.yaml +apiVersion: v1 +kind: Service +metadata: + name: flyte-contour + namespace: default + labels: + app.kubernetes.io/name: contour + helm.sh/chart: contour-4.1.2 + app.kubernetes.io/instance: flyte + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/component: contour +spec: + ports: + - port: 8001 + name: xds + protocol: TCP + targetPort: 8001 + selector: + app.kubernetes.io/name: contour + app.kubernetes.io/instance: flyte + app.kubernetes.io/component: contour + type: ClusterIP +--- +# Source: flyte/charts/contour/templates/envoy/service.yaml +apiVersion: v1 +kind: Service +metadata: + name: flyte-contour-envoy + namespace: default + labels: + app.kubernetes.io/name: contour + helm.sh/chart: contour-4.1.2 + app.kubernetes.io/instance: flyte + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/component: envoy + annotations: + # This annotation puts the AWS ELB into "TCP" mode so that it does not + # do HTTP negotiation for HTTPS connections at the ELB edge. + # The downside of this is the remote IP address of all connections will + # appear to be the internal address of the ELB. See docs/proxy-proto.md + # for information about enabling the PROXY protocol on the ELB to recover + # the original remote IP address. + # We don't set this for nlb, per the contour docs. + service.beta.kubernetes.io/aws-load-balancer-backend-protocol: tcp +spec: + externalTrafficPolicy: "Local" + ports: + - name: http + port: 80 + protocol: TCP + targetPort: http + nodePort: 30081 + - name: https + port: 443 + protocol: TCP + targetPort: https + selector: + app.kubernetes.io/name: contour + app.kubernetes.io/instance: flyte + app.kubernetes.io/component: envoy + type: NodePort +--- +# Source: flyte/charts/kubernetes-dashboard/templates/service.yaml +# Copyright 2017 The Kubernetes Authors. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +apiVersion: v1 +kind: Service +metadata: + name: flyte-kubernetes-dashboard + labels: + app.kubernetes.io/name: kubernetes-dashboard + helm.sh/chart: kubernetes-dashboard-4.0.2 + app.kubernetes.io/instance: flyte + app.kubernetes.io/version: "2.2.0" + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/component: kubernetes-dashboard + kubernetes.io/cluster-service: "true" +spec: + type: NodePort + ports: + - port: 30082 + targetPort: http + name: http + selector: + app.kubernetes.io/name: kubernetes-dashboard + app.kubernetes.io/instance: flyte + app.kubernetes.io/component: kubernetes-dashboard +--- +# Source: flyte/templates/admin/service.yaml +apiVersion: v1 +kind: Service +metadata: + name: flyteadmin + namespace: default + labels: + app.kubernetes.io/name: flyteadmin + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm + annotations: + projectcontour.io/upstream-protocol.h2c: grpc +spec: + type: ClusterIP + ports: + - name: http + port: 80 + protocol: TCP + targetPort: 8088 + - name: grpc + port: 81 + protocol: TCP + targetPort: 8089 + - name: redoc + protocol: TCP + port: 87 + targetPort: 8087 + - name: http-metrics + protocol: TCP + port: 10254 + selector: + app.kubernetes.io/name: flyteadmin + app.kubernetes.io/instance: flyte +--- +# Source: flyte/templates/console/service.yaml +apiVersion: v1 +kind: Service +metadata: + name: flyteconsole + namespace: default + labels: + app.kubernetes.io/name: flyteconsole + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm +spec: + type: ClusterIP + ports: + - port: 80 + protocol: TCP + targetPort: 8080 + selector: + app.kubernetes.io/name: flyteconsole + app.kubernetes.io/instance: flyte +--- +# Source: flyte/templates/datacatalog/service.yaml +apiVersion: v1 +kind: Service +metadata: + name: datacatalog + namespace: default + labels: + app.kubernetes.io/name: datacatalog + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm + annotations: + projectcontour.io/upstream-protocol.h2c: grpc +spec: + type: NodePort + ports: + - name: grpc-2 + port: 8089 + protocol: TCP + targetPort: 8089 + - name: http + port: 88 + protocol: TCP + targetPort: 8088 + - name: grpc + port: 89 + protocol: TCP + targetPort: 8089 + selector: + app.kubernetes.io/name: datacatalog + app.kubernetes.io/instance: flyte +--- +# Source: flyte/templates/minio/service.yaml +apiVersion: v1 +kind: Service +metadata: + name: minio + namespace: default + labels: + app.kubernetes.io/name: minio + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm +spec: + type: ClusterIP + ports: + - name: minio + port: 9000 + protocol: TCP + targetPort: minio + selector: + app.kubernetes.io/name: minio + app.kubernetes.io/instance: flyte +--- +# Source: flyte/templates/postgres/service.yaml +apiVersion: v1 +kind: Service +metadata: + name: postgres + namespace: default + labels: + app.kubernetes.io/name: postgres + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm +spec: + type: ClusterIP + ports: + - name: postgres + port: 5432 + protocol: TCP + targetPort: postgres + selector: + app.kubernetes.io/name: postgres + app.kubernetes.io/instance: flyte +--- +# Source: flyte/templates/propeller/webhook.yaml +# Service +apiVersion: v1 +kind: Service +metadata: + name: flyte-pod-webhook + namespace: default + annotations: + projectcontour.io/upstream-protocol.h2c: grpc +spec: + selector: + app: flyte-pod-webhook + ports: + - name: https + protocol: TCP + port: 443 + targetPort: 9443 +--- +# Source: flyte/charts/contour/templates/envoy/daemonset.yaml +apiVersion: apps/v1 +kind: DaemonSet +metadata: + name: flyte-contour-envoy + namespace: default + labels: + app.kubernetes.io/name: contour + helm.sh/chart: contour-4.1.2 + app.kubernetes.io/instance: flyte + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/component: envoy +spec: + updateStrategy: + type: RollingUpdate + rollingUpdate: + maxUnavailable: 10% + selector: + matchLabels: + app.kubernetes.io/name: contour + app.kubernetes.io/instance: flyte + app.kubernetes.io/component: envoy + template: + metadata: + labels: + app.kubernetes.io/name: contour + helm.sh/chart: contour-4.1.2 + app.kubernetes.io/instance: flyte + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/component: envoy + spec: + + affinity: + podAffinity: + + podAntiAffinity: + + nodeAffinity: + + terminationGracePeriodSeconds: 300 + hostNetwork: false + dnsPolicy: ClusterFirst + containers: + - command: + - contour + args: + - envoy + - shutdown-manager + image: docker.io/bitnami/contour:1.12.0-debian-10-r0 + imagePullPolicy: IfNotPresent + lifecycle: + preStop: + exec: + command: + - contour + - envoy + - shutdown + livenessProbe: + httpGet: + path: /healthz + port: 8090 + initialDelaySeconds: 120 + periodSeconds: 20 + timeoutSeconds: 5 + successThreshold: 1 + failureThreshold: 6 + name: shutdown-manager + - command: + - envoy + args: + - -c + - /config/envoy.json + - --service-cluster $(CONTOUR_NAMESPACE) + - --service-node $(ENVOY_POD_NAME) + - --log-level info + image: docker.io/bitnami/envoy:1.16.2-debian-10-r54 + imagePullPolicy: IfNotPresent + name: envoy + securityContext: + runAsUser: 0 + env: + - name: CONTOUR_NAMESPACE + valueFrom: + fieldRef: + apiVersion: v1 + fieldPath: metadata.namespace + - name: ENVOY_POD_NAME + valueFrom: + fieldRef: + apiVersion: v1 + fieldPath: metadata.name + ports: + - containerPort: 80 + hostPort: 80 + name: http + protocol: TCP + - containerPort: 443 + hostPort: 443 + name: https + protocol: TCP + - containerPort: 8002 + name: metrics + protocol: TCP + readinessProbe: + httpGet: + path: /ready + port: 8002 + initialDelaySeconds: 10 + periodSeconds: 3 + timeoutSeconds: 1 + successThreshold: 1 + failureThreshold: 3 + livenessProbe: + httpGet: + path: /ready + port: 8002 + initialDelaySeconds: 120 + periodSeconds: 20 + timeoutSeconds: 5 + successThreshold: 1 + failureThreshold: 6 + resources: + limits: + cpu: 100m + memory: 100Mi + requests: + cpu: 10m + memory: 50Mi + volumeMounts: + - name: envoy-config + mountPath: /config + - name: envoycert + mountPath: /certs + lifecycle: + preStop: + httpGet: + path: /shutdown + port: 8090 + scheme: HTTP + initContainers: + - command: + - contour + args: + - bootstrap + - /config/envoy.json + - --xds-address=flyte-contour + - --xds-port=8001 + - --resources-dir=/config/resources + - --envoy-cafile=/certs/ca.crt + - --envoy-cert-file=/certs/tls.crt + - --envoy-key-file=/certs/tls.key + image: docker.io/bitnami/contour:1.12.0-debian-10-r0 + imagePullPolicy: IfNotPresent + name: envoy-initconfig + resources: + limits: + cpu: 100m + memory: 100Mi + requests: + cpu: 10m + memory: 50Mi + volumeMounts: + - name: envoy-config + mountPath: /config + - name: envoycert + mountPath: /certs + readOnly: true + env: + - name: CONTOUR_NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace + automountServiceAccountToken: false + serviceAccountName: flyte-contour-envoy + volumes: + - name: envoy-config + emptyDir: {} + - name: envoycert + secret: + secretName: envoycert + restartPolicy: Always +--- +# Source: flyte/charts/contour/templates/contour/deployment.yaml +apiVersion: apps/v1 +kind: Deployment +metadata: + name: flyte-contour-contour + namespace: default + labels: + app.kubernetes.io/name: contour + helm.sh/chart: contour-4.1.2 + app.kubernetes.io/instance: flyte + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/component: contour +spec: + replicas: 1 + strategy: + type: RollingUpdate + rollingUpdate: + # This value of maxSurge means that during a rolling update + # the new ReplicaSet will be created first. + maxSurge: 50% + selector: + matchLabels: + app.kubernetes.io/name: contour + app.kubernetes.io/instance: flyte + app.kubernetes.io/component: contour + template: + metadata: + annotations: + checksum/config: 1c6f180e2479c6aff9159e6eea066d5238a8c9ee1f38f19a6de2f070a88e5ad9 + labels: + app.kubernetes.io/name: contour + helm.sh/chart: contour-4.1.2 + app.kubernetes.io/instance: flyte + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/component: contour + spec: + + affinity: + podAffinity: + + podAntiAffinity: + preferredDuringSchedulingIgnoredDuringExecution: + - podAffinityTerm: + labelSelector: + matchLabels: + app.kubernetes.io/name: contour + app.kubernetes.io/instance: flyte + app.kubernetes.io/component: contour + namespaces: + - "default" + topologyKey: kubernetes.io/hostname + weight: 1 + nodeAffinity: + + containers: + - command: + - contour + args: + - serve + - --incluster + - --xds-address=0.0.0.0 + - --xds-port=8001 + - --envoy-service-http-port=80 + - --envoy-service-https-port=443 + - --contour-cafile=/certs/ca.crt + - --contour-cert-file=/certs/tls.crt + - --contour-key-file=/certs/tls.key + - --config-path=/config/contour.yaml + image: docker.io/bitnami/contour:1.12.0-debian-10-r0 + imagePullPolicy: IfNotPresent + name: contour + ports: + - containerPort: 8001 + name: xds + protocol: TCP + - containerPort: 8000 + name: metrics + protocol: TCP + livenessProbe: + httpGet: + path: /healthz + port: 8000 + initialDelaySeconds: 120 + periodSeconds: 20 + timeoutSeconds: 5 + successThreshold: 1 + failureThreshold: 6 + readinessProbe: + tcpSocket: + port: 8001 + initialDelaySeconds: 15 + periodSeconds: 10 + timeoutSeconds: 5 + successThreshold: 1 + failureThreshold: 3 + resources: + limits: + cpu: 100m + memory: 100Mi + requests: + cpu: 10m + memory: 50Mi + volumeMounts: + - name: contourcert + mountPath: /certs + readOnly: true + - name: contour-config + mountPath: /config + readOnly: true + env: + - name: CONTOUR_NAMESPACE + valueFrom: + fieldRef: + apiVersion: v1 + fieldPath: metadata.namespace + - name: POD_NAME + valueFrom: + fieldRef: + apiVersion: v1 + fieldPath: metadata.name + dnsPolicy: ClusterFirst + serviceAccountName: flyte-contour-contour + securityContext: + runAsUser: 1001 + runAsGroup: 1001 + fsGroup: + runAsNonRoot: true + volumes: + - name: contourcert + secret: + secretName: contourcert + - name: contour-config + configMap: + name: flyte-contour + defaultMode: 0644 + items: + - key: contour.yaml + path: contour.yaml +--- +# Source: flyte/charts/kubernetes-dashboard/templates/deployment.yaml +# Copyright 2017 The Kubernetes Authors. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +apiVersion: apps/v1 +kind: Deployment +metadata: + name: flyte-kubernetes-dashboard + labels: + app.kubernetes.io/name: kubernetes-dashboard + helm.sh/chart: kubernetes-dashboard-4.0.2 + app.kubernetes.io/instance: flyte + app.kubernetes.io/version: "2.2.0" + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/component: kubernetes-dashboard +spec: + replicas: 1 + strategy: + rollingUpdate: + maxSurge: 0 + maxUnavailable: 1 + type: RollingUpdate + selector: + matchLabels: + app.kubernetes.io/name: kubernetes-dashboard + app.kubernetes.io/instance: flyte + app.kubernetes.io/component: kubernetes-dashboard + template: + metadata: + annotations: + seccomp.security.alpha.kubernetes.io/pod: runtime/default + labels: + app.kubernetes.io/name: kubernetes-dashboard + helm.sh/chart: kubernetes-dashboard-4.0.2 + app.kubernetes.io/instance: flyte + app.kubernetes.io/version: "2.2.0" + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/component: kubernetes-dashboard + spec: + serviceAccountName: flyte-kubernetes-dashboard + containers: + - name: kubernetes-dashboard + image: "kubernetesui/dashboard:v2.2.0" + imagePullPolicy: IfNotPresent + args: + - --namespace=default + - --metrics-provider=none + - --enable-skip-login + - --enable-insecure-login + - --disable-settings-authorizer + ports: + - name: http + containerPort: 9090 + protocol: TCP + volumeMounts: + - name: kubernetes-dashboard-certs + mountPath: /certs + # Create on-disk volume to store exec logs + - mountPath: /tmp + name: tmp-volume + livenessProbe: + httpGet: + scheme: HTTP + path: / + port: 9090 + initialDelaySeconds: 30 + timeoutSeconds: 30 + resources: + limits: + cpu: 2 + memory: 200Mi + requests: + cpu: 100m + memory: 200Mi + securityContext: + allowPrivilegeEscalation: false + readOnlyRootFilesystem: true + runAsGroup: 2001 + runAsUser: 1001 + volumes: + - name: kubernetes-dashboard-certs + secret: + secretName: flyte-kubernetes-dashboard-certs + - name: tmp-volume + emptyDir: {} +--- +# Source: flyte/charts/sparkoperator/templates/deployment.yaml +# If the admission webhook is enabled, then a post-install step is required +# to generate and install the secret in the operator namespace. + +# In the post-install hook, the token corresponding to the operator service account +# is used to authenticate with the Kubernetes API server to install the secret bundle. + +apiVersion: apps/v1 +kind: Deployment +metadata: + name: flyte-sparkoperator + labels: + helm.sh/chart: sparkoperator-1.0.6 + app.kubernetes.io/name: sparkoperator + app.kubernetes.io/instance: flyte + app.kubernetes.io/version: "v1beta2-1.2.0-3.0.0" + app.kubernetes.io/managed-by: Helm +spec: + replicas: 1 + selector: + matchLabels: + app.kubernetes.io/name: sparkoperator + app.kubernetes.io/instance: flyte + strategy: + type: Recreate + template: + metadata: + annotations: + prometheus.io/scrape: "true" + prometheus.io/port: "10254" + prometheus.io/path: /metrics + labels: + app.kubernetes.io/name: sparkoperator + app.kubernetes.io/instance: flyte + spec: + serviceAccountName: flyte-sparkoperator + securityContext: + {} + containers: + - name: sparkoperator + image: gcr.io/spark-operator/spark-operator:v1beta2-1.2.0-3.0.0 + imagePullPolicy: IfNotPresent + securityContext: + {} + ports: + - name: "metrics" + containerPort: 10254 + + args: + - -v=2 + - -logtostderr + - -namespace= + - -ingress-url-format= + - -controller-threads=10 + - -resync-interval=30 + - -enable-batch-scheduler=false + - -enable-metrics=true + - -metrics-labels=app_type + - -metrics-port=10254 + - -metrics-endpoint=/metrics + - -metrics-prefix= + - -enable-resource-quota-enforcement=false + resources: + limits: + cpu: 1000m + memory: 500M + requests: + cpu: 10m + memory: 50M +--- +# Source: flyte/templates/admin/deployment.yaml +apiVersion: apps/v1 +kind: Deployment +metadata: + name: flyteadmin + namespace: default + labels: + app.kubernetes.io/name: flyteadmin + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm +spec: + replicas: 1 + selector: + matchLabels: + app.kubernetes.io/name: flyteadmin + app.kubernetes.io/instance: flyte + template: + metadata: + annotations: + configChecksum: "8ad93c5353943acd48572ff9db564f0a8544904593c5eee727204420a4b4239" + labels: + app.kubernetes.io/name: flyteadmin + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm + spec: + initContainers: + - name: check-db-ready + image: postgres:10.16-alpine + command: + - sh + - -c + - until pg_isready -h postgres -p 5432; do echo waiting for database; sleep 2; done; + - command: + - flyteadmin + - --config + - /etc/flyte/config/*.yaml + - migrate + - run + image: "ghcr.io/flyteorg/flyteadmin:v0.5.7" + imagePullPolicy: "IfNotPresent" + name: run-migrations + volumeMounts: + + - mountPath: /etc/flyte/config + name: config-volume + - command: + - flyteadmin + - --config + - /etc/flyte/config/*.yaml + - migrate + - seed-projects + - flytesnacks + - flytetester + - flyteexamples + image: "ghcr.io/flyteorg/flyteadmin:v0.5.7" + imagePullPolicy: "IfNotPresent" + name: seed-projects + volumeMounts: + + - mountPath: /etc/flyte/config + name: config-volume + - command: + - flyteadmin + - --config + - /etc/flyte/config/*.yaml + - clusterresource + - sync + image: "ghcr.io/flyteorg/flyteadmin:v0.5.7" + imagePullPolicy: "IfNotPresent" + name: sync-cluster-resources + volumeMounts: + + - mountPath: /etc/flyte/clusterresource/templates + name: resource-templates + - mountPath: /etc/flyte/config + name: config-volume + - name: generate-secrets + image: "ghcr.io/flyteorg/flyteadmin:v0.5.7" + imagePullPolicy: "IfNotPresent" + command: ["/bin/sh", "-c"] + args: + [ + "flyteadmin --config=/etc/flyte/config/*.yaml secrets init --localPath /etc/secrets/auth && flyteadmin --config=/etc/flyte/config/*.yaml secrets create --fromPath /etc/secrets/auth", + ] + volumeMounts: + - name: config-volume + mountPath: /etc/flyte/config + env: + - name: POD_NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace + containers: + - command: + - flyteadmin + - --config + - /etc/flyte/config/*.yaml + - serve + image: "ghcr.io/flyteorg/flyteadmin:v0.5.7" + imagePullPolicy: "IfNotPresent" + name: flyteadmin + ports: + - containerPort: 8088 + - containerPort: 8089 + resources: + limits: + cpu: 250m + ephemeral-storage: 100Mi + memory: 500Mi + requests: + cpu: 10m + ephemeral-storage: 50Mi + memory: 50Mi + volumeMounts: + + - mountPath: /srv/flyte + name: shared-data + - mountPath: /etc/flyte/config + name: config-volume + - name: auth + mountPath: /etc/secrets/ + - command: + - sh + - -c + - ln -s /usr/share/nginx/html /usr/share/nginx/html/openapi && sh /usr/local/bin/docker-run.sh + env: + - name: PAGE_TITLE + value: Flyte Admin OpenAPI + - name: SPEC_URL + value: /api/v1/openapi + - name: PORT + value: "8087" + image: docker.io/redocly/redoc + imagePullPolicy: IfNotPresent + name: redoc + ports: + - containerPort: 8087 + resources: + limits: + cpu: "0.1" + memory: 200Mi + serviceAccountName: flyteadmin + volumes: + + - emptyDir: {} + name: shared-data + - configMap: + name: flyte-admin-config + name: config-volume + - configMap: + name: clusterresource-template + name: resource-templates + - name: auth + secret: + secretName: flyte-admin-auth +--- +# Source: flyte/templates/console/deployment.yaml +apiVersion: apps/v1 +kind: Deployment +metadata: + name: flyteconsole + namespace: default + labels: + app.kubernetes.io/name: flyteconsole + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm +spec: + replicas: 1 + selector: + matchLabels: + app.kubernetes.io/name: flyteconsole + app.kubernetes.io/instance: flyte + template: + metadata: + annotations: + configChecksum: "a14c2477f002d1392494a730c37ba7109e61df5cd417d431efa36841ee56bb9" + labels: + app.kubernetes.io/name: flyteconsole + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm + spec: + containers: + - image: "ghcr.io/flyteorg/flyteconsole:v0.20.0" + imagePullPolicy: "IfNotPresent" + name: flyteconsole + envFrom: + - configMapRef: + name: flyte-console-config + ports: + - containerPort: 8080 + resources: + limits: + cpu: 500m + memory: 250Mi + requests: + cpu: 10m + memory: 50Mi + volumeMounts: + - mountPath: /srv/flyte + name: shared-data + volumes: + - emptyDir: {} + name: shared-data +--- +# Source: flyte/templates/datacatalog/deployment.yaml +apiVersion: apps/v1 +kind: Deployment +metadata: + name: datacatalog + namespace: default + labels: + app.kubernetes.io/name: datacatalog + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm +spec: + replicas: 1 + selector: + matchLabels: + app.kubernetes.io/name: datacatalog + app.kubernetes.io/instance: flyte + template: + metadata: + annotations: + configChecksum: "4077c7da1c8c7f5b8ef93e421be4b28a7a27405dccb199798d0d9882cf90831" + labels: + app.kubernetes.io/name: datacatalog + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm + spec: + initContainers: + - command: + - datacatalog + - --config + - /etc/datacatalog/config/*.yaml + - migrate + - run + image: "ghcr.io/flyteorg/datacatalog:v0.3.4" + imagePullPolicy: "IfNotPresent" + name: run-migrations + volumeMounts: + + - mountPath: /etc/datacatalog/config + name: config-volume + containers: + - command: + - datacatalog + - --config + - /etc/datacatalog/config/*.yaml + - serve + image: "ghcr.io/flyteorg/datacatalog:v0.3.4" + imagePullPolicy: "IfNotPresent" + name: datacatalog + ports: + - containerPort: 8088 + - containerPort: 8089 + resources: + limits: + cpu: 500m + ephemeral-storage: 100Mi + memory: 500Mi + requests: + cpu: 10m + ephemeral-storage: 50Mi + memory: 50Mi + volumeMounts: + + - mountPath: /etc/datacatalog/config + name: config-volume + serviceAccountName: datacatalog + volumes: + + - emptyDir: {} + name: shared-data + - configMap: + name: datacatalog-config + name: config-volume +--- +# Source: flyte/templates/minio/deployment.yaml +apiVersion: apps/v1 +kind: Deployment +metadata: + name: minio + namespace: default + labels: + app.kubernetes.io/name: minio + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm +spec: + replicas: 1 + selector: + matchLabels: + app.kubernetes.io/name: minio + app.kubernetes.io/instance: flyte + template: + metadata: + labels: + app.kubernetes.io/name: minio + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm + spec: + containers: + - image: "minio/minio:RELEASE.2020-12-16T05-05-17Z" + imagePullPolicy: "IfNotPresent" + name: minio + args: + - server + - /data + env: + - name: MINIO_ACCESS_KEY + value: minio + - name: MINIO_SECRET_KEY + value: miniostorage + ports: + - containerPort: 9000 + name: minio + resources: + limits: + cpu: 200m + memory: 512Mi + requests: + cpu: 10m + memory: 128Mi + volumeMounts: + - name: minio-storage + mountPath: /var/lib/minioql/data + volumes: + - name: minio-storage + emptyDir: {} +--- +# Source: flyte/templates/postgres/deployment.yaml +apiVersion: apps/v1 +kind: Deployment +metadata: + name: postgres + namespace: default + labels: + app.kubernetes.io/name: postgres + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm +spec: + replicas: 1 + selector: + matchLabels: + app.kubernetes.io/name: postgres + app.kubernetes.io/instance: flyte + template: + metadata: + labels: + app.kubernetes.io/name: postgres + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm + spec: + containers: + - image: "postgres:10.16-alpine" + imagePullPolicy: "IfNotPresent" + name: postgres + env: + - name: POSTGRES_HOST_AUTH_METHOD + value: trust + ports: + - containerPort: 5432 + name: postgres + resources: + limits: + cpu: 1000m + memory: 512Mi + requests: + cpu: 10m + memory: 128Mi + volumeMounts: + - name: postgres-storage + mountPath: /var/lib/postgresql/data + volumes: + - name: postgres-storage + emptyDir: {} +--- +# Source: flyte/templates/propeller/deployment.yaml +apiVersion: apps/v1 +kind: Deployment +metadata: + name: flytepropeller + namespace: default + labels: + app.kubernetes.io/name: flytepropeller + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm +spec: + replicas: 1 + selector: + matchLabels: + app.kubernetes.io/name: flytepropeller + app.kubernetes.io/instance: flyte + template: + metadata: + annotations: + configChecksum: "911c7db31403549038457a5557608a310f4353ccf6e3fa5d0563e81740f11c1" + labels: + app.kubernetes.io/name: flytepropeller + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm + spec: + containers: + - command: + - flytepropeller + - --config + - /etc/flyte/config/*.yaml + env: + - name: POD_NAME + valueFrom: + fieldRef: + fieldPath: metadata.name + image: "ghcr.io/flyteorg/flytepropeller:v0.10.10" + imagePullPolicy: "IfNotPresent" + name: flytepropeller + ports: + - containerPort: 10254 + resources: + limits: + cpu: 200m + ephemeral-storage: 100Mi + memory: 200Mi + requests: + cpu: 10m + ephemeral-storage: 50Mi + memory: 50Mi + volumeMounts: + - name: config-volume + mountPath: /etc/flyte/config + - name: auth + mountPath: /etc/secrets/ + serviceAccountName: flytepropeller + volumes: + - configMap: + name: flyte-propeller-config + name: config-volume + - name: auth + secret: + secretName: flyte-propeller-auth +--- +# Source: flyte/templates/propeller/webhook.yaml +# Create the actual deployment +apiVersion: apps/v1 +kind: Deployment +metadata: + name: flyte-pod-webhook + namespace: default + labels: + app: flyte-pod-webhook +spec: + selector: + matchLabels: + app: flyte-pod-webhook + template: + metadata: + labels: + app: flyte-pod-webhook + app.kubernetes.io/name: flyte-pod-webhook + app.kubernetes.io/version: v0.10.10 + annotations: + configChecksum: "911c7db31403549038457a5557608a310f4353ccf6e3fa5d0563e81740f11c1" + spec: + serviceAccountName: flyte-pod-webhook + initContainers: + - name: generate-secrets + image: "ghcr.io/flyteorg/flytepropeller:v0.10.10" + imagePullPolicy: "IfNotPresent" + command: + - flytepropeller + args: + - webhook + - init-certs + - --config + - /etc/flyte/config/*.yaml + env: + - name: POD_NAME + valueFrom: + fieldRef: + fieldPath: metadata.name + - name: POD_NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace + volumeMounts: + - name: config-volume + mountPath: /etc/flyte/config + containers: + - name: webhook + image: "ghcr.io/flyteorg/flytepropeller:v0.10.10" + imagePullPolicy: "IfNotPresent" + command: + - flytepropeller + args: + - webhook + - --config + - /etc/flyte/config/*.yaml + env: + - name: POD_NAME + valueFrom: + fieldRef: + fieldPath: metadata.name + - name: POD_NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace + volumeMounts: + - name: config-volume + mountPath: /etc/flyte/config + readOnly: true + - name: webhook-certs + mountPath: /etc/webhook/certs + readOnly: true + volumes: + - name: config-volume + configMap: + name: flyte-propeller-config + - name: webhook-certs + secret: + secretName: flyte-pod-webhook +--- +# Source: flyte/templates/admin/cronjob.yaml +apiVersion: batch/v1beta1 +kind: CronJob +metadata: + name: syncresources + namespace: default + labels: + app.kubernetes.io/name: flyteadmin + app.kubernetes.io/instance: flyte + helm.sh/chart: flyte-0.2.0 + app.kubernetes.io/managed-by: Helm +spec: + schedule: '*/1 * * * *' + jobTemplate: + spec: + template: + spec: + containers: + - command: + - flyteadmin + - --config + - /etc/flyte/config/*.yaml + - clusterresource + - sync + image: "ghcr.io/flyteorg/flyteadmin:v0.5.7" + imagePullPolicy: "IfNotPresent" + name: sync-cluster-resources + volumeMounts: + + - mountPath: /etc/flyte/clusterresource/templates + name: resource-templates + - mountPath: /etc/flyte/config + name: config-volume + restartPolicy: OnFailure + serviceAccountName: flyteadmin + volumes: + + - configMap: + name: clusterresource-template + name: resource-templates + - configMap: + name: flyte-admin-config + name: config-volume +--- +# Source: flyte/templates/common/ingress.yaml +apiVersion: networking.k8s.io/v1beta1 +kind: Ingress +metadata: + name: flyte + namespace: default +spec: + rules: + - http: + paths: + # This is useful only for frontend development + - path: /__webpack_hmr + pathType: ImplementationSpecific + backend: + serviceName: flyteconsole + servicePort: 80 + # Port 87 in FlyteAdmin maps to the redoc container. + - path: /openapi + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 87 + # NOTE: If you change this, you must update the BASE_URL value in flyteconsole.yaml + - path: /console + pathType: ImplementationSpecific + backend: + serviceName: flyteconsole + servicePort: 80 + - path: /console/* + pathType: ImplementationSpecific + backend: + serviceName: flyteconsole + servicePort: 80 + - path: /api + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 80 + - path: /api/* + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 80 + - path: /healthcheck + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 80 + - path: /v1/* + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 80 + # Port 87 in FlyteAdmin maps to the redoc container. + - path: /openapi/* + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 80 + - path: /.well-known + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 80 + - path: /.well-known/* + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 80 + - path: /login + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 80 + - path: /login/* + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 80 + - path: /logout + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 80 + - path: /logout/* + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 80 + - path: /callback + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 80 + - path: /callback/* + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 80 + - path: /me + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 80 + - path: /config + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 80 + - path: /config/* + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 80 + # NOTE: Port 81 in flyteadmin is the GRPC server port for FlyteAdmin. + - path: /flyteidl.service.AdminService + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 81 + - path: /flyteidl.service.AdminService/* + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 81 + - path: /flyteidl.service.AuthMetadataService + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 81 + - path: /flyteidl.service.AuthMetadataService/* + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 81 + - path: /flyteidl.service.IdentityService + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 81 + - path: /flyteidl.service.IdentityService/* + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 81 +--- +# Source: flyte/charts/kubernetes-dashboard/templates/clusterrole-readonly.yaml +# Copyright 2017 The Kubernetes Authors. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +--- +# Source: flyte/charts/kubernetes-dashboard/templates/clusterrolebinding-readonly.yaml +# Copyright 2017 The Kubernetes Authors. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +--- +# Source: flyte/charts/kubernetes-dashboard/templates/ingress.yaml +# Copyright 2017 The Kubernetes Authors. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +--- +# Source: flyte/charts/kubernetes-dashboard/templates/networkpolicy.yaml +# Copyright 2017 The Kubernetes Authors. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +--- +# Source: flyte/charts/kubernetes-dashboard/templates/pdb.yaml +# Copyright 2017 The Kubernetes Authors. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +--- +# Source: flyte/charts/kubernetes-dashboard/templates/psp.yaml +# Copyright 2017 The Kubernetes Authors. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +--- +# Source: flyte/charts/contour/templates/certgen/serviceaccount.yaml +apiVersion: v1 +kind: ServiceAccount +metadata: + name: flyte-contour-contour-certgen + namespace: default + annotations: + "helm.sh/hook": "pre-install,pre-upgrade" + "helm.sh/hook-delete-policy": before-hook-creation,hook-succeeded + labels: + app.kubernetes.io/name: contour + helm.sh/chart: contour-4.1.2 + app.kubernetes.io/instance: flyte + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/component: contour-certgen +--- +# Source: flyte/charts/contour/templates/certgen/rbac.yaml +apiVersion: rbac.authorization.k8s.io/v1 +kind: Role +metadata: + name: flyte-contour-contour-certgen + namespace: default + annotations: + "helm.sh/hook": "pre-install,pre-upgrade" + "helm.sh/hook-delete-policy": before-hook-creation,hook-succeeded + labels: + app.kubernetes.io/name: contour + helm.sh/chart: contour-4.1.2 + app.kubernetes.io/instance: flyte + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/component: contour-certgen +rules: + - apiGroups: + - "" + resources: + - secrets + verbs: + - create + - update +--- +# Source: flyte/charts/contour/templates/certgen/rbac.yaml +apiVersion: rbac.authorization.k8s.io/v1 +kind: RoleBinding +metadata: + name: flyte-contour-contour-certgen + namespace: default + annotations: + "helm.sh/hook": "pre-install,pre-upgrade" + "helm.sh/hook-delete-policy": before-hook-creation,hook-succeeded + labels: + app.kubernetes.io/name: contour + helm.sh/chart: contour-4.1.2 + app.kubernetes.io/instance: flyte + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/component: contour-certgen +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: Role + name: flyte-contour-contour-certgen +subjects: + - kind: ServiceAccount + name: flyte-contour-contour-certgen +--- +# Source: flyte/charts/contour/templates/certgen/job.yaml +apiVersion: batch/v1 +kind: Job +metadata: + name: flyte-contour-contour-certgen + namespace: default + annotations: + "helm.sh/hook": "pre-install,pre-upgrade" + "helm.sh/hook-weight": "1" + "helm.sh/hook-delete-policy": before-hook-creation,hook-succeeded + labels: + app.kubernetes.io/name: contour + helm.sh/chart: contour-4.1.2 + app.kubernetes.io/instance: flyte + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/component: contour-certgen +spec: + ttlSecondsAfterFinished: 0 + template: + metadata: + labels: + app.kubernetes.io/name: contour + helm.sh/chart: contour-4.1.2 + app.kubernetes.io/instance: flyte + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/component: contour-certgen + spec: + + containers: + - name: contour + image: docker.io/bitnami/contour:1.12.0-debian-10-r0 + imagePullPolicy: IfNotPresent + command: + - contour + args: + - certgen + - --kube + - --incluster + - --overwrite + - --secrets-format=compact + - --namespace=$(CONTOUR_NAMESPACE) + env: + - name: CONTOUR_NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace + resources: + limits: + cpu: 100m + memory: 100Mi + requests: + cpu: 10m + memory: 50Mi + restartPolicy: Never + serviceAccountName: flyte-contour-contour-certgen + securityContext: + runAsUser: 1001 + runAsGroup: 1001 + fsGroup: + runAsNonRoot: true + parallelism: 1 + completions: 1 + backoffLimit: 1 diff --git a/doc-requirements.txt b/doc-requirements.txt index bc6b2635d6..8ecd8c81c2 100644 --- a/doc-requirements.txt +++ b/doc-requirements.txt @@ -67,8 +67,11 @@ sphinx-copybutton==0.3.1 # via -r doc-requirements.in sphinx-fontawesome==0.0.6 # via -r doc-requirements.in +<<<<<<< HEAD +======= sphinx-issues==1.2.0 # via -r doc-requirements.in +>>>>>>> master sphinx-prompt==1.4.0 # via -r doc-requirements.in sphinx-tabs==2.1.0 diff --git a/helm/.helmignore b/helm/.helmignore new file mode 100644 index 0000000000..0e8a0eb36f --- /dev/null +++ b/helm/.helmignore @@ -0,0 +1,23 @@ +# Patterns to ignore when building packages. +# This supports shell glob matching, relative path matching, and +# negation (prefixed with !). Only one pattern per line. +.DS_Store +# Common VCS dirs +.git/ +.gitignore +.bzr/ +.bzrignore +.hg/ +.hgignore +.svn/ +# Common backup files +*.swp +*.bak +*.tmp +*.orig +*~ +# Various IDEs +.project +.idea/ +*.tmproj +.vscode/ diff --git a/helm/Chart.yaml b/helm/Chart.yaml new file mode 100644 index 0000000000..b66ba74db3 --- /dev/null +++ b/helm/Chart.yaml @@ -0,0 +1,20 @@ +apiVersion: v2 +name: flyte +description: A Helm chart for Flyte +type: application +version: 0.2.0 +appVersion: 0.8.0 +dependencies: + - name: contour + version: 4.1.2 + repository: https://charts.bitnami.com/bitnami + condition: contour.enabled + - name: spark-operator + alias: sparkoperator + version: 1.0.6 + repository: https://googlecloudplatform.github.io/spark-on-k8s-operator + condition: spark.enabled + - name: kubernetes-dashboard + version: 4.0.2 + repository: https://kubernetes.github.io/dashboard/ + condition: kubernetes-dashboard.enabled diff --git a/helm/README.md b/helm/README.md new file mode 100644 index 0000000000..76f70a0cd3 --- /dev/null +++ b/helm/README.md @@ -0,0 +1,232 @@ +# flyte + +![Version: 0.2.0](https://img.shields.io/badge/Version-0.2.0-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: 0.8.0](https://img.shields.io/badge/AppVersion-0.8.0-informational?style=flat-square) + +A Helm chart for Flyte + +## Requirements + +| Repository | Name | Version | +|------------|------|---------| +| https://charts.bitnami.com/bitnami | contour | 4.1.2 | +| https://googlecloudplatform.github.io/spark-on-k8s-operator | sparkoperator(spark-operator) | 1.0.6 | +| https://kubernetes.github.io/dashboard/ | kubernetes-dashboard | 4.0.2 | + +### SANDBOX INSTALLATION: +- [Install helm 3](https://helm.sh/docs/intro/install/) +- Fetch chart dependencies `helm dep up` +- Install Flyte sandbox: + +```bash +cd helm +helm install -n flyte -f values-sandbox.yaml --create-namespace flyte . +``` + +Customize your installation by changing settings in `values-sandbox.yaml`. +You can use the helm diff plugin to review any value changes you've made to your values: + +```bash +helm plugin install https://github.com/databus23/helm-diff +helm diff upgrade -f values-sandbox.yaml flyte . +``` + +Then apply your changes: +```bash +helm upgrade -f values-sandbox.yaml flyte . +``` + +#### Alternative: Generate raw kubernetes yaml with helm template +- `helm template --name-template=flyte-sandbox . -n flyte -f values-sandbox.yaml > flyte_generated_sandbox.yaml` +- Deploy the manifest `kubectl apply -f flyte_generated_sandbox.yaml` + +- When all pods are running - run end2end tests: `kubectl apply -f ../end2end/tests/endtoend.yaml` +- Get flyte host `minikube service contour -n heptio-contour --url`. And then visit `http:///console` + +### CONFIGURATION NOTES: +- The docker images, their tags and other default parameters are configured in `values.yaml` file. +- Each Flyte installation type should have separate `values-*.yaml` file: for sandbox, EKS and etc. The configuration in `values.yaml` and the choosen config `values-*.yaml` are merged when generating the deployment manifest. +- The configuration in `values-sandbox.yaml` is ready for installation in minikube. But `values-eks.yaml` should be edited before installation: s3 bucket, RDS hosts, iam roles, secrets and etc need to be modified. + +## Values + +| Key | Type | Default | Description | +|-----|------|---------|-------------| +| cluster_resource_manager | object | `{"config":{"cluster_resources":{"customData":[{"production":[{"projectQuotaCpu":{"value":"5"}},{"projectQuotaMemory":{"value":"4000Mi"}}]},{"staging":[{"projectQuotaCpu":{"value":"2"}},{"projectQuotaMemory":{"value":"3000Mi"}}]},{"development":[{"projectQuotaCpu":{"value":"4"}},{"projectQuotaMemory":{"value":"3000Mi"}}]}],"refresh":"5m","templatePath":"/etc/flyte/clusterresource/templates"}},"enabled":true,"templates":[{"key":"aa_namespace","value":"apiVersion: v1\nkind: Namespace\nmetadata:\n name: {{ namespace }}\nspec:\n finalizers:\n - kubernetes\n"},{"key":"ab_project_resource_quota","value":"apiVersion: v1\nkind: ResourceQuota\nmetadata:\n name: project-quota\n namespace: {{ namespace }}\nspec:\n hard:\n limits.cpu: {{ projectQuotaCpu }}\n limits.memory: {{ projectQuotaMemory }}\n"}]}` | Configuration for the Cluster resource manager component. This is an optional component, that enables automatic cluster configuration. This is useful to set default quotas, manage namespaces etc that map to a project/domain | +| cluster_resource_manager.config | object | `{"cluster_resources":{"customData":[{"production":[{"projectQuotaCpu":{"value":"5"}},{"projectQuotaMemory":{"value":"4000Mi"}}]},{"staging":[{"projectQuotaCpu":{"value":"2"}},{"projectQuotaMemory":{"value":"3000Mi"}}]},{"development":[{"projectQuotaCpu":{"value":"4"}},{"projectQuotaMemory":{"value":"3000Mi"}}]}],"refresh":"5m","templatePath":"/etc/flyte/clusterresource/templates"}}` | Configmap for ClusterResource parameters | +| cluster_resource_manager.config.cluster_resources | object | `{"customData":[{"production":[{"projectQuotaCpu":{"value":"5"}},{"projectQuotaMemory":{"value":"4000Mi"}}]},{"staging":[{"projectQuotaCpu":{"value":"2"}},{"projectQuotaMemory":{"value":"3000Mi"}}]},{"development":[{"projectQuotaCpu":{"value":"4"}},{"projectQuotaMemory":{"value":"3000Mi"}}]}],"refresh":"5m","templatePath":"/etc/flyte/clusterresource/templates"}` | ClusterResource parameters Refer to the [structure](https://pkg.go.dev/github.com/lyft/flyteadmin@v0.3.37/pkg/runtime/interfaces#ClusterResourceConfig) to customize. | +| cluster_resource_manager.enabled | bool | `true` | Enables the Cluster resource manager component | +| cluster_resource_manager.templates | list | `[{"key":"aa_namespace","value":"apiVersion: v1\nkind: Namespace\nmetadata:\n name: {{ namespace }}\nspec:\n finalizers:\n - kubernetes\n"},{"key":"ab_project_resource_quota","value":"apiVersion: v1\nkind: ResourceQuota\nmetadata:\n name: project-quota\n namespace: {{ namespace }}\nspec:\n hard:\n limits.cpu: {{ projectQuotaCpu }}\n limits.memory: {{ projectQuotaMemory }}\n"}]` | Resource templates that should be applied | +| cluster_resource_manager.templates[0] | object | `{"key":"aa_namespace","value":"apiVersion: v1\nkind: Namespace\nmetadata:\n name: {{ namespace }}\nspec:\n finalizers:\n - kubernetes\n"}` | Template for namespaces resources | +| common.databaseSecret.name | string | `""` | Specify name of K8s Secret which contains Database password. Leave it empty if you don't need this Secret | +| common.databaseSecret.secretManifest | object | `{}` | Specify your Secret (with sensitive data) or pseudo-manifest (without sensitive data). See https://github.com/godaddy/kubernetes-external-secrets | +| common.flyteNamespaceTemplate.enabled | bool | `false` | | +| common.ingress.albSSLRedirect | bool | `false` | | +| common.ingress.annotations | object | `{}` | | +| common.ingress.enabled | bool | `true` | | +| common.ingress.separateGrpcIngress | bool | `false` | | +| common.ingress.separateGrpcIngressAnnotations."nginx.ingress.kubernetes.io/backend-protocol" | string | `"GRPC"` | | +| common.ingress.tls.enabled | bool | `false` | | +| common.ingress.webpackHMR | bool | `false` | | +| configmap.admin | object | `{"admin":{"clientId":"flytepropeller","clientSecretLocation":"/etc/secrets/client_secret","endpoint":"flyteadmin:81","insecure":true},"event":{"capacity":1000,"rate":500,"type":"admin"}}` | Admin Client configuration [structure](https://pkg.go.dev/github.com/flyteorg/flytepropeller/pkg/controller/nodes/subworkflow/launchplan#AdminConfig) | +| configmap.adminServer | object | `{"auth":{"appAuth":{"thirdPartyConfig":{"flyteClient":{"clientId":"flytectl","redirectUri":"https://localhost:53593/callback","scopes":["offline","all"]}}},"authorizedUris":["https://localhost:30081","http://flyteadmin:80","http://flyteadmin.flyte.svc.cluster.local:80"],"userAuth":{"openId":{"baseUrl":"https://accounts.google.com","clientId":"657465813211-6eog7ek7li5k7i7fvgv2921075063hpe.apps.googleusercontent.com","scopes":["profile","openid"]}}},"flyteadmin":{"eventVersion":1,"metadataStoragePrefix":["metadata","admin"],"metricsScope":"flyte:","profilerPort":10254,"roleNameKey":"iam.amazonaws.com/role","testing":{"host":"http://flyteadmin"}},"server":{"grpcPort":8089,"httpPort":8088,"security":{"allowCors":true,"allowedHeaders":["Content-Type","flyte-authorization"],"allowedOrigins":["*"],"secure":false,"useAuth":false}}}` | FlyteAdmin server configuration | +| configmap.adminServer.auth | object | `{"appAuth":{"thirdPartyConfig":{"flyteClient":{"clientId":"flytectl","redirectUri":"https://localhost:53593/callback","scopes":["offline","all"]}}},"authorizedUris":["https://localhost:30081","http://flyteadmin:80","http://flyteadmin.flyte.svc.cluster.local:80"],"userAuth":{"openId":{"baseUrl":"https://accounts.google.com","clientId":"657465813211-6eog7ek7li5k7i7fvgv2921075063hpe.apps.googleusercontent.com","scopes":["profile","openid"]}}}` | Authentication configuration | +| configmap.adminServer.server.security.secure | bool | `false` | Controls whether to serve requests over SSL/TLS. | +| configmap.adminServer.server.security.useAuth | bool | `false` | Controls whether to enforce authentication. Follow the guide in https://docs.flyte.org/ on how to setup authentication. | +| configmap.catalog | object | `{"catalog-cache":{"endpoint":"datacatalog:89","insecure":true,"type":"datacatalog"}}` | Catalog Client configuration [structure](https://pkg.go.dev/github.com/flyteorg/flytepropeller/pkg/controller/nodes/task/catalog#Config) Additional advanced Catalog configuration [here](https://pkg.go.dev/github.com/lyft/flyteplugins/go/tasks/pluginmachinery/catalog#Config) | +| configmap.console | object | `{"BASE_URL":"/console","CONFIG_DIR":"/etc/flyte/config","DISABLE_AUTH":"1"}` | Configuration for Flyte console UI | +| configmap.copilot | object | `{"plugins":{"k8s":{"co-pilot":{"image":"ghcr.io/lyft/flyteplugins/flytecopilot:dc4bdbd61cac88a39a5ff43e40f026bdbc2c78a2","name":"flyte-copilot-","start-timeout":"30s"}}}}` | Copilot configuration | +| configmap.copilot.plugins.k8s.co-pilot | object | `{"image":"ghcr.io/lyft/flyteplugins/flytecopilot:dc4bdbd61cac88a39a5ff43e40f026bdbc2c78a2","name":"flyte-copilot-","start-timeout":"30s"}` | Structure documented [here](https://pkg.go.dev/github.com/lyft/flyteplugins@v0.5.28/go/tasks/pluginmachinery/flytek8s/config#FlyteCoPilotConfig) | +| configmap.core | object | `{"propeller":{"downstream-eval-duration":"30s","enable-admin-launcher":true,"leader-election":{"enabled":true,"lease-duration":"15s","lock-config-map":{"name":"propeller-leader","namespace":"flyte"},"renew-deadline":"10s","retry-period":"2s"},"limit-namespace":"all","max-workflow-retries":30,"metadata-prefix":"metadata/propeller","metrics-prefix":"flyte","prof-port":10254,"queue":{"batch-size":-1,"batching-interval":"2s","queue":{"base-delay":"5s","capacity":1000,"max-delay":"120s","rate":100,"type":"maxof"},"sub-queue":{"capacity":100,"rate":10,"type":"bucket"},"type":"batch"},"rawoutput-prefix":"s3://my-s3-bucket/","workers":4,"workflow-reeval-duration":"30s"},"webhook":{"certDir":"/etc/webhook/certs","serviceName":"flyte-pod-webhook"}}` | Core propeller configuration | +| configmap.core.propeller | object | `{"downstream-eval-duration":"30s","enable-admin-launcher":true,"leader-election":{"enabled":true,"lease-duration":"15s","lock-config-map":{"name":"propeller-leader","namespace":"flyte"},"renew-deadline":"10s","retry-period":"2s"},"limit-namespace":"all","max-workflow-retries":30,"metadata-prefix":"metadata/propeller","metrics-prefix":"flyte","prof-port":10254,"queue":{"batch-size":-1,"batching-interval":"2s","queue":{"base-delay":"5s","capacity":1000,"max-delay":"120s","rate":100,"type":"maxof"},"sub-queue":{"capacity":100,"rate":10,"type":"bucket"},"type":"batch"},"rawoutput-prefix":"s3://my-s3-bucket/","workers":4,"workflow-reeval-duration":"30s"}` | follows the structure specified [here](https://pkg.go.dev/github.com/flyteorg/flytepropeller/pkg/controller/config). | +| configmap.datacatalogServer | object | `{"application":{"grpcPort":8089,"grpcServerReflection":true,"httpPort":8080},"datacatalog":{"metrics-scope":"datacatalog","profiler-port":10254,"storage-prefix":"metadata/datacatalog"}}` | Datacatalog server config | +| configmap.domain | object | `{"domains":[{"id":"development","name":"development"},{"id":"staging","name":"staging"},{"id":"production","name":"production"}]}` | Domains configuration for Flyte projects. This enables the specified number of domains across all projects in Flyte. | +| configmap.enabled_plugins.tasks | object | `{"task-plugins":{"default-for-task-types":{"container":"container","container_array":"k8s-array","sidecar":"sidecar"},"enabled-plugins":["container","sidecar","k8s-array"]}}` | Tasks specific configuration [structure](https://pkg.go.dev/github.com/flyteorg/flytepropeller/pkg/controller/nodes/task/config#GetConfig) | +| configmap.enabled_plugins.tasks.task-plugins | object | `{"default-for-task-types":{"container":"container","container_array":"k8s-array","sidecar":"sidecar"},"enabled-plugins":["container","sidecar","k8s-array"]}` | Plugins configuration, [structure](https://pkg.go.dev/github.com/flyteorg/flytepropeller/pkg/controller/nodes/task/config#TaskPluginConfig) | +| configmap.enabled_plugins.tasks.task-plugins.enabled-plugins | list | `["container","sidecar","k8s-array"]` | [Enabled Plugins](https://pkg.go.dev/github.com/lyft/flyteplugins/go/tasks/config#Config). Enable sagemaker*, athena if you install the backend plugins | +| configmap.k8s | object | `{"plugins":{"k8s":{"default-cpus":"100m","default-env-vars":[],"default-memory":"100Mi"}}}` | Kubernetes specific Flyte configuration | +| configmap.k8s.plugins.k8s | object | `{"default-cpus":"100m","default-env-vars":[],"default-memory":"100Mi"}` | Configuration section for all K8s specific plugins [Configuration structure](https://pkg.go.dev/github.com/lyft/flyteplugins/go/tasks/pluginmachinery/flytek8s/config) | +| configmap.logger | object | `{"logger":{"level":4,"show-source":true}}` | Logger configuration | +| configmap.remoteData.remoteData.region | string | `"us-east-1"` | | +| configmap.remoteData.remoteData.scheme | string | `"local"` | | +| configmap.remoteData.remoteData.signedUrls.durationMinutes | int | `3` | | +| configmap.resource_manager | object | `{"propeller":{"resourcemanager":{"redis":{"hostKey":"mypassword","hostPath":"redis-resource-manager:6379"},"resourceMaxQuota":10000,"type":"redis"}}}` | Resource manager configuration | +| configmap.resource_manager.propeller | object | `{"resourcemanager":{"redis":{"hostKey":"mypassword","hostPath":"redis-resource-manager:6379"},"resourceMaxQuota":10000,"type":"redis"}}` | resource manager configuration | +| configmap.task_logs | object | `{"plugins":{"logs":{"cloudwatch-enabled":false,"kubernetes-enabled":false}}}` | Section that configures how the Task logs are displayed on the UI. This has to be changed based on your actual logging provider. Refer to [structure](https://pkg.go.dev/github.com/lyft/flyteplugins/go/tasks/logs#LogConfig) to understand how to configure various logging engines | +| configmap.task_logs.plugins.logs.cloudwatch-enabled | bool | `false` | One option is to enable cloudwatch logging for EKS, update the region and log group accordingly | +| configmap.task_resource_defaults | object | `{"task_resources":{"defaults":{"cpu":"100m","memory":"100Mi","storage":"5Mi"},"limits":{"cpu":2,"gpu":1,"memory":"8Gi","storage":"20Mi"}}}` | Task default resources configuration Refer to the full [structure](https://pkg.go.dev/github.com/lyft/flyteadmin@v0.3.37/pkg/runtime/interfaces#TaskResourceConfiguration). | +| configmap.task_resource_defaults.task_resources | object | `{"defaults":{"cpu":"100m","memory":"100Mi","storage":"5Mi"},"limits":{"cpu":2,"gpu":1,"memory":"8Gi","storage":"20Mi"}}` | Task default resources parameters | +| contour.affinity | object | `{}` | affinity for Contour deployment | +| contour.contour.resources | object | `{"limits":{"cpu":"100m","memory":"100Mi"},"requests":{"cpu":"10m","memory":"50Mi"}}` | Default resources requests and limits for Contour | +| contour.enabled | bool | `true` | | +| contour.envoy.resources | object | `{"limits":{"cpu":"100m","memory":"100Mi"},"requests":{"cpu":"10m","memory":"50Mi"}}` | Default resources requests and limits for Envoy | +| contour.nodeSelector | object | `{}` | nodeSelector for Contour deployment | +| contour.podAnnotations | object | `{}` | Annotations for Contour pods | +| contour.replicaCount | int | `1` | Replicas count for Contour deployment | +| contour.serviceAccountAnnotations | object | `{}` | Annotations for ServiceAccount attached to Contour pods | +| contour.tolerations | list | `[]` | tolerations for Contour deployment | +| datacatalog.affinity | object | `{}` | affinity for Datacatalog deployment | +| datacatalog.configPath | string | `"/etc/datacatalog/config/*.yaml"` | Default regex string for searching configuration files | +| datacatalog.image.pullPolicy | string | `"IfNotPresent"` | | +| datacatalog.image.repository | string | `"ghcr.io/flyteorg/datacatalog"` | Docker image for Datacatalog deployment | +| datacatalog.image.tag | string | `"v0.3.4"` | | +| datacatalog.nodeSelector | object | `{}` | nodeSelector for Datacatalog deployment | +| datacatalog.podAnnotations | object | `{}` | Annotations for Datacatalog pods | +| datacatalog.replicaCount | int | `1` | Replicas count for Datacatalog deployment | +| datacatalog.resources | object | `{"limits":{"cpu":"500m","ephemeral-storage":"100Mi","memory":"500Mi"},"requests":{"cpu":"10m","ephemeral-storage":"50Mi","memory":"50Mi"}}` | Default resources requests and limits for Datacatalog deployment | +| datacatalog.service | object | `{"annotations":{"projectcontour.io/upstream-protocol.h2c":"grpc"},"type":"NodePort"}` | Service settings for Datacatalog | +| datacatalog.serviceAccount | object | `{"annotations":{},"create":true,"imagePullSecrets":{}}` | Configuration for service accounts for Datacatalog | +| datacatalog.serviceAccount.annotations | object | `{}` | Annotations for ServiceAccount attached to Datacatalog pods | +| datacatalog.serviceAccount.create | bool | `true` | Should a service account be created for Datacatalog | +| datacatalog.serviceAccount.imagePullSecrets | object | `{}` | ImapgePullSecrets to automatically assign to the service account | +| datacatalog.tolerations | list | `[]` | tolerations for Datacatalog deployment | +| db.database | object | `{}` | | +| flyteadmin.affinity | object | `{}` | affinity for Flyteadmin deployment | +| flyteadmin.configPath | string | `"/etc/flyte/config/*.yaml"` | Default regex string for searching configuration files | +| flyteadmin.image.pullPolicy | string | `"IfNotPresent"` | | +| flyteadmin.image.repository | string | `"ghcr.io/flyteorg/flyteadmin"` | Docker image for Flyteadmin deployment | +| flyteadmin.image.tag | string | `"v0.5.7"` | | +| flyteadmin.nodeSelector | object | `{}` | nodeSelector for Flyteadmin deployment | +| flyteadmin.podAnnotations | object | `{}` | Annotations for Flyteadmin pods | +| flyteadmin.replicaCount | int | `1` | Replicas count for Flyteadmin deployment | +| flyteadmin.resources | object | `{"limits":{"cpu":"250m","ephemeral-storage":"100Mi","memory":"500Mi"},"requests":{"cpu":"10m","ephemeral-storage":"50Mi","memory":"50Mi"}}` | Default resources requests and limits for Flyteadmin deployment | +| flyteadmin.service | object | `{"annotations":{"projectcontour.io/upstream-protocol.h2c":"grpc"},"loadBalancerSourceRanges":[],"type":"ClusterIP"}` | Service settings for Flyteadmin | +| flyteadmin.serviceAccount | object | `{"annotations":{},"create":true,"imagePullSecrets":{}}` | Configuration for service accounts for FlyteAdmin | +| flyteadmin.serviceAccount.annotations | object | `{}` | Annotations for ServiceAccount attached to Flyteadmin pods | +| flyteadmin.serviceAccount.create | bool | `true` | Should a service account be created for flyteadmin | +| flyteadmin.serviceAccount.imagePullSecrets | object | `{}` | ImapgePullSecrets to automatically assign to the service account | +| flyteadmin.tolerations | list | `[]` | tolerations for Flyteadmin deployment | +| flyteconsole.affinity | object | `{}` | affinity for Flyteconsole deployment | +| flyteconsole.image.pullPolicy | string | `"IfNotPresent"` | | +| flyteconsole.image.repository | string | `"ghcr.io/flyteorg/flyteconsole"` | Docker image for Flyteconsole deployment | +| flyteconsole.image.tag | string | `"v0.20.0"` | | +| flyteconsole.nodeSelector | object | `{}` | nodeSelector for Flyteconsole deployment | +| flyteconsole.podAnnotations | object | `{}` | Annotations for Flyteconsole pods | +| flyteconsole.replicaCount | int | `1` | Replicas count for Flyteconsole deployment | +| flyteconsole.resources | object | `{"limits":{"cpu":"500m","memory":"250Mi"},"requests":{"cpu":"10m","memory":"50Mi"}}` | Default resources requests and limits for Flyteconsole deployment | +| flyteconsole.service | object | `{"annotations":{},"type":"ClusterIP"}` | Service settings for Flyteconsole | +| flyteconsole.tolerations | list | `[]` | tolerations for Flyteconsole deployment | +| flytepropeller.affinity | object | `{}` | affinity for Flytepropeller deployment | +| flytepropeller.cacheSizeMbs | int | `0` | | +| flytepropeller.configPath | string | `"/etc/flyte/config/*.yaml"` | Default regex string for searching configuration files | +| flytepropeller.image.pullPolicy | string | `"IfNotPresent"` | | +| flytepropeller.image.repository | string | `"ghcr.io/flyteorg/flytepropeller"` | Docker image for Flytepropeller deployment | +| flytepropeller.image.tag | string | `"v0.10.10"` | | +| flytepropeller.nodeSelector | object | `{}` | nodeSelector for Flytepropeller deployment | +| flytepropeller.podAnnotations | object | `{}` | Annotations for Flytepropeller pods | +| flytepropeller.replicaCount | int | `1` | Replicas count for Flytepropeller deployment | +| flytepropeller.resources | object | `{"limits":{"cpu":"200m","ephemeral-storage":"100Mi","memory":"200Mi"},"requests":{"cpu":"10m","ephemeral-storage":"50Mi","memory":"50Mi"}}` | Default resources requests and limits for Flytepropeller deployment | +| flytepropeller.serviceAccount | object | `{"annotations":{},"create":true,"imagePullSecrets":{}}` | Configuration for service accounts for FlytePropeller | +| flytepropeller.serviceAccount.annotations | object | `{}` | Annotations for ServiceAccount attached to FlytePropeller pods | +| flytepropeller.serviceAccount.create | bool | `true` | Should a service account be created for FlytePropeller | +| flytepropeller.serviceAccount.imagePullSecrets | object | `{}` | ImapgePullSecrets to automatically assign to the service account | +| flytepropeller.tolerations | list | `[]` | tolerations for Flytepropeller deployment | +| kubernetes-dashboard.enabled | bool | `false` | | +| minio.affinity | object | `{}` | affinity for Minio deployment | +| minio.enabled | bool | `true` | | +| minio.image.pullPolicy | string | `"IfNotPresent"` | | +| minio.image.repository | string | `"minio/minio"` | Docker image for Minio deployment | +| minio.image.tag | string | `"RELEASE.2020-12-16T05-05-17Z"` | | +| minio.nodeSelector | object | `{}` | nodeSelector for Minio deployment | +| minio.podAnnotations | object | `{}` | Annotations for Minio pods | +| minio.replicaCount | int | `1` | Replicas count for Minio deployment | +| minio.resources | object | `{"limits":{"cpu":"200m","memory":"512Mi"},"requests":{"cpu":"10m","memory":"128Mi"}}` | Default resources requests and limits for Minio deployment | +| minio.service | object | `{"annotations":{},"type":"ClusterIP"}` | Service settings for Minio | +| minio.tolerations | list | `[]` | tolerations for Minio deployment | +| postgres.affinity | object | `{}` | affinity for Postgres deployment | +| postgres.enabled | bool | `true` | | +| postgres.image.pullPolicy | string | `"IfNotPresent"` | | +| postgres.image.repository | string | `"postgres"` | Docker image for Postgres deployment | +| postgres.image.tag | string | `"10.16"` | | +| postgres.nodeSelector | object | `{}` | nodeSelector for Postgres deployment | +| postgres.podAnnotations | object | `{}` | Annotations for Postgres pods | +| postgres.replicaCount | int | `1` | Replicas count for Postgres deployment | +| postgres.resources | object | `{"limits":{"cpu":"1000m","memory":"512Mi"},"requests":{"cpu":"10m","memory":"128Mi"}}` | Default resources requests and limits for Postgres deployment | +| postgres.service | object | `{"annotations":{},"type":"ClusterIP"}` | Service settings for Postgres | +| postgres.tolerations | list | `[]` | tolerations for Postgres deployment | +| pytorchoperator.affinity | object | `{}` | affinity for Pytorchoperator deployment | +| pytorchoperator.enabled | bool | `true` | | +| pytorchoperator.image.pullPolicy | string | `"IfNotPresent"` | | +| pytorchoperator.image.repository | string | `"gcr.io/kubeflow-images-public/pytorch-operator"` | Docker image for Pytorchoperator | +| pytorchoperator.image.tag | string | `"v1.0.0-g047cf0f"` | | +| pytorchoperator.nodeSelector | object | `{}` | nodeSelector for Pytorchoperator deployment | +| pytorchoperator.podAnnotations | object | `{}` | Annotations for Pytorchoperator pods | +| pytorchoperator.replicaCount | int | `1` | Replicas count for Pytorchoperator deployment | +| pytorchoperator.resources | object | `{"limits":{"cpu":"500m","memory":"1000M"},"requests":{"cpu":"10m","memory":"50M"}}` | Default resources requests and limits for Pytorchoperator | +| pytorchoperator.service | object | `{"annotations":{},"type":"ClusterIP"}` | Service settings for Pytorchoperator | +| pytorchoperator.serviceAccountAnnotations | object | `{}` | Annotations for ServiceAccount attached to Pytorchoperator pods | +| pytorchoperator.tolerations | list | `[]` | tolerations for Pytorchoperator deployment | +| redis.affinity | object | `{}` | affinity for Redis Statefulset | +| redis.enabled | bool | `true` | | +| redis.image.pullPolicy | string | `"IfNotPresent"` | | +| redis.image.repository | string | `"docker.io/bitnami/redis"` | Docker image for Redis Statefulset | +| redis.image.tag | string | `"4.0.2-r1"` | | +| redis.nodeSelector | object | `{}` | nodeSelector for Redis Statefulset | +| redis.podAnnotations | object | `{}` | Annotations for Redis pods | +| redis.replicaCount | int | `1` | Replicas count for Redis Statefulset | +| redis.resources | object | `{"limits":{"cpu":"1000m","memory":"1Gi"},"requests":{"cpu":"10m","memory":"50Mi"}}` | Default resources requests and limits for Redis Statefulset | +| redis.service | object | `{"annotations":{},"type":"ClusterIP"}` | Service settings for Redis | +| redis.tolerations | list | `[]` | tolerations for Redis Statefulset | +| sagemaker.enabled | bool | `false` | | +| sagemaker.plugin_config.plugins.sagemaker.region | string | `""` | | +| sagemaker.plugin_config.plugins.sagemaker.roleArn | string | `""` | | +| sparkoperator.enabled | bool | `true` | | +| sparkoperator.image.tag | string | `"v1beta2-1.2.0-3.0.0"` | Docker image for Sparkoperator | +| sparkoperator.plugin_config | object | `{"plugins":{"spark":{"spark-config-default":[{"spark.hadoop.mapreduce.fileoutputcommitter.algorithm.version":"2"},{"spark.kubernetes.allocation.batch.size":"50"},{"spark.hadoop.fs.s3a.acl.default":"BucketOwnerFullControl"},{"spark.hadoop.fs.s3n.impl":"org.apache.hadoop.fs.s3a.S3AFileSystem"},{"spark.hadoop.fs.AbstractFileSystem.s3n.impl":"org.apache.hadoop.fs.s3a.S3A"},{"spark.hadoop.fs.s3.impl":"org.apache.hadoop.fs.s3a.S3AFileSystem"},{"spark.hadoop.fs.AbstractFileSystem.s3.impl":"org.apache.hadoop.fs.s3a.S3A"},{"spark.hadoop.fs.s3a.impl":"org.apache.hadoop.fs.s3a.S3AFileSystem"},{"spark.hadoop.fs.AbstractFileSystem.s3a.impl":"org.apache.hadoop.fs.s3a.S3A"},{"spark.hadoop.fs.s3a.multipart.threshold":"536870912"},{"spark.blacklist.enabled":"true"},{"spark.blacklist.timeout":"5m"},{"spark.task.maxfailures":"8"}]}}}` | Spark plugin configuration | +| sparkoperator.plugin_config.plugins.spark.spark-config-default | list | `[{"spark.hadoop.mapreduce.fileoutputcommitter.algorithm.version":"2"},{"spark.kubernetes.allocation.batch.size":"50"},{"spark.hadoop.fs.s3a.acl.default":"BucketOwnerFullControl"},{"spark.hadoop.fs.s3n.impl":"org.apache.hadoop.fs.s3a.S3AFileSystem"},{"spark.hadoop.fs.AbstractFileSystem.s3n.impl":"org.apache.hadoop.fs.s3a.S3A"},{"spark.hadoop.fs.s3.impl":"org.apache.hadoop.fs.s3a.S3AFileSystem"},{"spark.hadoop.fs.AbstractFileSystem.s3.impl":"org.apache.hadoop.fs.s3a.S3A"},{"spark.hadoop.fs.s3a.impl":"org.apache.hadoop.fs.s3a.S3AFileSystem"},{"spark.hadoop.fs.AbstractFileSystem.s3a.impl":"org.apache.hadoop.fs.s3a.S3A"},{"spark.hadoop.fs.s3a.multipart.threshold":"536870912"},{"spark.blacklist.enabled":"true"},{"spark.blacklist.timeout":"5m"},{"spark.task.maxfailures":"8"}]` | Spark default configuration | +| sparkoperator.replicaCount | int | `1` | Replicas count for Sparkoperator deployment | +| sparkoperator.resources | object | `{"limits":{"cpu":"1000m","memory":"500M"},"requests":{"cpu":"10m","memory":"50M"}}` | Default resources requests and limits for Sparkoperator | +| storage.bucketName | string | `"my-s3-bucket"` | bucketName defines the storage bucket flyte will use. Required for all types except for sandbox. | +| storage.custom | object | `{}` | GCP project ID. Required for storage type gcs. projectId: -- Settings for storage type custom. See https://github:com/graymeta/stow for supported storage providers/settings. | +| storage.gcs | string | `nil` | settings for storage type gcs | +| storage.s3 | object | `{"region":"us-east-1"}` | settings for storage type s3 | +| storage.type | string | `"sandbox"` | Sets the storage type. Supported values are sandbox, s3, gcs and custom. | +| tf_operator.enabled | bool | `false` | | +| webhook.enabled | bool | `true` | enable or disable secrets webhook | +| webhook.service | object | `{"annotations":{"projectcontour.io/upstream-protocol.h2c":"grpc"},"type":"ClusterIP"}` | Service settings for the webhook | +| webhook.serviceAccount | object | `{"annotations":{},"create":true,"imagePullSecrets":{}}` | Configuration for service accounts for the webhook | +| webhook.serviceAccount.annotations | object | `{}` | Annotations for ServiceAccount attached to the webhook | +| webhook.serviceAccount.create | bool | `true` | Should a service account be created for the webhook | +| webhook.serviceAccount.imagePullSecrets | object | `{}` | ImapgePullSecrets to automatically assign to the service account | +| workflow_notifications | object | `{"config":{},"enabled":false}` | **Optional Component** Workflow notifications module is an optional dependency. Flyte uses cloud native pub-sub systems to notify users of various events in their workflows | +| workflow_scheduler.config | object | `{}` | | +| workflow_scheduler.enabled | bool | `false` | | diff --git a/helm/README.md.gotmpl b/helm/README.md.gotmpl new file mode 100644 index 0000000000..abbdb920b1 --- /dev/null +++ b/helm/README.md.gotmpl @@ -0,0 +1,52 @@ +{{ template "chart.header" . }} +{{ template "chart.deprecationWarning" . }} + +{{ template "chart.versionBadge" . }}{{ template "chart.typeBadge" . }}{{ template "chart.appVersionBadge" . }} + +{{ template "chart.description" . }} + +{{ template "chart.homepageLine" . }} + +{{ template "chart.maintainersSection" . }} + +{{ template "chart.sourcesSection" . }} + +{{ template "chart.requirementsSection" . }} + +### SANDBOX INSTALLATION: +- [Install helm 3](https://helm.sh/docs/intro/install/) +- Fetch chart dependencies `helm dep up` +- Install Flyte sandbox: + +```bash +cd helm +helm install -n flyte -f values-sandbox.yaml --create-namespace flyte . +``` + +Customize your installation by changing settings in `values-sandbox.yaml`. +You can use the helm diff plugin to review any value changes you've made to your values: + +```bash +helm plugin install https://github.com/databus23/helm-diff +helm diff upgrade -f values-sandbox.yaml flyte . +``` + +Then apply your changes: +```bash +helm upgrade -f values-sandbox.yaml flyte . +``` + +#### Alternative: Generate raw kubernetes yaml with helm template +- `helm template --name-template=flyte-sandbox . -n flyte -f values-sandbox.yaml > flyte_generated_sandbox.yaml` +- Deploy the manifest `kubectl apply -f flyte_generated_sandbox.yaml` + + +- When all pods are running - run end2end tests: `kubectl apply -f ../end2end/tests/endtoend.yaml` +- Get flyte host `minikube service contour -n heptio-contour --url`. And then visit `http:///console` + +### CONFIGURATION NOTES: +- The docker images, their tags and other default parameters are configured in `values.yaml` file. +- Each Flyte installation type should have separate `values-*.yaml` file: for sandbox, EKS and etc. The configuration in `values.yaml` and the choosen config `values-*.yaml` are merged when generating the deployment manifest. +- The configuration in `values-sandbox.yaml` is ready for installation in minikube. But `values-eks.yaml` should be edited before installation: s3 bucket, RDS hosts, iam roles, secrets and etc need to be modified. + +{{ template "chart.valuesSection" . }} diff --git a/helm/templates/_helpers.tpl b/helm/templates/_helpers.tpl new file mode 100644 index 0000000000..0229d30271 --- /dev/null +++ b/helm/templates/_helpers.tpl @@ -0,0 +1,216 @@ +{{/* vim: set filetype=mustache: */}} + +{{- define "flyte.name" -}} +{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" -}} +{{- end -}} + +{{- define "flyte.chart" -}} +{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" -}} +{{- end -}} + +{{- define "flyte.namespace" -}} +{{- default .Release.Namespace .Values.forceNamespace | trunc 63 | trimSuffix "-" -}} +{{- end -}} + + +{{- define "flyteadmin.name" -}} +flyteadmin +{{- end -}} + +{{- define "flyteadmin.selectorLabels" -}} +app.kubernetes.io/name: {{ template "flyteadmin.name" . }} +app.kubernetes.io/instance: {{ .Release.Name }} +{{- end -}} + +{{- define "flyteadmin.labels" -}} +{{ include "flyteadmin.selectorLabels" . }} +helm.sh/chart: {{ include "flyte.chart" . }} +app.kubernetes.io/managed-by: {{ .Release.Service }} +{{- end -}} + + +{{- define "datacatalog.name" -}} +datacatalog +{{- end -}} + +{{- define "datacatalog.selectorLabels" -}} +app.kubernetes.io/name: {{ template "datacatalog.name" . }} +app.kubernetes.io/instance: {{ .Release.Name }} +{{- end -}} + +{{- define "datacatalog.labels" -}} +{{ include "datacatalog.selectorLabels" . }} +helm.sh/chart: {{ include "flyte.chart" . }} +app.kubernetes.io/managed-by: {{ .Release.Service }} +{{- end -}} + + +{{- define "flytepropeller.name" -}} +flytepropeller +{{- end -}} + +{{- define "flytepropeller.selectorLabels" -}} +app.kubernetes.io/name: {{ template "flytepropeller.name" . }} +app.kubernetes.io/instance: {{ .Release.Name }} +{{- end -}} + +{{- define "flytepropeller.labels" -}} +{{ include "flytepropeller.selectorLabels" . }} +helm.sh/chart: {{ include "flyte.chart" . }} +app.kubernetes.io/managed-by: {{ .Release.Service }} +{{- end -}} + +{{- define "flyte-pod-webhook.name" -}} +flyte-pod-webhook +{{- end -}} + + +{{- define "flyteconsole.name" -}} +flyteconsole +{{- end -}} + +{{- define "flyteconsole.selectorLabels" -}} +app.kubernetes.io/name: {{ template "flyteconsole.name" . }} +app.kubernetes.io/instance: {{ .Release.Name }} +{{- end -}} + +{{- define "flyteconsole.labels" -}} +{{ include "flyteconsole.selectorLabels" . }} +helm.sh/chart: {{ include "flyte.chart" . }} +app.kubernetes.io/managed-by: {{ .Release.Service }} +{{- end -}} + + +{{- define "redis.name" -}} +redis +{{- end -}} + +{{- define "redis.selectorLabels" -}} +app.kubernetes.io/name: {{ template "redis.name" . }} +app.kubernetes.io/instance: {{ .Release.Name }} +{{- end -}} + +{{- define "redis.labels" -}} +{{ include "redis.selectorLabels" . }} +helm.sh/chart: {{ include "flyte.chart" . }} +app.kubernetes.io/managed-by: {{ .Release.Service }} +{{- end -}} + + +{{- define "postgres.name" -}} +postgres +{{- end -}} + +{{- define "postgres.selectorLabels" -}} +app.kubernetes.io/name: {{ template "postgres.name" . }} +app.kubernetes.io/instance: {{ .Release.Name }} +{{- end -}} + +{{- define "postgres.labels" -}} +{{ include "postgres.selectorLabels" . }} +helm.sh/chart: {{ include "flyte.chart" . }} +app.kubernetes.io/managed-by: {{ .Release.Service }} +{{- end -}} + + +{{- define "minio.name" -}} +minio +{{- end -}} + +{{- define "minio.selectorLabels" -}} +app.kubernetes.io/name: {{ template "minio.name" . }} +app.kubernetes.io/instance: {{ .Release.Name }} +{{- end -}} + +{{- define "minio.labels" -}} +{{ include "minio.selectorLabels" . }} +helm.sh/chart: {{ include "flyte.chart" . }} +app.kubernetes.io/managed-by: {{ .Release.Service }} +{{- end -}} + +{{- define "pytorch-operator.name" -}} +pytorch-operator +{{- end -}} + +{{- define "pytorch-operator.namespace" -}} +pytorch-operator +{{- end -}} + +{{- define "pytorch-operator.selectorLabels" -}} +app.kubernetes.io/name: {{ template "pytorch-operator.name" . }} +app.kubernetes.io/instance: {{ .Release.Name }} +{{- end -}} + +{{- define "pytorch-operator.labels" -}} +{{ include "pytorch-operator.selectorLabels" . }} +helm.sh/chart: {{ include "flyte.chart" . }} +app.kubernetes.io/managed-by: {{ .Release.Service }} +{{- end -}} + +# Optional blocks for secret mount + +{{- define "databaseSecret.volume" -}} +{{- with .Values.common.databaseSecret.name -}} +- name: {{ . }} + secret: + secretName: {{ . }} +{{- end }} +{{- end }} + +{{- define "databaseSecret.volumeMount" -}} +{{- with .Values.common.databaseSecret.name -}} +- mountPath: /etc/db + name: {{ . }} +{{- end }} +{{- end }} + +{{- define "storage.base" -}} +storage: +{{- if eq .Values.storage.type "s3" }} + type: s3 + container: {{ .Values.storage.bucketName | quote }} + connection: + auth-type: iam + region: {{ .Values.storage.s3.region }} +{{- else if eq .Values.storage.type "gcs" }} + type: stow + stow: + kind: google + config: + json: "" + project_id: {{ .Values.storage.gcs.projectId }} + scopes: https://www.googleapis.com/auth/devstorage.read_write + container: {{ .Values.storage.bucketName | quote }} +{{- else if eq .Values.storage.type "sandbox" }} + type: minio + container: {{ .Values.storage.bucketName | quote }} + connection: + access-key: minio + auth-type: accesskey + secret-key: miniostorage + disable-ssl: true + endpoint: http://minio.flyte.svc.cluster.local:9000 + region: us-east-1 +{{- else if eq .Values.storage.type "custom" }} +{{- with .Values.storage.custom -}} + {{ toYaml . | nindent 2 }} +{{- end }} +{{- end }} +{{- end }} + +{{- define "storage" -}} +{{ include "storage.base" .}} + limits: + maxDownloadMBs: 10 +{{- end }} + +{{- define "copilot.config" -}} +kind: ConfigMap +apiVersion: v1 +metadata: + name: flyte-data-config + namespace: {{`{{ namespace }}`}} +data: + config.yaml: | {{ include "storage.base" . | nindent 4 }} + enable-multicontainer: true +{{- end }} diff --git a/helm/templates/admin/cluster_resource_configmap.yaml b/helm/templates/admin/cluster_resource_configmap.yaml new file mode 100644 index 0000000000..f5f62f58b5 --- /dev/null +++ b/helm/templates/admin/cluster_resource_configmap.yaml @@ -0,0 +1,13 @@ +{{- if .Values.cluster_resource_manager.enabled }} +apiVersion: v1 +kind: ConfigMap +metadata: + name: clusterresource-template + namespace: {{ template "flyte.namespace" . }} + labels: {{ include "flyteadmin.labels" . | nindent 4 }} +data: +{{- range .Values.cluster_resource_manager.templates }} + {{ .key }}.yaml: | {{ .value | nindent 4 }} +{{- end }} + zz_copilot_config.yaml: | {{ include "copilot.config" . | nindent 4 }} +{{- end }} diff --git a/helm/templates/admin/configmap.yaml b/helm/templates/admin/configmap.yaml new file mode 100644 index 0000000000..504f30c717 --- /dev/null +++ b/helm/templates/admin/configmap.yaml @@ -0,0 +1,41 @@ +apiVersion: v1 +kind: ConfigMap +metadata: + name: flyte-admin-config + namespace: {{ template "flyte.namespace" . }} + labels: {{ include "flyteadmin.labels" . | nindent 4 }} +data: +{{- with .Values.db }} + db.yaml: | {{ toYaml . | nindent 4 }} +{{- end }} +{{- with .Values.configmap.domain }} + domain.yaml: | {{ toYaml . | nindent 4 }} +{{- end }} +{{- with .Values.configmap.logger }} + logger.yaml: | {{ toYaml . | nindent 4 }} +{{- end }} +{{- with .Values.configmap.adminServer }} + server.yaml: | {{ toYaml . | nindent 4 }} +{{- end }} +{{- with .Values.configmap.remoteData }} + remoteData.yaml: | {{ toYaml . | nindent 4 }} +{{- end }} + storage.yaml: | {{ include "storage" . | nindent 4 }} +{{- with .Values.configmap.task_resource_defaults }} + task_resource_defaults.yaml: | {{ toYaml . | nindent 4 }} +{{- end }} +{{- if .Values.workflow_notifications.enabled }} +{{- with .Values.workflow_notifications.config }} + notifications.yaml: | {{ toYaml . | nindent 4 }} +{{- end }} +{{- end }} +{{- if .Values.cluster_resource_manager.enabled }} +{{- with .Values.cluster_resource_manager.config }} + cluster_resources.yaml: | {{ toYaml . | nindent 4 }} +{{- end }} +{{- end }} +{{- if .Values.workflow_scheduler.enabled }} +{{- with .Values.workflow_scheduler.config }} + scheduler.yaml: | {{ toYaml . | nindent 4 }} +{{- end }} +{{- end }} diff --git a/helm/templates/admin/cronjob.yaml b/helm/templates/admin/cronjob.yaml new file mode 100644 index 0000000000..0c13789fba --- /dev/null +++ b/helm/templates/admin/cronjob.yaml @@ -0,0 +1,42 @@ +{{- if .Values.cluster_resource_manager.enabled }} +apiVersion: batch/v1beta1 +kind: CronJob +metadata: + name: syncresources + namespace: {{ template "flyte.namespace" . }} + labels: {{ include "flyteadmin.labels" . | nindent 4 }} +spec: + schedule: '*/1 * * * *' + jobTemplate: + spec: + template: + {{- with .Values.flyteadmin.podAnnotations }} + metadata: + annotations: {{ toYaml . | nindent 12 }} + {{- end }} + spec: + containers: + - command: + - flyteadmin + - --config + - {{ .Values.flyteadmin.configPath }} + - clusterresource + - sync + image: "{{ .Values.flyteadmin.image.repository }}:{{ .Values.flyteadmin.image.tag }}" + imagePullPolicy: "{{ .Values.flyteadmin.image.pullPolicy }}" + name: sync-cluster-resources + volumeMounts: {{- include "databaseSecret.volumeMount" . | nindent 12 }} + - mountPath: /etc/flyte/clusterresource/templates + name: resource-templates + - mountPath: /etc/flyte/config + name: config-volume + restartPolicy: OnFailure + serviceAccountName: flyteadmin + volumes: {{- include "databaseSecret.volume" . | nindent 10 }} + - configMap: + name: clusterresource-template + name: resource-templates + - configMap: + name: flyte-admin-config + name: config-volume + {{- end }} \ No newline at end of file diff --git a/helm/templates/admin/deployment.yaml b/helm/templates/admin/deployment.yaml new file mode 100644 index 0000000000..8650e23b75 --- /dev/null +++ b/helm/templates/admin/deployment.yaml @@ -0,0 +1,154 @@ +apiVersion: apps/v1 +kind: Deployment +metadata: + name: {{ template "flyteadmin.name" . }} + namespace: {{ template "flyte.namespace" . }} + labels: {{ include "flyteadmin.labels" . | nindent 4 }} +spec: + replicas: {{ .Values.flyteadmin.replicaCount }} + selector: + matchLabels: {{ include "flyteadmin.selectorLabels" . | nindent 6 }} + template: + metadata: + annotations: + configChecksum: {{ include (print .Template.BasePath "/admin/configmap.yaml") . | sha256sum | trunc 63 | quote }} + {{- with .Values.flyteadmin.podAnnotations }} + {{- toYaml . | nindent 8 }} + {{- end }} + labels: {{ include "flyteadmin.labels" . | nindent 8 }} + spec: + initContainers: + {{- if .Values.postgres.enabled }} + - name: check-db-ready + image: postgres:10.16-alpine + command: + - sh + - -c + - until pg_isready -h postgres -p 5432; do echo waiting for database; sleep 2; done; + {{- end }} + - command: + - flyteadmin + - --config + - {{ .Values.flyteadmin.configPath }} + - migrate + - run + image: "{{ .Values.flyteadmin.image.repository }}:{{ .Values.flyteadmin.image.tag }}" + imagePullPolicy: "{{ .Values.flyteadmin.image.pullPolicy }}" + name: run-migrations + volumeMounts: {{- include "databaseSecret.volumeMount" . | nindent 10 }} + - mountPath: /etc/flyte/config + name: config-volume + - command: + - flyteadmin + - --config + - {{ .Values.flyteadmin.configPath }} + - migrate + - seed-projects + - flytesnacks + - flytetester + - flyteexamples + image: "{{ .Values.flyteadmin.image.repository }}:{{ .Values.flyteadmin.image.tag }}" + imagePullPolicy: "{{ .Values.flyteadmin.image.pullPolicy }}" + name: seed-projects + volumeMounts: {{- include "databaseSecret.volumeMount" . | nindent 10 }} + - mountPath: /etc/flyte/config + name: config-volume + {{- if .Values.cluster_resource_manager.enabled }} + - command: + - flyteadmin + - --config + - {{ .Values.flyteadmin.configPath }} + - clusterresource + - sync + image: "{{ .Values.flyteadmin.image.repository }}:{{ .Values.flyteadmin.image.tag }}" + imagePullPolicy: "{{ .Values.flyteadmin.image.pullPolicy }}" + name: sync-cluster-resources + volumeMounts: {{- include "databaseSecret.volumeMount" . | nindent 10 }} + - mountPath: /etc/flyte/clusterresource/templates + name: resource-templates + - mountPath: /etc/flyte/config + name: config-volume + {{- end }} + - name: generate-secrets + image: "{{ .Values.flyteadmin.image.repository }}:{{ .Values.flyteadmin.image.tag }}" + imagePullPolicy: "{{ .Values.flyteadmin.image.pullPolicy }}" + command: ["/bin/sh", "-c"] + args: + [ + "flyteadmin --config={{ .Values.flyteadmin.configPath }} secrets init --localPath /etc/secrets/auth && flyteadmin --config=/etc/flyte/config/*.yaml secrets create --fromPath /etc/secrets/auth", + ] + volumeMounts: + - name: config-volume + mountPath: /etc/flyte/config + env: + - name: POD_NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace + containers: + - command: + - flyteadmin + - --config + - {{ .Values.flyteadmin.configPath }} + {{- with .Values.flyteadmin.extraArgs }} + {{- toYaml . | nindent 8 }} + {{- end }} + - serve + image: "{{ .Values.flyteadmin.image.repository }}:{{ .Values.flyteadmin.image.tag }}" + imagePullPolicy: "{{ .Values.flyteadmin.image.pullPolicy }}" + name: flyteadmin + ports: + - containerPort: 8088 + - containerPort: 8089 + resources: {{ toYaml .Values.flyteadmin.resources | nindent 10 }} + volumeMounts: {{- include "databaseSecret.volumeMount" . | nindent 8 }} + - mountPath: /srv/flyte + name: shared-data + - mountPath: /etc/flyte/config + name: config-volume + - name: auth + mountPath: /etc/secrets/ + - command: + - sh + - -c + - ln -s /usr/share/nginx/html /usr/share/nginx/html/openapi && sh /usr/local/bin/docker-run.sh + env: + - name: PAGE_TITLE + value: Flyte Admin OpenAPI + - name: SPEC_URL + value: /api/v1/openapi + - name: PORT + value: "8087" + image: docker.io/redocly/redoc + imagePullPolicy: IfNotPresent + name: redoc + ports: + - containerPort: 8087 + resources: + limits: + cpu: "0.1" + memory: 200Mi + serviceAccountName: {{ template "flyteadmin.name" . }} + volumes: {{- include "databaseSecret.volume" . | nindent 8 }} + - emptyDir: {} + name: shared-data + - configMap: + name: flyte-admin-config + name: config-volume + {{- if .Values.cluster_resource_manager.enabled }} + - configMap: + name: clusterresource-template + name: resource-templates + {{- end }} + - name: auth + secret: + secretName: flyte-admin-auth + {{- with .Values.flyteadmin.nodeSelector }} + nodeSelector: {{ toYaml . | nindent 8 }} + {{- end }} + {{- with .Values.flyteadmin.affinity }} + affinity: {{ toYaml . | nindent 8 }} + {{- end }} + {{- with .Values.flyteadmin.tolerations }} + tolerations: {{ toYaml . | nindent 8 }} + {{- end }} diff --git a/helm/templates/admin/rbac.yaml b/helm/templates/admin/rbac.yaml new file mode 100644 index 0000000000..9bcb0e2ceb --- /dev/null +++ b/helm/templates/admin/rbac.yaml @@ -0,0 +1,57 @@ + +--- +{{- if .Values.flyteadmin.serviceAccount.create }} +apiVersion: v1 +kind: ServiceAccount +metadata: + name: {{ template "flyteadmin.name" . }} + namespace: {{ template "flyte.namespace" . }} + labels: {{ include "flyteadmin.labels" . | nindent 4 }} + {{- with .Values.flyteadmin.serviceAccount.annotations }} + annotations: {{ toYaml . | nindent 4 }} + {{- end }} +{{- with .Values.flyteadmin.serviceAccount.imagePullSecrets }} +imagePullSecrets: {{ toYaml . | nindent 2 }} +{{- end }} +{{- end }} + +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: {{ template "flyteadmin.name" . }} + labels: {{ include "flyteadmin.labels" . | nindent 4 }} +rules: +- apiGroups: + - "" + - flyte.lyft.com + - rbac.authorization.k8s.io + resources: + - configmaps + - flyteworkflows + - namespaces + - pods + - resourcequotas + - roles + - rolebindings + - secrets + - services + - serviceaccounts + - spark-role + verbs: + - '*' + +--- +apiVersion: rbac.authorization.k8s.io/v1beta1 +kind: ClusterRoleBinding +metadata: + name: {{ template "flyteadmin.name" . }}-binding + labels: {{ include "flyteadmin.labels" . | nindent 4 }} +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: {{ template "flyteadmin.name" . }} +subjects: +- kind: ServiceAccount + name: {{ template "flyteadmin.name" . }} + namespace: {{ template "flyte.namespace" . }} diff --git a/helm/templates/admin/secret-auth.yaml b/helm/templates/admin/secret-auth.yaml new file mode 100644 index 0000000000..0d5bf72fca --- /dev/null +++ b/helm/templates/admin/secret-auth.yaml @@ -0,0 +1,6 @@ +apiVersion: v1 +kind: Secret +metadata: + name: flyte-admin-auth + namespace: {{ template "flyte.namespace" . }} +type: Opaque diff --git a/helm/templates/admin/service.yaml b/helm/templates/admin/service.yaml new file mode 100644 index 0000000000..07132a45d4 --- /dev/null +++ b/helm/templates/admin/service.yaml @@ -0,0 +1,34 @@ +apiVersion: v1 +kind: Service +metadata: + name: {{ template "flyteadmin.name" . }} + namespace: {{ template "flyte.namespace" . }} + labels: {{ include "flyteadmin.labels" . | nindent 4 }} + {{- with .Values.flyteadmin.service.annotations }} + annotations: {{ toYaml . | nindent 4 }} + {{- end }} +spec: + {{- with .Values.flyteadmin.service.type}} + type: {{ . }} + {{- end }} + {{- with .Values.flyteadmin.service.loadBalancerSourceRanges }} + loadBalancerSourceRanges: + {{ . }} + {{- end }} + ports: + - name: http + port: 80 + protocol: TCP + targetPort: 8088 + - name: grpc + port: 81 + protocol: TCP + targetPort: 8089 + - name: redoc + protocol: TCP + port: 87 + targetPort: 8087 + - name: http-metrics + protocol: TCP + port: 10254 + selector: {{ include "flyteadmin.selectorLabels" . | nindent 4 }} diff --git a/helm/templates/common/ingress.yaml b/helm/templates/common/ingress.yaml new file mode 100644 index 0000000000..10e3f10a82 --- /dev/null +++ b/helm/templates/common/ingress.yaml @@ -0,0 +1,212 @@ +{{- define "grpcRoutes" -}} +# NOTE: Port 81 in flyteadmin is the GRPC server port for FlyteAdmin. +- path: /flyteidl.service.AdminService + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 81 +- path: /flyteidl.service.AdminService/* + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 81 +- path: /flyteidl.service.AuthMetadataService + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 81 +- path: /flyteidl.service.AuthMetadataService/* + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 81 +- path: /flyteidl.service.IdentityService + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 81 +- path: /flyteidl.service.IdentityService/* + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 81 +{{- end }} +{{- if .Values.common.ingress.enabled }} +apiVersion: networking.k8s.io/v1beta1 +kind: Ingress +metadata: + name: {{ template "flyte.name" . }} + namespace: {{ template "flyte.namespace" . }} + {{- with .Values.common.ingress.annotations }} + annotations: {{ toYaml . | nindent 4}} + {{- end }} +spec: + rules: + - http: + paths: + {{- if .Values.common.ingress.albSSLRedirect }} + - path: /* + pathType: ImplementationSpecific + backend: + serviceName: ssl-redirect + servicePort: use-annotation + {{- end }} + # This is useful only for frontend development + {{- if .Values.common.ingress.webpackHMR }} + - path: /__webpack_hmr + pathType: ImplementationSpecific + backend: + serviceName: flyteconsole + servicePort: 80 + {{- end }} + # Port 87 in FlyteAdmin maps to the redoc container. + - path: /openapi + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 87 + # NOTE: If you change this, you must update the BASE_URL value in flyteconsole.yaml + - path: /console + pathType: ImplementationSpecific + backend: + serviceName: flyteconsole + servicePort: 80 + - path: /console/* + pathType: ImplementationSpecific + backend: + serviceName: flyteconsole + servicePort: 80 + - path: /api + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 80 + - path: /api/* + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 80 + - path: /healthcheck + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 80 + - path: /v1/* + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 80 + # Port 87 in FlyteAdmin maps to the redoc container. + - path: /openapi/* + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 80 + - path: /.well-known + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 80 + - path: /.well-known/* + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 80 + - path: /login + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 80 + - path: /login/* + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 80 + - path: /logout + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 80 + - path: /logout/* + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 80 + - path: /callback + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 80 + - path: /callback/* + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 80 + - path: /me + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 80 + - path: /config + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 80 + - path: /config/* + pathType: ImplementationSpecific + backend: + serviceName: flyteadmin + servicePort: 80 + {{- if not .Values.common.ingress.separateGrpcIngress }} + {{- include "grpcRoutes" . | nindent 10 -}} + {{- end }} + {{- with .Values.common.ingress.host }} + host: {{ . }} + {{- end }} + {{- if .Values.common.ingress.tls.enabled }} + tls: + - secretName: {{ .Release.Name }}-flyte-tls + hosts: + - {{ .Values.common.ingress.host }} + {{ end }} + + +{{- if .Values.common.ingress.separateGrpcIngress }} +# Certain ingress controllers like nginx cannot serve HTTP 1 and GRPC with a single ingress because GRPC can only +# enabled on the ingress object, not on backend services (GRPC annotation is set on the ingress, not on the services). +--- +apiVersion: networking.k8s.io/v1beta1 +kind: Ingress +metadata: + name: {{ template "flyte.name" . }}-grpc + namespace: {{ template "flyte.namespace" . }} + annotations: + {{- with .Values.common.ingress.annotations }} + {{- toYaml . | nindent 4}} + {{- end }} + {{- with .Values.common.ingress.separateGrpcIngressAnnotations }} + {{- toYaml . | nindent 4}} + {{- end }} +spec: + rules: + - http: + paths: + # {{- if .Values.common.ingress.albSSLRedirect }} + # - backend: + # serviceName: ssl-redirect + # servicePort: use-annotation + # path: /* + # pathType: ImplementationSpecific + # {{- end }} + {{- include "grpcRoutes" . | nindent 10 -}} + {{- with .Values.common.ingress.host }} + host: {{ . }} + {{- end }} + {{- if .Values.common.ingress.tls.enabled }} + tls: + - secretName: {{ .Release.Name }}-flyte-tls + hosts: + - {{ .Values.common.ingress.host }} + {{ end }} +{{- end }} +{{- end }} diff --git a/helm/templates/common/namespace.yaml b/helm/templates/common/namespace.yaml new file mode 100644 index 0000000000..7e4f26d461 --- /dev/null +++ b/helm/templates/common/namespace.yaml @@ -0,0 +1,9 @@ +{{- if .Values.common.flyteNamespaceTemplate.enabled }} +apiVersion: v1 +kind: Namespace +metadata: + name: {{ template "flyte.namespace" . }} +spec: + finalizers: + - kubernetes +{{- end }} diff --git a/helm/templates/common/secret.yaml b/helm/templates/common/secret.yaml new file mode 100644 index 0000000000..0282eda01d --- /dev/null +++ b/helm/templates/common/secret.yaml @@ -0,0 +1,3 @@ +{{- with .Values.common.databaseSecret.secretManifest }} +{{ toYaml . }} +{{- end }} diff --git a/helm/templates/console/configmap.yaml b/helm/templates/console/configmap.yaml new file mode 100644 index 0000000000..084091b781 --- /dev/null +++ b/helm/templates/console/configmap.yaml @@ -0,0 +1,7 @@ +apiVersion: v1 +kind: ConfigMap +metadata: + name: flyte-console-config + namespace: {{ template "flyte.namespace" . }} + labels: {{ include "flyteconsole.labels" . | nindent 4 }} +data: {{ toYaml .Values.configmap.console | nindent 2 }} diff --git a/helm/templates/console/deployment.yaml b/helm/templates/console/deployment.yaml new file mode 100644 index 0000000000..298fb2661e --- /dev/null +++ b/helm/templates/console/deployment.yaml @@ -0,0 +1,44 @@ +apiVersion: apps/v1 +kind: Deployment +metadata: + name: {{ template "flyteconsole.name" . }} + namespace: {{ template "flyte.namespace" . }} + labels: {{ include "flyteconsole.labels" . | nindent 4 }} +spec: + replicas: {{ .Values.flyteconsole.replicaCount }} + selector: + matchLabels: {{ include "flyteconsole.selectorLabels" . | nindent 6 }} + template: + metadata: + annotations: + configChecksum: {{ include (print .Template.BasePath "/console/configmap.yaml") . | sha256sum | trunc 63 | quote }} + {{- with .Values.flyteconsole.podAnnotations }} + {{- toYaml . | nindent 8 }} + {{- end }} + labels: {{ include "flyteconsole.labels" . | nindent 8 }} + spec: + containers: + - image: "{{ .Values.flyteconsole.image.repository }}:{{ .Values.flyteconsole.image.tag }}" + imagePullPolicy: "{{ .Values.flyteconsole.image.pullPolicy }}" + name: flyteconsole + envFrom: + - configMapRef: + name: flyte-console-config + ports: + - containerPort: 8080 + resources: {{ toYaml .Values.flyteconsole.resources | nindent 10 }} + volumeMounts: + - mountPath: /srv/flyte + name: shared-data + volumes: + - emptyDir: {} + name: shared-data + {{- with .Values.flyteconsole.nodeSelector }} + nodeSelector: {{ toYaml . | nindent 8 }} + {{- end }} + {{- with .Values.flyteconsole.affinity }} + affinity: {{ toYaml . | nindent 8 }} + {{- end }} + {{- with .Values.flyteconsole.tolerations }} + tolerations: {{ toYaml . | nindent 8 }} + {{- end }} \ No newline at end of file diff --git a/helm/templates/console/service.yaml b/helm/templates/console/service.yaml new file mode 100644 index 0000000000..c007243455 --- /dev/null +++ b/helm/templates/console/service.yaml @@ -0,0 +1,18 @@ +apiVersion: v1 +kind: Service +metadata: + name: {{ template "flyteconsole.name" . }} + namespace: {{ template "flyte.namespace" . }} + labels: {{ include "flyteconsole.labels" . | nindent 4 }} + {{- with .Values.flyteconsole.service.annotations }} + annotations: {{ toYaml . | nindent 4 }} + {{- end }} +spec: + {{- with .Values.flyteconsole.service.type}} + type: {{ . }} + {{- end }} + ports: + - port: 80 + protocol: TCP + targetPort: 8080 + selector: {{ include "flyteconsole.selectorLabels" . | nindent 4 }} diff --git a/helm/templates/datacatalog/configmap.yaml b/helm/templates/datacatalog/configmap.yaml new file mode 100644 index 0000000000..2f05fe1c2d --- /dev/null +++ b/helm/templates/datacatalog/configmap.yaml @@ -0,0 +1,18 @@ +apiVersion: v1 +kind: ConfigMap +metadata: + name: datacatalog-config + namespace: {{ template "flyte.namespace" . }} + labels: {{ include "datacatalog.labels" . | nindent 4 }} +data: +{{- with .Values.db }} + db.yaml: | {{ toYaml . | nindent 4 }} +{{- end }} +{{- with .Values.configmap.logger }} + logger.yaml: | {{ toYaml . | nindent 4 }} +{{- end }} +{{- with .Values.configmap.datacatalogServer }} + server.yaml: | {{ toYaml . | nindent 4 }} +{{- end }} + storage.yaml: | {{ include "storage" . | nindent 4 }} + diff --git a/helm/templates/datacatalog/deployment.yaml b/helm/templates/datacatalog/deployment.yaml new file mode 100644 index 0000000000..1e3e2b6c6a --- /dev/null +++ b/helm/templates/datacatalog/deployment.yaml @@ -0,0 +1,67 @@ +apiVersion: apps/v1 +kind: Deployment +metadata: + name: {{ template "datacatalog.name" . }} + namespace: {{ template "flyte.namespace" . }} + labels: {{ include "datacatalog.labels" . | nindent 4 }} +spec: + replicas: {{ .Values.datacatalog.replicaCount }} + selector: + matchLabels: {{ include "datacatalog.selectorLabels" . | nindent 6 }} + template: + metadata: + annotations: + configChecksum: {{ include (print .Template.BasePath "/datacatalog/configmap.yaml") . | sha256sum | trunc 63 | quote }} + {{- with .Values.datacatalog.podAnnotations }} + {{- toYaml . | nindent 8 }} + {{- end }} + labels: {{ include "datacatalog.labels" . | nindent 8 }} + spec: + initContainers: + - command: + - datacatalog + - --config + - {{ .Values.datacatalog.configPath }} + - migrate + - run + image: "{{ .Values.datacatalog.image.repository }}:{{ .Values.datacatalog.image.tag }}" + imagePullPolicy: "{{ .Values.datacatalog.image.pullPolicy }}" + name: run-migrations + volumeMounts: {{- include "databaseSecret.volumeMount" . | nindent 8 }} + - mountPath: /etc/datacatalog/config + name: config-volume + containers: + - command: + - datacatalog + - --config + - {{ .Values.datacatalog.configPath }} + {{- with .Values.datacatalog.extraArgs }} + {{- toYaml . | nindent 8 }} + {{- end }} + - serve + image: "{{ .Values.datacatalog.image.repository }}:{{ .Values.datacatalog.image.tag }}" + imagePullPolicy: "{{ .Values.datacatalog.image.pullPolicy }}" + name: datacatalog + ports: + - containerPort: 8088 + - containerPort: 8089 + resources: {{ toYaml .Values.datacatalog.resources | nindent 10 }} + volumeMounts: {{- include "databaseSecret.volumeMount" . | nindent 8 }} + - mountPath: /etc/datacatalog/config + name: config-volume + serviceAccountName: {{ template "datacatalog.name" . }} + volumes: {{- include "databaseSecret.volume" . | nindent 6 }} + - emptyDir: {} + name: shared-data + - configMap: + name: datacatalog-config + name: config-volume + {{- with .Values.datacatalog.nodeSelector }} + nodeSelector: {{ toYaml . | nindent 8 }} + {{- end }} + {{- with .Values.datacatalog.affinity }} + affinity: {{ toYaml . | nindent 8 }} + {{- end }} + {{- with .Values.datacatalog.tolerations }} + tolerations: {{ toYaml . | nindent 8 }} + {{- end }} diff --git a/helm/templates/datacatalog/rbac.yaml b/helm/templates/datacatalog/rbac.yaml new file mode 100644 index 0000000000..86a13b344f --- /dev/null +++ b/helm/templates/datacatalog/rbac.yaml @@ -0,0 +1,16 @@ + +--- +{{- if .Values.datacatalog.serviceAccount.create }} +apiVersion: v1 +kind: ServiceAccount +metadata: + name: {{ template "datacatalog.name" . }} + namespace: {{ template "flyte.namespace" . }} + labels: {{ include "datacatalog.labels" . | nindent 4 }} + {{- with .Values.datacatalog.serviceAccount.annotations }} + annotations: {{ toYaml . | nindent 4 }} + {{- end }} +{{- with .Values.datacatalog.serviceAccount.imagePullSecrets }} +imagePullSecrets: {{ toYaml . | nindent 2 }} +{{- end }} +{{- end }} \ No newline at end of file diff --git a/helm/templates/datacatalog/service.yaml b/helm/templates/datacatalog/service.yaml new file mode 100644 index 0000000000..07958d2f34 --- /dev/null +++ b/helm/templates/datacatalog/service.yaml @@ -0,0 +1,27 @@ +apiVersion: v1 +kind: Service +metadata: + name: {{ template "datacatalog.name" . }} + namespace: {{ template "flyte.namespace" . }} + labels: {{ include "datacatalog.labels" . | nindent 4 }} + {{- with .Values.datacatalog.service.annotations }} + annotations: {{ toYaml . | nindent 4 }} + {{- end }} +spec: + {{- with .Values.datacatalog.service.type}} + type: {{ . }} + {{- end }} + ports: + - name: grpc-2 + port: 8089 + protocol: TCP + targetPort: 8089 + - name: http + port: 88 + protocol: TCP + targetPort: 8088 + - name: grpc + port: 89 + protocol: TCP + targetPort: 8089 + selector: {{ include "datacatalog.selectorLabels" . | nindent 4 }} diff --git a/helm/templates/minio/deployment.yaml b/helm/templates/minio/deployment.yaml new file mode 100644 index 0000000000..63b57d293c --- /dev/null +++ b/helm/templates/minio/deployment.yaml @@ -0,0 +1,50 @@ +{{- if .Values.minio.enabled }} +apiVersion: apps/v1 +kind: Deployment +metadata: + name: {{ template "minio.name" . }} + namespace: {{ template "flyte.namespace" . }} + labels: {{ include "minio.labels" . | nindent 4 }} +spec: + replicas: {{ .Values.minio.replicaCount }} + selector: + matchLabels: {{ include "minio.selectorLabels" . | nindent 6 }} + template: + metadata: + {{- with .Values.minio.podAnnotations }} + annotations: {{ toYaml . | nindent 8 }} + {{- end }} + labels: {{ include "minio.labels" . | nindent 8 }} + spec: + containers: + - image: "{{ .Values.minio.image.repository }}:{{ .Values.minio.image.tag }}" + imagePullPolicy: "{{ .Values.minio.image.pullPolicy }}" + name: minio + args: + - server + - /data + env: + - name: MINIO_ACCESS_KEY + value: minio + - name: MINIO_SECRET_KEY + value: miniostorage + ports: + - containerPort: 9000 + name: minio + resources: {{ toYaml .Values.minio.resources | nindent 10 }} + volumeMounts: + - name: minio-storage + mountPath: /var/lib/minioql/data + volumes: + - name: minio-storage + emptyDir: {} + {{- with .Values.minio.nodeSelector }} + nodeSelector: {{ toYaml . | nindent 8 }} + {{- end }} + {{- with .Values.minio.affinity }} + affinity: {{ toYaml . | nindent 8 }} + {{- end }} + {{- with .Values.minio.tolerations }} + tolerations: {{ toYaml . | nindent 8 }} + {{- end }} +{{- end }} diff --git a/helm/templates/minio/service.yaml b/helm/templates/minio/service.yaml new file mode 100644 index 0000000000..b9b56cc1bc --- /dev/null +++ b/helm/templates/minio/service.yaml @@ -0,0 +1,21 @@ +{{- if .Values.minio.enabled }} +apiVersion: v1 +kind: Service +metadata: + name: {{ template "minio.name" . }} + namespace: {{ template "flyte.namespace" . }} + labels: {{ include "minio.labels" . | nindent 4 }} + {{- with .Values.minio.service.annotations }} + annotations: {{ toYaml . | nindent 4 }} + {{- end }} +spec: + {{- with .Values.minio.service.type}} + type: {{ . }} + {{- end }} + ports: + - name: minio + port: 9000 + protocol: TCP + targetPort: minio + selector: {{ include "minio.selectorLabels" . | nindent 4 }} +{{- end }} diff --git a/helm/templates/postgres/deployment.yaml b/helm/templates/postgres/deployment.yaml new file mode 100644 index 0000000000..ff678d36ad --- /dev/null +++ b/helm/templates/postgres/deployment.yaml @@ -0,0 +1,45 @@ +{{- if .Values.postgres.enabled }} +apiVersion: apps/v1 +kind: Deployment +metadata: + name: {{ template "postgres.name" . }} + namespace: {{ template "flyte.namespace" . }} + labels: {{ include "postgres.labels" . | nindent 4 }} +spec: + replicas: {{ .Values.postgres.replicaCount }} + selector: + matchLabels: {{ include "postgres.selectorLabels" . | nindent 6 }} + template: + metadata: + {{- with .Values.postgres.podAnnotations }} + annotations: {{ toYaml . | nindent 8 }} + {{- end }} + labels: {{ include "postgres.labels" . | nindent 8 }} + spec: + containers: + - image: "{{ .Values.postgres.image.repository }}:{{ .Values.postgres.image.tag }}" + imagePullPolicy: "{{ .Values.postgres.image.pullPolicy }}" + name: postgres + env: + - name: POSTGRES_HOST_AUTH_METHOD + value: trust + ports: + - containerPort: 5432 + name: postgres + resources: {{ toYaml .Values.postgres.resources | nindent 10 }} + volumeMounts: + - name: postgres-storage + mountPath: /var/lib/postgresql/data + volumes: + - name: postgres-storage + emptyDir: {} + {{- with .Values.postgres.nodeSelector }} + nodeSelector: {{ toYaml . | nindent 8 }} + {{- end }} + {{- with .Values.postgres.affinity }} + affinity: {{ toYaml . | nindent 8 }} + {{- end }} + {{- with .Values.postgres.tolerations }} + tolerations: {{ toYaml . | nindent 8 }} + {{- end }} +{{- end }} diff --git a/helm/templates/postgres/service.yaml b/helm/templates/postgres/service.yaml new file mode 100644 index 0000000000..4309df215e --- /dev/null +++ b/helm/templates/postgres/service.yaml @@ -0,0 +1,21 @@ +{{- if .Values.minio.enabled }} +apiVersion: v1 +kind: Service +metadata: + name: {{ template "postgres.name" . }} + namespace: {{ template "flyte.namespace" . }} + labels: {{ include "postgres.labels" . | nindent 4 }} + {{- with .Values.postgres.service.annotations }} + annotations: {{ toYaml . | nindent 4 }} + {{- end }} +spec: + {{- with .Values.postgres.service.type}} + type: {{ . }} + {{- end }} + ports: + - name: postgres + port: 5432 + protocol: TCP + targetPort: postgres + selector: {{ include "postgres.selectorLabels" . | nindent 4 }} +{{- end }} diff --git a/helm/templates/propeller/configmap.yaml b/helm/templates/propeller/configmap.yaml new file mode 100644 index 0000000000..3d38de0221 --- /dev/null +++ b/helm/templates/propeller/configmap.yaml @@ -0,0 +1,50 @@ +apiVersion: v1 +kind: ConfigMap +metadata: + name: flyte-propeller-config + namespace: {{ template "flyte.namespace" . }} + labels: {{ include "flyteadmin.labels" . | nindent 4 }} +data: +{{- with .Values.configmap.admin }} + admin.yaml: | {{ toYaml . | nindent 4 }} +{{- end }} +{{- with .Values.configmap.catalog }} + catalog.yaml: | {{ toYaml . | nindent 4 }} +{{- end }} +{{- with .Values.configmap.catalog_cache }} + catalog_cache.yaml: | {{ toYaml . | nindent 4 }} +{{- end }} +{{- with .Values.configmap.copilot }} + copilot.yaml: | {{ toYaml . | nindent 4 }} +{{- end }} +{{- with .Values.configmap.core }} + core.yaml: | {{ toYaml . | nindent 4 }} +{{- end }} +{{- with .Values.configmap.enabled_plugins }} + enabled_plugins.yaml: | {{ toYaml . | nindent 4 }} +{{- end }} +{{- with .Values.configmap.k8s }} + k8s.yaml: | {{ toYaml . | nindent 4 }} +{{- end }} +{{- with .Values.configmap.logger }} + logger.yaml: | {{ toYaml . | nindent 4 }} +{{- end }} +{{- with .Values.configmap.qubole }} + qubole.yaml: | {{ toYaml . | nindent 4 }} +{{- end }} +{{- with .Values.configmap.resource_manager }} + resource_manager.yaml: | {{ toYaml . | nindent 4 }} +{{- end }} +{{- if .Values.sparkoperator.enabled }} +{{- with .Values.sparkoperator.plugin_config }} + spark.yaml: | {{ toYaml . | nindent 4 }} +{{- end }} +{{- end }} + storage.yaml: | {{ include "storage" . | nindent 4 }} + cache.yaml: | + cache: + max_size_mbs: {{ .Values.flytepropeller.cacheSizeMbs }} + target_gc_percent: 70 +{{- with .Values.configmap.task_logs }} + task_logs.yaml: | {{ toYaml . | nindent 4 }} +{{- end }} diff --git a/helm/templates/propeller/crds/flyteworkflow.yaml b/helm/templates/propeller/crds/flyteworkflow.yaml new file mode 100644 index 0000000000..1095feca6b --- /dev/null +++ b/helm/templates/propeller/crds/flyteworkflow.yaml @@ -0,0 +1,14 @@ +apiVersion: apiextensions.k8s.io/v1beta1 +kind: CustomResourceDefinition +metadata: + name: flyteworkflows.flyte.lyft.com +spec: + group: flyte.lyft.com + names: + kind: FlyteWorkflow + plural: flyteworkflows + shortNames: + - fly + singular: flyteworkflow + scope: Namespaced + version: v1alpha1 diff --git a/helm/templates/propeller/deployment.yaml b/helm/templates/propeller/deployment.yaml new file mode 100644 index 0000000000..7b5ae7e63a --- /dev/null +++ b/helm/templates/propeller/deployment.yaml @@ -0,0 +1,60 @@ +apiVersion: apps/v1 +kind: Deployment +metadata: + name: {{ template "flytepropeller.name" . }} + namespace: {{ template "flyte.namespace" . }} + labels: {{ include "flytepropeller.labels" . | nindent 4 }} +spec: + replicas: {{ .Values.flytepropeller.replicaCount }} + selector: + matchLabels: {{ include "flytepropeller.selectorLabels" . | nindent 6 }} + template: + metadata: + annotations: + configChecksum: {{ include (print .Template.BasePath "/propeller/configmap.yaml") . | sha256sum | trunc 63 | quote }} + {{- with .Values.flytepropeller.podAnnotations }} + {{- toYaml . | nindent 8 }} + {{- end }} + labels: {{ include "flytepropeller.labels" . | nindent 8 }} + spec: + containers: + - command: + - flytepropeller + - --config + - {{ .Values.flytepropeller.configPath }} + {{- with .Values.flytepropeller.extraArgs }} + {{- toYaml . | nindent 8 }} + {{- end }} + env: + - name: POD_NAME + valueFrom: + fieldRef: + fieldPath: metadata.name + image: "{{ .Values.flytepropeller.image.repository }}:{{ .Values.flytepropeller.image.tag }}" + imagePullPolicy: "{{ .Values.flytepropeller.image.pullPolicy }}" + name: flytepropeller + ports: + - containerPort: 10254 + resources: {{ toYaml .Values.flytepropeller.resources | nindent 10 }} + volumeMounts: + - name: config-volume + mountPath: /etc/flyte/config + - name: auth + mountPath: /etc/secrets/ + serviceAccountName: {{ template "flytepropeller.name" . }} + volumes: + - configMap: + name: flyte-propeller-config + name: config-volume + - name: auth + secret: + secretName: flyte-propeller-auth + {{- with .Values.flytepropeller.nodeSelector }} + nodeSelector: {{ toYaml . | nindent 8 }} + {{- end }} + {{- with .Values.flytepropeller.affinity }} + affinity: {{ toYaml . | nindent 8 }} + {{- end }} + {{- with .Values.flytepropeller.tolerations }} + tolerations: {{ toYaml . | nindent 8 }} + {{- end }} diff --git a/helm/templates/propeller/rbac.yaml b/helm/templates/propeller/rbac.yaml new file mode 100644 index 0000000000..3667b3b8c5 --- /dev/null +++ b/helm/templates/propeller/rbac.yaml @@ -0,0 +1,98 @@ + +--- +{{- if .Values.flytepropeller.serviceAccount.create }} +apiVersion: v1 +kind: ServiceAccount +metadata: + name: {{ template "flytepropeller.name" . }} + namespace: {{ template "flyte.namespace" . }} + labels: {{ include "flytepropeller.labels" . | nindent 4 }} + {{- with .Values.flytepropeller.serviceAccount.annotations }} + annotations: {{ toYaml . | nindent 4 }} + {{- end}} +{{- with .Values.flytepropeller.serviceAccount.imagePullSecrets }} +imagePullSecrets: {{ toYaml . | nindent 2 }} +{{- end }} +{{- end }} +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: {{ template "flytepropeller.name" . }} + labels: {{ include "flytepropeller.labels" . | nindent 4 }} +rules: +# Allow RO access to PODS +- apiGroups: + - "" + resources: + - pods + verbs: + - get + - list + - watch +# Allow Event recording access +- apiGroups: + - "" + resources: + - events + verbs: + - create + - update + - delete + - patch +# Allow Access All plugin objects +- apiGroups: + - '*' + resources: + - '*' + verbs: + - get + - list + - watch + - create + - update + - delete + - patch +# Allow Access to CRD +- apiGroups: + - apiextensions.k8s.io + resources: + - customresourcedefinitions + verbs: + - get + - list + - watch + - create + - delete + - update +# Allow Access to all resources under flyte.lyft.com +- apiGroups: + - flyte.lyft.com + resources: + - flyteworkflows + - flyteworkflows/finalizers + verbs: + - get + - list + - watch + - create + - update + - delete + - patch + - post + - deletecollection + +--- +apiVersion: rbac.authorization.k8s.io/v1beta1 +kind: ClusterRoleBinding +metadata: + name: {{ template "flytepropeller.name" . }} + labels: {{ include "flytepropeller.labels" . | nindent 4 }} +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: {{ template "flytepropeller.name" . }} +subjects: +- kind: ServiceAccount + name: {{ template "flytepropeller.name" . }} + namespace: {{ template "flyte.namespace" . }} diff --git a/helm/templates/propeller/secret-auth.yaml b/helm/templates/propeller/secret-auth.yaml new file mode 100644 index 0000000000..0430727057 --- /dev/null +++ b/helm/templates/propeller/secret-auth.yaml @@ -0,0 +1,8 @@ +apiVersion: v1 +kind: Secret +metadata: + name: flyte-propeller-auth + namespace: {{ template "flyte.namespace" . }} +type: Opaque +stringData: + client_secret: foobar diff --git a/helm/templates/propeller/webhook.yaml b/helm/templates/propeller/webhook.yaml new file mode 100644 index 0000000000..b0e018b527 --- /dev/null +++ b/helm/templates/propeller/webhook.yaml @@ -0,0 +1,158 @@ +{{- if .Values.webhook.enabled }} +# Create an empty secret that the first propeller pod will populate +apiVersion: v1 +kind: Secret +metadata: + name: flyte-pod-webhook + namespace: {{ template "flyte.namespace" . }} +type: Opaque +--- +# Create the actual deployment +apiVersion: apps/v1 +kind: Deployment +metadata: + name: {{ template "flyte-pod-webhook.name" . }} + namespace: {{ template "flyte.namespace" . }} + labels: + app: {{ template "flyte-pod-webhook.name" . }} +spec: + selector: + matchLabels: + app: {{ template "flyte-pod-webhook.name" . }} + template: + metadata: + labels: + app: {{ template "flyte-pod-webhook.name" . }} + app.kubernetes.io/name: {{ template "flyte-pod-webhook.name" . }} + app.kubernetes.io/version: {{ .Values.flytepropeller.image.tag }} + annotations: + configChecksum: {{ include (print .Template.BasePath "/propeller/configmap.yaml") . | sha256sum | trunc 63 | quote }} + {{- with .Values.flyteadmin.podAnnotations }} + {{- toYaml . | nindent 8 }} + {{- end }} + spec: + serviceAccountName: {{ template "flyte-pod-webhook.name" . }} +{{- if .Values.webhook.enabled }} + initContainers: + - name: generate-secrets + image: "{{ .Values.flytepropeller.image.repository }}:{{ .Values.flytepropeller.image.tag }}" + imagePullPolicy: "{{ .Values.flytepropeller.image.pullPolicy }}" + command: + - flytepropeller + args: + - webhook + - init-certs + - --config + - /etc/flyte/config/*.yaml + env: + - name: POD_NAME + valueFrom: + fieldRef: + fieldPath: metadata.name + - name: POD_NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace + volumeMounts: + - name: config-volume + mountPath: /etc/flyte/config +{{- end }} + containers: + - name: webhook + image: "{{ .Values.flytepropeller.image.repository }}:{{ .Values.flytepropeller.image.tag }}" + imagePullPolicy: "{{ .Values.flytepropeller.image.pullPolicy }}" + command: + - flytepropeller + args: + - webhook + - --config + - /etc/flyte/config/*.yaml + env: + - name: POD_NAME + valueFrom: + fieldRef: + fieldPath: metadata.name + - name: POD_NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace + volumeMounts: + - name: config-volume + mountPath: /etc/flyte/config + readOnly: true + - name: webhook-certs + mountPath: /etc/webhook/certs + readOnly: true + volumes: + - name: config-volume + configMap: + name: flyte-propeller-config + - name: webhook-certs + secret: + secretName: flyte-pod-webhook +--- +# Service +apiVersion: v1 +kind: Service +metadata: + name: {{ template "flyte-pod-webhook.name" . }} + namespace: {{ template "flyte.namespace" . }} + {{- with .Values.webhook.service.annotations }} + annotations: {{ toYaml . | nindent 4 }} + {{- end }} +spec: + selector: + app: {{ template "flyte-pod-webhook.name" . }} + ports: + - name: https + protocol: TCP + port: 443 + targetPort: 9443 +--- +# Create a ClusterRole for the webhook +# https://kubernetes.io/docs/admin/authorization/rbac/ +kind: ClusterRole +apiVersion: rbac.authorization.k8s.io/v1 +metadata: + name: {{ template "flyte-pod-webhook.name" . }} + namespace: {{ template "flyte.namespace" . }} +rules: + - apiGroups: + - "*" + resources: + - mutatingwebhookconfigurations + - secrets + - pods + verbs: + - get + - create + - update + - patch +--- +{{- if .Values.webhook.serviceAccount.create }} +# Create a Service Account for webhook +apiVersion: v1 +kind: ServiceAccount +metadata: + name: {{ template "flyte-pod-webhook.name" . }} + namespace: {{ template "flyte.namespace" . }} + {{- with .Values.webhook.serviceAccount.annotations }} + annotations: {{ toYaml . | nindent 4 }} + {{- end}} +{{- end }} +--- +# Create a binding from Role -> ServiceAccount +kind: ClusterRoleBinding +apiVersion: rbac.authorization.k8s.io/v1 +metadata: + name: {{ template "flyte-pod-webhook.name" . }} + namespace: {{ template "flyte.namespace" . }} +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: {{ template "flyte-pod-webhook.name" . }} +subjects: + - kind: ServiceAccount + name: {{ template "flyte-pod-webhook.name" . }} + namespace: {{ template "flyte.namespace" . }} +{{- end }} \ No newline at end of file diff --git a/helm/templates/pytorch-operator/crds/pytorchjobs.yaml b/helm/templates/pytorch-operator/crds/pytorchjobs.yaml new file mode 100644 index 0000000000..ae2b7b7e78 --- /dev/null +++ b/helm/templates/pytorch-operator/crds/pytorchjobs.yaml @@ -0,0 +1,44 @@ +{{- if .Values.pytorchoperator.enabled }} +apiVersion: apiextensions.k8s.io/v1beta1 +kind: CustomResourceDefinition +metadata: + name: pytorchjobs.kubeflow.org +spec: + additionalPrinterColumns: + - JSONPath: .status.conditions[-1:].type + name: State + type: string + - JSONPath: .metadata.creationTimestamp + name: Age + type: date + group: kubeflow.org + names: + kind: PyTorchJob + plural: pytorchjobs + singular: pytorchjob + scope: Namespaced + subresources: + status: {} + validation: + openAPIV3Schema: + properties: + spec: + properties: + pytorchReplicaSpecs: + properties: + Master: + properties: + replicas: + maximum: 1 + minimum: 1 + type: integer + Worker: + properties: + replicas: + minimum: 1 + type: integer + versions: + - name: v1 + served: true + storage: true +{{- end }} diff --git a/helm/templates/pytorch-operator/deployment.yaml b/helm/templates/pytorch-operator/deployment.yaml new file mode 100644 index 0000000000..02dc98fb01 --- /dev/null +++ b/helm/templates/pytorch-operator/deployment.yaml @@ -0,0 +1,48 @@ +{{- if .Values.pytorchoperator.enabled }} +apiVersion: apps/v1 +kind: Deployment +metadata: + name: {{ template "pytorch-operator.name" . }} + namespace: {{ template "pytorch-operator.namespace" . }} + labels: {{ include "pytorch-operator.labels" . | nindent 4 }} +spec: + replicas: {{ .Values.pytorchoperator.replicaCount }} + selector: + matchLabels: {{ include "pytorch-operator.selectorLabels" . | nindent 6 }} + template: + metadata: + {{- with .Values.pytorchoperator.podAnnotations }} + annotations: {{ toYaml . | nindent 8 }} + {{- end }} + labels: {{ include "pytorch-operator.labels" . | nindent 8 }} + spec: + containers: + - image: "{{ .Values.pytorchoperator.image.repository }}:{{ .Values.pytorchoperator.image.tag }}" + imagePullPolicy: "{{ .Values.pytorchoperator.image.pullPolicy }}" + name: pytorch-operator + command: + - /pytorch-operator.v1 + - --alsologtostderr + - -v=1 + - --monitoring-port=8443 + env: + - name: MY_POD_NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace + - name: MY_POD_NAME + valueFrom: + fieldRef: + fieldPath: metadata.name + resources: {{ toYaml .Values.pytorchoperator.resources | nindent 10 }} + serviceAccountName: {{ template "pytorch-operator.name" . }} + {{- with .Values.pytorchoperator.nodeSelector }} + nodeSelector: {{ toYaml . | nindent 8 }} + {{- end }} + {{- with .Values.pytorchoperator.affinity }} + affinity: {{ toYaml . | nindent 8 }} + {{- end }} + {{- with .Values.pytorchoperator.tolerations }} + tolerations: {{ toYaml . | nindent 8 }} + {{- end }} +{{- end }} diff --git a/helm/templates/pytorch-operator/namespace.yaml b/helm/templates/pytorch-operator/namespace.yaml new file mode 100644 index 0000000000..6e15ce3dc0 --- /dev/null +++ b/helm/templates/pytorch-operator/namespace.yaml @@ -0,0 +1,6 @@ +{{- if .Values.pytorchoperator.enabled }} +apiVersion: v1 +kind: Namespace +metadata: + name: {{ template "pytorch-operator.namespace" . }} +{{- end }} diff --git a/helm/templates/pytorch-operator/rbac.yaml b/helm/templates/pytorch-operator/rbac.yaml new file mode 100644 index 0000000000..fe980a9b87 --- /dev/null +++ b/helm/templates/pytorch-operator/rbac.yaml @@ -0,0 +1,108 @@ +{{- if .Values.pytorchoperator.enabled }} +--- +apiVersion: v1 +kind: ServiceAccount +metadata: + name: {{ template "pytorch-operator.name" . }} + namespace: {{ template "pytorch-operator.namespace" . }} + labels: {{ include "pytorch-operator.labels" . | nindent 4 }} + {{- with .Values.pytorchoperator.serviceAccountAnnotations }} + annotations: {{ toYaml . | nindent 4 }} + {{- end }} + +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: {{ template "pytorch-operator.name" . }}-admin + labels: {{ include "pytorch-operator.labels" . | nindent 4 }} +rules: [] +aggregationRule: + clusterRoleSelectors: + - matchLabels: + rbac.authorization.kubeflow.org/aggregate-to-kubeflow-pytorchjobs-admin: "true" + +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: {{ template "pytorch-operator.name" . }}-edit + labels: {{ include "pytorch-operator.labels" . | nindent 4 }} +rules: +- apiGroups: + - kubeflow.org + resources: + - pytorchjobs + - pytorchjobs/status + verbs: + - get + - list + - watch + - create + - delete + - deletecollection + - patch + - update + +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: {{ template "pytorch-operator.name" . }}-view + labels: {{ include "pytorch-operator.labels" . | nindent 4 }} +rules: +- apiGroups: + - kubeflow.org + resources: + - pytorchjobs + - pytorchjobs/status + verbs: + - get + - list + - watch + +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: {{ template "pytorch-operator.name" . }} + labels: {{ include "pytorch-operator.labels" . | nindent 4 }} +rules: +- apiGroups: + - kubeflow.org + resources: + - pytorchjobs + - pytorchjobs/status + verbs: + - '*' +- apiGroups: + - apiextensions.k8s.io + resources: + - customresourcedefinitions + verbs: + - '*' +- apiGroups: + - "" + resources: + - pods + - services + - endpoints + - events + verbs: + - '*' + +--- +apiVersion: rbac.authorization.k8s.io/v1beta1 +kind: ClusterRoleBinding +metadata: + name: {{ template "pytorch-operator.name" . }} + labels: {{ include "pytorch-operator.labels" . | nindent 4 }} +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: {{ template "pytorch-operator.name" . }} +subjects: +- kind: ServiceAccount + name: {{ template "pytorch-operator.name" . }} + namespace: {{ template "pytorch-operator.namespace" . }} +{{- end }} diff --git a/helm/templates/pytorch-operator/service.yaml b/helm/templates/pytorch-operator/service.yaml new file mode 100644 index 0000000000..3deea516e7 --- /dev/null +++ b/helm/templates/pytorch-operator/service.yaml @@ -0,0 +1,20 @@ +{{- if .Values.pytorchoperator.enabled }} +apiVersion: v1 +kind: Service +metadata: + name: {{ template "pytorch-operator.name" . }} + namespace: {{ template "pytorch-operator.namespace" . }} + labels: {{ include "pytorch-operator.labels" . | nindent 4 }} + {{- with .Values.pytorchoperator.service.annotations }} + annotations: {{ toYaml . | nindent 4 }} + {{- end }} +spec: + {{- with .Values.pytorchoperator.service.type}} + type: {{ . }} + {{- end }} + ports: + - name: monitoring-port + port: 8443 + targetPort: 8443 + selector: {{ include "pytorch-operator.selectorLabels" . | nindent 4 }} +{{- end }} diff --git a/helm/templates/redis/service.yaml b/helm/templates/redis/service.yaml new file mode 100644 index 0000000000..46afc98df5 --- /dev/null +++ b/helm/templates/redis/service.yaml @@ -0,0 +1,21 @@ +{{- if .Values.redis.enabled }} +apiVersion: v1 +kind: Service +metadata: + name: {{ template "redis.name" . }}-resource-manager + namespace: {{ template "flyte.namespace" . }} + labels: {{ include "redis.labels" . | nindent 4 }} + {{- with .Values.redis.service.annotations }} + annotations: {{ toYaml . | nindent 4 }} + {{- end }} +spec: + {{- with .Values.redis.service.type}} + type: {{ . }} + {{- end }} + ports: + - name: redis + port: 6379 + protocol: TCP + targetPort: redis + selector: {{ include "redis.selectorLabels" . | nindent 4 }} +{{- end }} diff --git a/helm/templates/redis/statefulset.yaml b/helm/templates/redis/statefulset.yaml new file mode 100644 index 0000000000..93062efb22 --- /dev/null +++ b/helm/templates/redis/statefulset.yaml @@ -0,0 +1,70 @@ +{{- if .Values.redis.enabled }} +apiVersion: apps/v1 +kind: StatefulSet +metadata: + name: {{ template "redis.name" . }} + namespace: {{ template "flyte.namespace" . }} + labels: {{ include "redis.labels" . | nindent 4 }} +spec: + replicas: {{ .Values.redis.replicaCount }} + selector: + matchLabels: {{ include "redis.selectorLabels" . | nindent 6 }} + serviceName: {{ template "redis.name" . }}-resource-manager + template: + metadata: + {{- with .Values.redis.podAnnotations }} + annotations: {{ toYaml . | nindent 8 }} + {{- end }} + labels: {{ include "redis.labels" . | nindent 8 }} + spec: + containers: + - image: "{{ .Values.redis.image.repository }}:{{ .Values.redis.image.tag }}" + imagePullPolicy: "{{ .Values.redis.image.pullPolicy }}" + name: redis + env: + - name: REDIS_PASSWORD + value: mypassword + livenessProbe: + exec: + command: + - redis-cli + - ping + failureThreshold: 3 + initialDelaySeconds: 30 + periodSeconds: 10 + successThreshold: 1 + timeoutSeconds: 5 + readinessProbe: + exec: + command: + - redis-cli + - ping + failureThreshold: 3 + initialDelaySeconds: 5 + periodSeconds: 10 + successThreshold: 1 + timeoutSeconds: 1 + ports: + - containerPort: 6379 + name: redis + protocol: TCP + resources: {{ toYaml .Values.redis.resources | nindent 10 }} + volumeMounts: + - mountPath: /bitnami + name: redis-data + + dnsPolicy: ClusterFirst + restartPolicy: Always + volumes: + - emptyDir: {} + name: redis-data + {{- with .Values.redis.nodeSelector }} + nodeSelector: {{ toYaml . | nindent 8 }} + {{- end }} + {{- with .Values.redis.affinity }} + affinity: {{ toYaml . | nindent 8 }} + {{- end }} + {{- with .Values.redis.tolerations }} + tolerations: {{ toYaml . | nindent 8 }} + {{- end }} +{{- end }} diff --git a/helm/values-eks.yaml b/helm/values-eks.yaml new file mode 100644 index 0000000000..a244a4a634 --- /dev/null +++ b/helm/values-eks.yaml @@ -0,0 +1,505 @@ +# +# FLYTEADMIN +# + +flyteadmin: + replicaCount: 2 + # -- IAM role for SA: https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html + serviceAccount: + # -- If the service account is created by you, make this false, else a new service account will be created and the iam-role-flyte will be added + # you can change the name of this role + create: false + annotations: + eks.amazonaws.com/role-arn: arn:aws:iam:::role/iam-role-flyte + + resources: + limits: + ephemeral-storage: 200Mi + requests: + cpu: 50m + ephemeral-storage: 200Mi + memory: 200Mi + + service: + annotations: + service.beta.kubernetes.io/aws-load-balancer-connection-idle-timeout: "600" + # TODO alter domain + external-dns.alpha.kubernetes.io/hostname: "flyte.example.com" + type: LoadBalancer + loadBalancerSourceRanges: + # TODO change source ip range if desired + - 0.0.0.0/0 + + affinity: + podAntiAffinity: + requiredDuringSchedulingIgnoredDuringExecution: + - labelSelector: + matchLabels: + app.kubernetes.io/name: flyteadmin + topologyKey: kubernetes.io/hostname + +# +# DATACATALOG +# + +datacatalog: + replicaCount: 2 + serviceAccount: + # -- If the service account is created by you, make this false + create: false + annotations: + eks.amazonaws.com/role-arn: arn:aws:iam:::role/iam-role-flyte + resources: + limits: + cpu: 500m + ephemeral-storage: 200Mi + requests: + cpu: 50m + ephemeral-storage: 200Mi + memory: 200Mi + service: + annotations: + service.beta.kubernetes.io/aws-load-balancer-connection-idle-timeout: "600" + type: LoadBalancer + loadBalancerSourceRanges: + # TODO change source ip range if desired + - 0.0.0.0/0 + affinity: + podAntiAffinity: + requiredDuringSchedulingIgnoredDuringExecution: + - labelSelector: + matchLabels: + app.kubernetes.io/name: datacatalog + topologyKey: kubernetes.io/hostname + +# +# FLYTEPROPELLER +# + +flytepropeller: + replicaCount: 2 + serviceAccount: + # -- If the service account is created by you, make this false + create: false + annotations: + eks.amazonaws.com/role-arn: arn:aws:iam:::role/iam-role-flyte + resources: + limits: + cpu: 1 + ephemeral-storage: 1Gi + memory: 2Gi + requests: + cpu: 1 + ephemeral-storage: 1Gi + memory: 2Gi + cacheSizeMbs: 1024 + affinity: + podAntiAffinity: + requiredDuringSchedulingIgnoredDuringExecution: + - labelSelector: + matchLabels: + app.kubernetes.io/name: flytepropeller + topologyKey: kubernetes.io/hostname + +# +# FLYTECONSOLE +# + +flyteconsole: + replicaCount: 2 + resources: + limits: + cpu: 250m + service: + annotations: + service.beta.kubernetes.io/aws-load-balancer-connection-idle-timeout: "600" + # TODO alter domain + external-dns.alpha.kubernetes.io/hostname: "flyte.example.com" + type: LoadBalancer + loadBalancerSourceRanges: + # TODO change source ip range if desired + - 0.0.0.0/0 + affinity: + podAntiAffinity: + requiredDuringSchedulingIgnoredDuringExecution: + - labelSelector: + matchLabels: + app.kubernetes.io/name: flyteconsole + topologyKey: kubernetes.io/hostname + +# +# REDIS +# + +redis: + resources: + requests: + cpu: 100m + memory: 250Mi + +# +# POSTGRES +# + +postgres: + enabled: false + +# +# MINIO +# + +minio: + enabled: false + +# +# CONTOUR +# + +contour: + enabled: false + +# +# COMMON +# + +common: + ingress: + host: flyte.example.com + albSSLRedirect: true + separateGrpcIngress: true + annotations: + # -- aws-load-balancer-controller v2.1 or higher is required - https://kubernetes-sigs.github.io/aws-load-balancer-controller/v2.1/ + # For EKS if using [ALB](https://kubernetes-sigs.github.io/aws-load-balancer-controller/guide/ingress/annotations/), these annotations are set + kubernetes.io/ingress.class: alb + alb.ingress.kubernetes.io/tags: service_instance=production + alb.ingress.kubernetes.io/scheme: internet-facing + alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:us-east-2::certificate/e2f04275-2dff-4118-a493-ed3ec8f41605 + alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}, {"HTTPS":443}]' + alb.ingress.kubernetes.io/actions.ssl-redirect: '{"Type": "redirect", "RedirectConfig": { "Protocol": "HTTPS", "Port": "443", "StatusCode": "HTTP_301"}}' + # -- Instruct ALB Controller to not create multiple load balancers (and hence maintain a single endpoint for both GRPC and Http) + alb.ingress.kubernetes.io/group.name: flyte + databaseSecret: + name: db-pass + secretManifest: + # -- Leave it empty if your secret already exists + # Else you can create your own secret object. You can use Kubernetes secrets, else you can configure external secrets + # For external secrets please install Necessary dependencies, like, of your choice + # - https://github.com/hashicorp/vault + # - https://github.com/godaddy/kubernetes-external-secrets + apiVersion: v1 + kind: Secret + metadata: + name: db-pass + type: Opaque + stringData: + # -- If using plain text you can provide the password here + pass.txt: awesomesauce + +# ----------------------------------------------------- +# Core dependencies that should be configured for Flyte to work on any platform +# Specifically 2 - Storage (s3, gcs etc), Production RDBMS - Aurora, CloudSQL etc +# ------------------------------------------------------ +# +# STORAGE SETTINGS +# + +storage: + # -- Sets the storage type. Supported values are sandbox, s3, gcs and custom. + type: s3 + # -- bucketName defines the storage bucket flyte will use. Required for all types except for sandbox. + bucketName: + s3: + region: + +db: + database: + port: 5432 + # -- Create a user called flyte + username: flyte + host: + # -- Create a DB called flyteadmin (OR change the name here) + dbname: flyteadmin + passwordPath: /etc/db/pass.txt + +# +# CONFIGMAPS +# + +configmap: + adminServer: + server: + httpPort: 8088 + grpcPort: 8089 + security: + secure: false + useAuth: false + allowCors: true + allowedOrigins: + # Accepting all domains for Sandbox installation + - "*" + allowedHeaders: + - "Content-Type" + + task_resource_defaults: + task_resources: + defaults: + cpu: 1000m + memory: 1000Mi + storage: 1000Mi + limits: + storage: 2000Mi + + core: + propeller: + rawoutput-prefix: s3://s3-bucket-for-flyte/ + workers: 40 + gc-interval: 12h + max-workflow-retries: 50 + kube-client-config: + qps: 100 + burst: 25 + timeout: 30s + queue: + sub-queue: + type: bucket + rate: 100 + capacity: 1000 + + enabled_plugins: + # -- Tasks specific configuration [structure](https://pkg.go.dev/github.com/flyteorg/flytepropeller/pkg/controller/nodes/task/config#GetConfig) + tasks: + # -- Plugins configuration, [structure](https://pkg.go.dev/github.com/flyteorg/flytepropeller/pkg/controller/nodes/task/config#TaskPluginConfig) + task-plugins: + # -- [Enabled Plugins](https://pkg.go.dev/github.com/lyft/flyteplugins/go/tasks/config#Config). Enable sagemaker*, athena if you install the backend + # plugins + enabled-plugins: + - container + - sidecar + - spark + - k8s-array + - pytorch + - athena + # - sagemaker_hyperparameter_tuning + # - sagemaker_custom_training + # - sagemaker_training + default-for-task-types: + container: container + sidecar: sidecar + spark: spark + container_array: k8s-array + pytorch: pytorch + hive: athena + # sagemaker_custom_training_task: sagemaker_custom_training + # sagemaker_custom_training_job_task: sagemaker_custom_training + + logger: + logger: + level: 5 + + # -- Section that configures how the Task logs are displayed on the UI. This has to be changed based on your actual logging provider. + # Refer to [structure](https://pkg.go.dev/github.com/lyft/flyteplugins/go/tasks/logs#LogConfig) to understand how to configure various + # logging engines + task_logs: + plugins: + logs: + kubernetes-enabled: false + # -- One option is to enable cloudwatch logging for EKS, update the region and log group accordingly + # You can even disable this + cloudwatch-enabled: true + # -- region where logs are hosted + cloudwatch-region: + # -- cloudwatch log-group + cloudwatch-log-group: + +# ---------------------------------------------------------------- +# Optional Modules +# Flyte built extensions that enable various additional features in Flyte. +# All these features are optional, but are critical to run certain features +# ------------------------------------------------------------------------ + +# -- # Flyte uses a cloud hosted Cron scheduler to run workflows on a schedule. The following module is optional. Without, +# this module, you will not have scheduled launchplans/workflows. +workflow_scheduler: + enabled: false + config: + scheduler: + # -- This is configured to use Cloudwatch schedules as explained [here](https://docs.aws.amazon.com/AmazonCloudWatch/latest/events/Create-CloudWatch-Events-Scheduled-Rule.html) + eventScheduler: + scheme: aws + region: us-east-2 + scheduleRole: arn:aws:iam:::role/flyte_cron_scheduler_role + targetName: arn:aws:sqs:us-east-2::flyte-helm-test-cron-scheduler-queue + scheduleNamePrefix: flyte + workflowExecutor: + scheme: aws + region: us-east-2 + scheduleQueueName: flyte-helm-test-cron-scheduler-queue + accountId: + reconnectAttempts: 10 + reconnectDelaySeconds: 30 + +# -- +# Workflow notifications module is an optional dependency. Flyte uses cloud native pub-sub systems to notify users of +# various events in their workflows +workflow_notifications: + enabled: false + config: + notifications: + type: aws + region: us-east-2 + publisher: + topicName: arn:aws:sns:us-east-1::flyte-notifications-topic + processor: + queueName: flyte-notifications-queue + accountId: + emailer: + subject: "Flyte: {{ project }}/{{ domain }}/{{ launch_plan.name }} has '{{ phase }}'" + sender: "flyte@example.com" + body: | + Execution {{ workflow.project }}/{{ workflow.domain }}/{{ workflow.name }}/{{ name }} has {{ phase }}. + Details: https://flyte.example.com/console/projects/{{ project }}/domains/{{ domain }}/executions/{{ name }}. + {{ error }} + +# -- Configuration for the Cluster resource manager component. This is an optional component, that enables automatic +# cluster configuration. This is useful to set default quotas, manage namespaces etc that map to a project/domain +cluster_resource_manager: + # -- Enables the Cluster resource manager component + enabled: true + templates: + # -- Template for namespaces resources + - key: aa_namespace + value: | + apiVersion: v1 + kind: Namespace + metadata: + name: {{ namespace }} + spec: + finalizers: + - kubernetes + + # -- Patch default service account + - key: aab_default_service_account + value: | + apiVersion: v1 + kind: ServiceAccount + metadata: + name: default + namespace: {{ namespace }} + annotations: + eks.amazonaws.com/role-arn: {{ defaultIamRole }} + + - key: ab_project_resource_quota + value: | + apiVersion: v1 + kind: ResourceQuota + metadata: + name: project-quota + namespace: {{ namespace }} + spec: + hard: + limits.cpu: {{ projectQuotaCpu }} + limits.memory: {{ projectQuotaMemory }} + + # -- Template for spark role + - key: ad_spark_role + value: | + apiVersion: rbac.authorization.k8s.io/v1beta1 + kind: Role + metadata: + name: spark-role + namespace: {{ namespace }} + rules: + - apiGroups: + - "" + resources: + - pods + verbs: + - '*' + - apiGroups: + - "" + resources: + - services + verbs: + - '*' + - apiGroups: + - "" + resources: + - configmaps + verbs: + - '*' + + # -- Template for spark SA + - key: ae_spark_service_account + value: | + apiVersion: v1 + kind: ServiceAccount + metadata: + name: spark + namespace: {{ namespace }} + annotations: + eks.amazonaws.com/role-arn: {{ defaultIamRole }} + + # -- Template for spark role binding + - key: af_spark_role_binding + value: | + apiVersion: rbac.authorization.k8s.io/v1beta1 + kind: RoleBinding + metadata: + name: spark-role-binding + namespace: {{ namespace }} + roleRef: + apiGroup: rbac.authorization.k8s.io + kind: Role + name: spark-role + subjects: + - kind: ServiceAccount + name: spark + namespace: {{ namespace }} + +# +# SPARKOPERATOR +# + +sparkoperator: + enabled: true + resources: + limits: + cpu: 1000m + memory: 1000Mi + requests: + cpu: 50m + memory: 250Mi + +# +# PYTORCHOPERATOR +# + +pytorchoperator: + enabled: true + resources: + limits: + cpu: 1000m + memory: 1000Mi + requests: + cpu: 50m + memory: 250Mi + +# ---------------------- +# -- Distributed tensorflow training using the TF-Operator. This can be installed as explained [here](https://github.com/kubeflow/tf-operator) +# There is no helm chart maintained and hence it might need an independent installation +tf_operator: + enabled: false + +# ----------------- +# -- Training on AWS Sagemaker using AWS Sagemaker operator. To actually install the operator, please follow instructions [here](https://github.com/aws/amazon-sagemaker-operator-for-k8s/tree/master/hack/charts/installer/rolebased) +# Use the config section here to just enable sagemaker plugin in Flyte, after you have installed the operator using the information +sagemaker: + enabled: false + plugin_config: + plugins: + sagemaker: + roleArn: arn:aws:iam:::role/ + region: us-east-2 +# --------------------------- +# Flink jobs using the Flink Operator +# diff --git a/helm/values-gcp.yaml b/helm/values-gcp.yaml new file mode 100644 index 0000000000..7fa40b2a3f --- /dev/null +++ b/helm/values-gcp.yaml @@ -0,0 +1,48 @@ +# +# COMMON +# + +common: + databaseSecret: {} + ingress: {} + +# ----------------------------------------------------- +# Core dependencies that should be configured for Flyte to work on any platform +# Specifically 2 - Storage (s3, gcs etc), Production RDBMS - Aurora, CloudSQL etc +# ------------------------------------------------------ +# +# STORAGE SETTINGS +# + +storage: + # -- Sets the storage type. Supported values are sandbox, s3, gcs and custom. + type: gcs + # -- bucketName defines the storage bucket flyte will use. Required for all types except for sandbox. + bucketName: + gcs: + projectId: + +# +# CONFIGMAPS +# + +configmap: + remoteData: + remoteData: + scheme: "gcp" + + tasks: + max-plugin-phase-versions: 1000000 + task-plugins: + enabled-plugins: + - container + - sidecar + - spark + - k8s-array + - pytorch + default-for-task-types: + container: container + sidecar: sidecar + spark: spark + container_array: k8s-array + pytorch: pytorch diff --git a/helm/values-sandbox.yaml b/helm/values-sandbox.yaml new file mode 100644 index 0000000000..8d7ca4d0f7 --- /dev/null +++ b/helm/values-sandbox.yaml @@ -0,0 +1,239 @@ +# --------------------------------------------------------------------- +# Core System settings +# This section consists of Core components of Flyte and their deployment +# settings. This includes FlyteAdmin service, Datacatalog, FlytePropeller and +# Flyteconsole +# --------------------------------------------------------------------- + +# +# FLYTEADMIN +# + +flyteadmin: {} # use default config + +# +# DATACATALOG +# + +datacatalog: {} + +# +# FLYTEPROPELLER +# + +flytepropeller: {} + +# +# FLYTECONSOLE +# + +flyteconsole: {} + +# ---------------------------------------------- +# Sandbox Configuration +# Sandbox allows to run flyte without any cloud dependencies and can be run even locally on your laptop. +# This is achieved by replacing cloud service dependencies by k8s local alternatives. These may not be ideal +# for a high performance setup, but are great to try out flyte +# ----------------------------------------------- +# +# REDIS SETTINGS +# + +redis: + enabled: false + +# +# POSTGRES +# + +postgres: + image: + # Override postgres image to use alpine based (rather smaller) docker image + tag: "10.16-alpine" + +# +# MINIO +# + +minio: {} + +# +# CONTOUR +# + +contour: + contour: {} + envoy: + service: + type: NodePort + ports: + http: 80 + nodePorts: + http: 30081 + +# +# KUBERNETES DASHBOARD +# +kubernetes-dashboard: + enabled: true + extraArgs: + - --enable-skip-login + - --enable-insecure-login + - --disable-settings-authorizer + protocolHttp: true + service: + type: NodePort + externalPort: 30082 + +# ------------------------------------------------ +# +# COMMON SETTINGS +# + +common: + databaseSecret: {} + ingress: + # enable HMR route to flyteconsole for frontend development. + webpackHMR: true + tls: + enabled: false + flyteNamespaceTemplate: {} + +# ----------------------------------------------------- +# Core dependencies that should be configured for Flyte to work on any platform +# Specifically 2 - Storage (s3, gcs etc), Production RDBMS - Aurora, CloudSQL etc +# ------------------------------------------------------ +# +# STORAGE SETTINGS +# + +storage: + # -- Sets the storage type. Supported values are sandbox, s3, gcs and custom. + type: sandbox + # -- bucketName defines the storage bucket flyte will use. Required for all types except for sandbox. + bucketName: my-s3-bucket + +# Database configuration +db: + database: + port: 5432 + username: postgres + host: postgres + dbname: flyte_development + +# -------------------------------------------------------------------- +# Specializing your deployment using configuration +# ------------------------------------------------------------------- +# +# CONFIGMAPS SETTINGS +# + +configmap: + # Task default resources configuration + task_resource_defaults: + task_resources: + defaults: + memory: 200Mi + + # Core propeller configuration + core: + propeller: + rawoutput-prefix: s3://my-s3-bucket/ + workers: 20 + max-workflow-retries: 30 + + # Plugins configuration + enabled_plugins: + tasks: + task-plugins: + enabled-plugins: + - container + - sidecar + - k8s-array + default-for-task-types: + container: container + sidecar: sidecar + container_array: k8s-array + + # Kubernetes plugin configuration + k8s: + plugins: + k8s: + default-env-vars: + - FLYTE_AWS_ENDPOINT: "http://minio.flyte:9000" + - FLYTE_AWS_ACCESS_KEY_ID: minio + - FLYTE_AWS_SECRET_ACCESS_KEY: miniostorage + default-memory: 200Mi + + # Logger configuration + logger: + logger: + show-source: true + level: 4 + + # Resource manager configuration + resource_manager: + propeller: + resourcemanager: + type: noop + redis: null + + # Tasks logs plugin configuration + task_logs: + plugins: + logs: + kubernetes-enabled: true + kubernetes-template-uri: "http://localhost:30082/#/log/{{ .namespace }}/{{ .podName }}/pod?namespace={{ .namespace }}" + +# ---------------------------------------------------------------- +# Optional Modules +# Flyte built extensions that enable various additional features in Flyte. +# All these features are optional, but are critical to run certain features +# ------------------------------------------------------------------------ + +# -- +# Flyte uses a cloud hosted Cron scheduler to run workflows on a schedule. The following module is optional. Without, +# this module, you will not have scheduled launchplans/workflows. +workflow_scheduler: + enabled: false + +# -- +# Workflow notifications module is an optional dependency. Flyte uses cloud native pub-sub systems to notify users of +# various events in their workflows +workflow_notifications: + enabled: false + +# -- Configuration for the Cluster resource manager component. This is an optional component, that enables automatic +# cluster configuration. This is useful to set default quotas, manage namespaces etc that map to a project/domain +cluster_resource_manager: + # -- Enables the Cluster resource manager component + enabled: true + +# -------------------------------------------------------- +# Optional Plugins +# -------------------------------------------------------- + +# +# SPARKOPERATOR +# + +sparkoperator: + enabled: false + +# +# PYTORCHOPERATOR +# + +pytorchoperator: + enabled: false +# ---------------------- +# Distributed tensorflow training using the TF-Operator +# + +# ---------------------- +# Training on AWS Sagemaker using AWS Sagemaker operator +# + +# --------------------------- +# Flink jobs using the Flink Operator +# diff --git a/helm/values.yaml b/helm/values.yaml new file mode 100644 index 0000000000..dc30311cac --- /dev/null +++ b/helm/values.yaml @@ -0,0 +1,842 @@ + +# --------------------------------------------------------------------- +# Core System settings +# This section consists of Core components of Flyte and their deployment +# settings. This includes FlyteAdmin service, Datacatalog, FlytePropeller and +# Flyteconsole +# --------------------------------------------------------------------- + +# +# FLYTEADMIN SETTINGS +# + +flyteadmin: + # -- Replicas count for Flyteadmin deployment + replicaCount: 1 + image: + # -- Docker image for Flyteadmin deployment + repository: ghcr.io/flyteorg/flyteadmin + tag: v0.5.7 + pullPolicy: IfNotPresent + # -- Default resources requests and limits for Flyteadmin deployment + resources: + limits: + cpu: 250m + ephemeral-storage: 100Mi + memory: 500Mi + requests: + cpu: 10m + ephemeral-storage: 50Mi + memory: 50Mi + # -- Default regex string for searching configuration files + configPath: /etc/flyte/config/*.yaml + # -- Service settings for Flyteadmin + service: + annotations: + projectcontour.io/upstream-protocol.h2c: grpc + type: ClusterIP + loadBalancerSourceRanges: [] + # -- Configuration for service accounts for FlyteAdmin + serviceAccount: + # -- Should a service account be created for flyteadmin + create: true + # -- Annotations for ServiceAccount attached to Flyteadmin pods + annotations: {} + # -- ImapgePullSecrets to automatically assign to the service account + imagePullSecrets: {} + # -- Annotations for Flyteadmin pods + podAnnotations: {} + # -- nodeSelector for Flyteadmin deployment + nodeSelector: {} + # -- tolerations for Flyteadmin deployment + tolerations: [] + # -- affinity for Flyteadmin deployment + affinity: {} + +# +# DATACATALOG SETTINGS +# + +datacatalog: + # -- Replicas count for Datacatalog deployment + replicaCount: 1 + image: + # -- Docker image for Datacatalog deployment + repository: ghcr.io/flyteorg/datacatalog + tag: v0.3.4 + pullPolicy: IfNotPresent + # -- Default resources requests and limits for Datacatalog deployment + resources: + limits: + cpu: 500m + ephemeral-storage: 100Mi + memory: 500Mi + requests: + cpu: 10m + ephemeral-storage: 50Mi + memory: 50Mi + # -- Default regex string for searching configuration files + configPath: /etc/datacatalog/config/*.yaml + # -- Service settings for Datacatalog + service: + annotations: + projectcontour.io/upstream-protocol.h2c: grpc + type: NodePort + # -- Configuration for service accounts for Datacatalog + serviceAccount: + # -- Should a service account be created for Datacatalog + create: true + # -- Annotations for ServiceAccount attached to Datacatalog pods + annotations: {} + # -- ImapgePullSecrets to automatically assign to the service account + imagePullSecrets: {} + # -- Annotations for Datacatalog pods + podAnnotations: {} + # -- nodeSelector for Datacatalog deployment + nodeSelector: {} + # -- tolerations for Datacatalog deployment + tolerations: [] + # -- affinity for Datacatalog deployment + affinity: {} + +# +# FLYTEPROPELLER SETTINGS +# + +flytepropeller: + # -- Replicas count for Flytepropeller deployment + replicaCount: 1 + image: + # -- Docker image for Flytepropeller deployment + repository: ghcr.io/flyteorg/flytepropeller + tag: v0.10.10 + pullPolicy: IfNotPresent + # -- Default resources requests and limits for Flytepropeller deployment + resources: + limits: + cpu: 200m + ephemeral-storage: 100Mi + memory: 200Mi + requests: + cpu: 10m + ephemeral-storage: 50Mi + memory: 50Mi + cacheSizeMbs: 0 + # -- Default regex string for searching configuration files + configPath: /etc/flyte/config/*.yaml + + # -- Configuration for service accounts for FlytePropeller + serviceAccount: + # -- Should a service account be created for FlytePropeller + create: true + # -- Annotations for ServiceAccount attached to FlytePropeller pods + annotations: {} + # -- ImapgePullSecrets to automatically assign to the service account + imagePullSecrets: {} + # -- Annotations for Flytepropeller pods + podAnnotations: {} + # -- nodeSelector for Flytepropeller deployment + nodeSelector: {} + # -- tolerations for Flytepropeller deployment + tolerations: [] + # -- affinity for Flytepropeller deployment + affinity: {} + +# +# FLYTECONSOLE SETTINGS +# + +flyteconsole: + # -- Replicas count for Flyteconsole deployment + replicaCount: 1 + image: + # -- Docker image for Flyteconsole deployment + repository: ghcr.io/flyteorg/flyteconsole + tag: v0.20.0 + pullPolicy: IfNotPresent + # -- Default resources requests and limits for Flyteconsole deployment + resources: + limits: + cpu: 500m + memory: 250Mi + requests: + cpu: 10m + memory: 50Mi + # -- Service settings for Flyteconsole + service: + annotations: {} + type: ClusterIP + # -- Annotations for Flyteconsole pods + podAnnotations: {} + # -- nodeSelector for Flyteconsole deployment + nodeSelector: {} + # -- tolerations for Flyteconsole deployment + tolerations: [] + # -- affinity for Flyteconsole deployment + affinity: {} + +# +# WEBHOOK SETTINGS +# + +webhook: + # -- enable or disable secrets webhook + enabled: true + # -- Configuration for service accounts for the webhook + serviceAccount: + # -- Should a service account be created for the webhook + create: true + # -- Annotations for ServiceAccount attached to the webhook + annotations: {} + # -- ImapgePullSecrets to automatically assign to the service account + imagePullSecrets: {} + # -- Service settings for the webhook + service: + annotations: + projectcontour.io/upstream-protocol.h2c: grpc + type: ClusterIP + +# ---------------------------------------------- +# Sandbox Configuration +# Sandbox allows to run flyte without any cloud dependencies and can be run even locally on your laptop. +# This is achieved by replacing cloud service dependencies by k8s local alternatives. These may not be ideal +# for a high performance setup, but are great to try out flyte +# ----------------------------------------------- +# +# REDIS SETTINGS +# + +redis: + # --- enable or disable Redis Statefulset installation + enabled: true + # -- Replicas count for Redis Statefulset + replicaCount: 1 + image: + # -- Docker image for Redis Statefulset + repository: docker.io/bitnami/redis + tag: 4.0.2-r1 + pullPolicy: IfNotPresent + # -- Default resources requests and limits for Redis Statefulset + resources: + requests: + cpu: 10m + memory: 50Mi + limits: + cpu: 1000m + memory: 1Gi + # -- Service settings for Redis + service: + annotations: {} + type: ClusterIP + # -- Annotations for Redis pods + podAnnotations: {} + # -- nodeSelector for Redis Statefulset + nodeSelector: {} + # -- tolerations for Redis Statefulset + tolerations: [] + # -- affinity for Redis Statefulset + affinity: {} + +# +# POSTGRES SETTINGS +# + +postgres: + # --- enable or disable Postgres deployment installation + enabled: true + # -- Replicas count for Postgres deployment + replicaCount: 1 + image: + # -- Docker image for Postgres deployment + repository: postgres + tag: "10.16" + pullPolicy: IfNotPresent + # -- Default resources requests and limits for Postgres deployment + resources: + requests: + cpu: 10m + memory: 128Mi + limits: + cpu: 1000m + memory: 512Mi + # -- Service settings for Postgres + service: + annotations: {} + type: ClusterIP + # -- Annotations for Postgres pods + podAnnotations: {} + # -- nodeSelector for Postgres deployment + nodeSelector: {} + # -- tolerations for Postgres deployment + tolerations: [] + # -- affinity for Postgres deployment + affinity: {} + +# +# MINIO SETTINGS +# + +minio: + # --- enable or disable Minio deployment installation + enabled: true + # -- Replicas count for Minio deployment + replicaCount: 1 + image: + # -- Docker image for Minio deployment + repository: minio/minio + tag: RELEASE.2020-12-16T05-05-17Z + pullPolicy: IfNotPresent + # -- Default resources requests and limits for Minio deployment + resources: + requests: + cpu: 10m + memory: 128Mi + limits: + cpu: 200m + memory: 512Mi + # -- Service settings for Minio + service: + annotations: {} + type: ClusterIP + # -- Annotations for Minio pods + podAnnotations: {} + # -- nodeSelector for Minio deployment + nodeSelector: {} + # -- tolerations for Minio deployment + tolerations: [] + # -- affinity for Minio deployment + affinity: {} + +# +# CONTOUR SETTINGS +# + +contour: + # --- enable or disable Contour deployment installation + enabled: true + # -- Replicas count for Contour deployment + replicaCount: 1 + contour: + # -- Default resources requests and limits for Contour + resources: + requests: + cpu: 10m + memory: 50Mi + limits: + cpu: 100m + memory: 100Mi + envoy: + # -- Default resources requests and limits for Envoy + resources: + requests: + cpu: 10m + memory: 50Mi + limits: + cpu: 100m + memory: 100Mi + # -- Annotations for ServiceAccount attached to Contour pods + serviceAccountAnnotations: {} + # -- Annotations for Contour pods + podAnnotations: {} + # -- nodeSelector for Contour deployment + nodeSelector: {} + # -- tolerations for Contour deployment + tolerations: [] + # -- affinity for Contour deployment + affinity: {} + + +# +# KUBERNETES DASHBOARD +# + +kubernetes-dashboard: + enabled: false + +# ------------------------------------------------ +# +# COMMON SETTINGS +# + +common: + databaseSecret: + # -- Specify name of K8s Secret which contains Database password. Leave it empty if you don't need this Secret + name: "" + # -- Specify your Secret (with sensitive data) or pseudo-manifest (without sensitive data). See https://github.com/godaddy/kubernetes-external-secrets + secretManifest: {} + ingress: + # --- Enable or disable creating Ingress for Flyte. Relevant to disable when using e.g. Istio as ingress controller. + enabled: true + # --- Enable or disable HMR route to flyteconsole. This is useful only for frontend development. + webpackHMR: false + # --- separateGrpcIngress puts GRPC routes into a separate ingress if true. Required for certain ingress controllers like nginx. + separateGrpcIngress: false + # --- Extra Ingress annotations applied only to the GRPC ingress. Only makes sense if `separateGrpcIngress` is enabled. + separateGrpcIngressAnnotations: + nginx.ingress.kubernetes.io/backend-protocol: "GRPC" + # --- Ingress annotations applied to both HTTP and GRPC ingresses. + annotations: {} + # --- albSSLRedirect adds a special route for ssl redirect. Only useful in combination with the AWS LoadBalancer Controller. + albSSLRedirect: false + # --- Ingress hostname + # host: + tls: + enabled: false + flyteNamespaceTemplate: + # --- Enable or disable creating Flyte namespace in template. Enable when using helm as template-engine only. Disable when using `helm install ...`. + enabled: false + + +# ----------------------------------------------------- +# Core dependencies that should be configured for Flyte to work on any platform +# Specifically 2 - Storage (s3, gcs etc), Production RDBMS - Aurora, CloudSQL etc +# ------------------------------------------------------ +# +# STORAGE SETTINGS +# + +storage: + # -- Sets the storage type. Supported values are sandbox, s3, gcs and custom. + type: sandbox + # -- bucketName defines the storage bucket flyte will use. Required for all types except for sandbox. + bucketName: my-s3-bucket + # -- settings for storage type s3 + s3: + region: us-east-1 + # -- settings for storage type gcs + gcs: + # -- GCP project ID. Required for storage type gcs. + # projectId: + # -- Settings for storage type custom. See https://github:com/graymeta/stow for supported storage providers/settings. + custom: {} + +# Database configuration +db: + database: {} + # port: 5432 + # username: postgres + # host: postgres + # dbname: flyte + # passwordPath: /etc/db/pass.txt + # options: + + +# -------------------------------------------------------------------- +# Specializing your deployment using configuration +# ------------------------------------------------------------------- +# +# CONFIGMAPS SETTINGS +# + +configmap: + # -- Configuration for Flyte console UI + console: + BASE_URL: /console + CONFIG_DIR: /etc/flyte/config + DISABLE_AUTH: "1" + + # -- Domains configuration for Flyte projects. This enables the specified number of domains across all projects in Flyte. + domain: + domains: + - id: development + name: development + - id: staging + name: staging + - id: production + name: production + + # -- FlyteAdmin server configuration + adminServer: + # Refer to the [server config](https://pkg.go.dev/github.com/lyft/flyteadmin@v0.3.37/pkg/config#ServerConfig). + server: + httpPort: 8088 + grpcPort: 8089 + security: + # -- Controls whether to serve requests over SSL/TLS. + secure: false + # -- Controls whether to enforce authentication. Follow the guide in https://docs.flyte.org/ on how to setup authentication. + useAuth: false + allowCors: true + allowedOrigins: + # Accepting all domains for Sandbox installation + - "*" + allowedHeaders: + - "Content-Type" + - "flyte-authorization" + # Refer to the full [structure](https://pkg.go.dev/github.com/lyft/flyteadmin@v0.3.37/pkg/runtime/interfaces#ApplicationConfig) for documentation. + flyteadmin: + roleNameKey: "iam.amazonaws.com/role" + profilerPort: 10254 + metricsScope: "flyte:" + metadataStoragePrefix: + - "metadata" + - "admin" + eventVersion: 1 + testing: + host: http://flyteadmin + # -- Authentication configuration + auth: + authorizedUris: + # This should point at your public http Uri. + - https://localhost:30081 + # This will be used by internal services in the same namespace as flyteadmin + - http://flyteadmin:80 + # This will be used by internal services in the same cluster but different namespaces + - http://flyteadmin.flyte.svc.cluster.local:80 + + # Controls app authentication config + appAuth: + thirdPartyConfig: + flyteClient: + clientId: flytectl + redirectUri: https://localhost:53593/callback + scopes: + - offline + - all + # Controls user authentication + userAuth: + openId: + baseUrl: https://accounts.google.com + scopes: + - profile + - openid + clientId: 657465813211-6eog7ek7li5k7i7fvgv2921075063hpe.apps.googleusercontent.com + + # -- Datacatalog server config + datacatalogServer: + datacatalog: + storage-prefix: metadata/datacatalog + metrics-scope: datacatalog + profiler-port: 10254 + application: + grpcPort: 8089 + httpPort: 8080 + grpcServerReflection: true + + # -- Task default resources configuration + # Refer to the full [structure](https://pkg.go.dev/github.com/lyft/flyteadmin@v0.3.37/pkg/runtime/interfaces#TaskResourceConfiguration). + task_resource_defaults: + # -- Task default resources parameters + task_resources: + defaults: + cpu: 100m + memory: 100Mi + storage: 5Mi + limits: + cpu: 2 + memory: 8Gi + storage: 20Mi + gpu: 1 + + # -- Admin Client configuration [structure](https://pkg.go.dev/github.com/flyteorg/flytepropeller/pkg/controller/nodes/subworkflow/launchplan#AdminConfig) + admin: + event: + type: admin + rate: 500 + capacity: 1000 + admin: + endpoint: flyteadmin:81 + insecure: true + clientId: flytepropeller + clientSecretLocation: /etc/secrets/client_secret + + # -- Catalog Client configuration [structure](https://pkg.go.dev/github.com/flyteorg/flytepropeller/pkg/controller/nodes/task/catalog#Config) + # Additional advanced Catalog configuration [here](https://pkg.go.dev/github.com/lyft/flyteplugins/go/tasks/pluginmachinery/catalog#Config) + catalog: + catalog-cache: + endpoint: datacatalog:89 + type: datacatalog + insecure: true + + # -- Copilot configuration + copilot: + plugins: + k8s: + # -- Structure documented [here](https://pkg.go.dev/github.com/lyft/flyteplugins@v0.5.28/go/tasks/pluginmachinery/flytek8s/config#FlyteCoPilotConfig) + co-pilot: + name: flyte-copilot- + image: ghcr.io/lyft/flyteplugins/flytecopilot:dc4bdbd61cac88a39a5ff43e40f026bdbc2c78a2 + start-timeout: 30s + + # -- Core propeller configuration + core: + # -- follows the structure specified [here](https://pkg.go.dev/github.com/flyteorg/flytepropeller/pkg/controller/config). + propeller: + rawoutput-prefix: s3://my-s3-bucket/ + metadata-prefix: metadata/propeller + workers: 4 + max-workflow-retries: 30 + workflow-reeval-duration: 30s + downstream-eval-duration: 30s + limit-namespace: "all" + prof-port: 10254 + metrics-prefix: flyte + enable-admin-launcher: true + leader-election: + lock-config-map: + name: propeller-leader + namespace: flyte + enabled: true + lease-duration: 15s + renew-deadline: 10s + retry-period: 2s + queue: + type: batch + batching-interval: 2s + batch-size: -1 + queue: + type: maxof + rate: 100 + capacity: 1000 + base-delay: 5s + max-delay: 120s + sub-queue: + type: bucket + rate: 10 + capacity: 100 + webhook: + certDir: /etc/webhook/certs + serviceName: flyte-pod-webhook + # -- For Workflow store use configuration [here](https://pkg.go.dev/github.com/flyteorg/flytepropeller/pkg/controller/workflowstore#Config) + # + + enabled_plugins: + # -- Tasks specific configuration [structure](https://pkg.go.dev/github.com/flyteorg/flytepropeller/pkg/controller/nodes/task/config#GetConfig) + tasks: + # -- Plugins configuration, [structure](https://pkg.go.dev/github.com/flyteorg/flytepropeller/pkg/controller/nodes/task/config#TaskPluginConfig) + task-plugins: + # -- [Enabled Plugins](https://pkg.go.dev/github.com/lyft/flyteplugins/go/tasks/config#Config). Enable sagemaker*, athena if you install the backend + # plugins + enabled-plugins: + - container + - sidecar + - k8s-array + default-for-task-types: + container: container + sidecar: sidecar + container_array: k8s-array + + # -- Kubernetes specific Flyte configuration + k8s: + plugins: + # -- Configuration section for all K8s specific plugins [Configuration structure](https://pkg.go.dev/github.com/lyft/flyteplugins/go/tasks/pluginmachinery/flytek8s/config) + k8s: + default-env-vars: [] + # DEFAULT_ENV_VAR: VALUE + default-cpus: 100m + default-memory: 100Mi + + # -- Logger configuration + logger: + logger: + show-source: true + level: 4 + + remoteData: + remoteData: + region: "us-east-1" + scheme: "local" + signedUrls: + durationMinutes: 3 + + # -- Resource manager configuration + resource_manager: + # -- resource manager configuration + propeller: + resourcemanager: + type: redis + resourceMaxQuota: 10000 + redis: + hostPath: redis-resource-manager:6379 + hostKey: mypassword + + # -- Section that configures how the Task logs are displayed on the UI. This has to be changed based on your actual logging provider. + # Refer to [structure](https://pkg.go.dev/github.com/lyft/flyteplugins/go/tasks/logs#LogConfig) to understand how to configure various + # logging engines + task_logs: + plugins: + logs: + kubernetes-enabled: false + # -- One option is to enable cloudwatch logging for EKS, update the region and log group accordingly + cloudwatch-enabled: false + +# ---------------------------------------------------------------- +# Optional Modules +# Flyte built extensions that enable various additional features in Flyte. +# All these features are optional, but are critical to run certain features +# ------------------------------------------------------------------------ + +# -- **Optional Component** +# Flyte uses a cloud hosted Cron scheduler to run workflows on a schedule. The following module is optional. Without, +# this module, you will not have scheduled launchplans / workflows. +# Docs: https://docs.flyte.org/en/latest/howto/enable_and_use_schedules.html#setting-up-scheduled-workflows +workflow_scheduler: + enabled: false + config: {} + + + +# -- **Optional Component** +# Workflow notifications module is an optional dependency. Flyte uses cloud native pub-sub systems to notify users of +# various events in their workflows +workflow_notifications: + enabled: false + config: {} + + + +# -- Configuration for the Cluster resource manager component. This is an optional component, that enables automatic +# cluster configuration. This is useful to set default quotas, manage namespaces etc that map to a project/domain +cluster_resource_manager: + # -- Enables the Cluster resource manager component + enabled: true + # -- Configmap for ClusterResource parameters + config: + # -- ClusterResource parameters + # Refer to the [structure](https://pkg.go.dev/github.com/lyft/flyteadmin@v0.3.37/pkg/runtime/interfaces#ClusterResourceConfig) to customize. + cluster_resources: + templatePath: "/etc/flyte/clusterresource/templates" + customData: + - production: + - projectQuotaCpu: + value: "5" + - projectQuotaMemory: + value: "4000Mi" + - staging: + - projectQuotaCpu: + value: "2" + - projectQuotaMemory: + value: "3000Mi" + - development: + - projectQuotaCpu: + value: "4" + - projectQuotaMemory: + value: "3000Mi" + refresh: 5m + + # -- Resource templates that should be applied + templates: + # -- Template for namespaces resources + - key: aa_namespace + value: | + apiVersion: v1 + kind: Namespace + metadata: + name: {{ namespace }} + spec: + finalizers: + - kubernetes + + - key: ab_project_resource_quota + value: | + apiVersion: v1 + kind: ResourceQuota + metadata: + name: project-quota + namespace: {{ namespace }} + spec: + hard: + limits.cpu: {{ projectQuotaCpu }} + limits.memory: {{ projectQuotaMemory }} + +# -------------------------------------------------------- +# Optional Plugins +# -------------------------------------------------------- + +# -- Optional: Spark Plugin using the Spark Operator +sparkoperator: + # --- enable or disable Sparkoperator deployment installation + enabled: true + # -- Replicas count for Sparkoperator deployment + replicaCount: 1 + image: + # -- Docker image for Sparkoperator + tag: v1beta2-1.2.0-3.0.0 # Set to v1beta2-1.1.2-2.4.5 for Spark 2 + # -- Default resources requests and limits for Sparkoperator + resources: + limits: + cpu: 1000m + memory: 500M + requests: + cpu: 10m + memory: 50M + + # -- Spark plugin configuration + plugin_config: + plugins: + spark: + # -- Spark default configuration + spark-config-default: + - spark.hadoop.mapreduce.fileoutputcommitter.algorithm.version: "2" + - spark.kubernetes.allocation.batch.size: "50" + - spark.hadoop.fs.s3a.acl.default: "BucketOwnerFullControl" + - spark.hadoop.fs.s3n.impl: "org.apache.hadoop.fs.s3a.S3AFileSystem" + - spark.hadoop.fs.AbstractFileSystem.s3n.impl: "org.apache.hadoop.fs.s3a.S3A" + - spark.hadoop.fs.s3.impl: "org.apache.hadoop.fs.s3a.S3AFileSystem" + - spark.hadoop.fs.AbstractFileSystem.s3.impl: "org.apache.hadoop.fs.s3a.S3A" + - spark.hadoop.fs.s3a.impl: "org.apache.hadoop.fs.s3a.S3AFileSystem" + - spark.hadoop.fs.AbstractFileSystem.s3a.impl: "org.apache.hadoop.fs.s3a.S3A" + - spark.hadoop.fs.s3a.multipart.threshold: "536870912" + - spark.blacklist.enabled: "true" + - spark.blacklist.timeout: "5m" + - spark.task.maxfailures: "8" + + +# ---------------------- +# Distributed Pytorch execution using the K8s Pytorch operator +# + +pytorchoperator: + # --- enable or disable Pytorchoperator deployment installation + enabled: true # Set false to disable + # -- Replicas count for Pytorchoperator deployment + replicaCount: 1 + image: + # -- Docker image for Pytorchoperator + repository: gcr.io/kubeflow-images-public/pytorch-operator + tag: v1.0.0-g047cf0f + pullPolicy: IfNotPresent + # -- Default resources requests and limits for Pytorchoperator + resources: + limits: + cpu: 500m + memory: 1000M + requests: + cpu: 10m + memory: 50M + # -- Service settings for Pytorchoperator + service: + annotations: {} + type: ClusterIP + # -- Annotations for ServiceAccount attached to Pytorchoperator pods + serviceAccountAnnotations: {} + # -- Annotations for Pytorchoperator pods + podAnnotations: {} + # -- nodeSelector for Pytorchoperator deployment + nodeSelector: {} + # -- tolerations for Pytorchoperator deployment + tolerations: [] + # -- affinity for Pytorchoperator deployment + affinity: {} + +# ---------------------- +# -- Distributed tensorflow training using the TF-Operator. This can be installed as explained [here](https://github.com/kubeflow/tf-operator) +# There is no helm chart maintained and hence it might need an independent installation +tf_operator: + enabled: false + + +# ----------------- +# -- Training on AWS Sagemaker using AWS Sagemaker operator. To actually install the operator, please follow instructions [here](https://github.com/aws/amazon-sagemaker-operator-for-k8s/tree/master/hack/charts/installer/rolebased) +# Use the config section here to just enable sagemaker plugin in Flyte, after you have installed the operator using the information +sagemaker: + enabled: false + plugin_config: + plugins: + sagemaker: + roleArn: + region: + + +# --------------------------- +# -- Flink jobs using the Flink Operator: this is work in progress diff --git a/script/generate_helm.sh b/script/generate_helm.sh new file mode 100644 index 0000000000..f772801e5a --- /dev/null +++ b/script/generate_helm.sh @@ -0,0 +1,39 @@ +#!/usr/bin/env bash + +set -ex + +echo "Generating Helm" +# All the values files to be built +DEPLOYMENT=${1:-sandbox eks gcp} + +DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null && pwd )" + +helm dep update ${DIR}/../helm/ + +for deployment in ${DEPLOYMENT}; do + helm template flyte ${DIR}/../helm/ -f ${DIR}/../helm/values-${deployment}.yaml > ${DIR}/../deployment/${deployment}/flyte_helm_generated.yaml +done + +echo "Generating helm docs" +if ! command -v helm-docs &> /dev/null +then + GO111MODULE=on go get github.com/norwoodj/helm-docs/cmd/helm-docs +fi + +~/go/bin/helm-docs -t ${DIR}/../helm/README.md.gotmpl ${DIR}/../helm/ + +# This section is used by GitHub workflow to ensure that the generation step was run +if [ -n "$DELTA_CHECK" ]; then + DIRTY=$(git status --porcelain) + if [ -n "$DIRTY" ]; then + echo "FAILED: helm code updated without commiting generated code." + echo "Ensure make helm has run and all changes are committed." + DIFF=$(git diff) + echo "diff detected: $DIFF" + DIFF=$(git diff --name-only) + echo "files different: $DIFF" + exit 1 + else + echo "SUCCESS: Generated code is up to date." + fi +fi