Auth Setup Doc Polish (flyteorg#303)

* Fix master github wf Signed-off-by: Haytham Abuelfutuh <[email protected]> * Some cleanup for the auth doc Signed-off-by: Haytham Abuelfutuh <[email protected]> * Don't run docker build steps for doc updates Signed-off-by: Haytham Abuelfutuh <[email protected]> * add tabs Signed-off-by: Haytham Abuelfutuh <[email protected]> * Update auth_setup.py Signed-off-by: Haytham Abuelfutuh <[email protected]> * wrapping Signed-off-by: Haytham Abuelfutuh <[email protected]> * fix requirements compilation Signed-off-by: Haytham Abuelfutuh <[email protected]> * bump for DCO Signed-off-by: Haytham Abuelfutuh <[email protected]> * bump for DCO 2 Signed-off-by: Haytham Abuelfutuh <[email protected]> * Break down auth into 3 parts Signed-off-by: Haytham Abuelfutuh <[email protected]> * Use Sphinx-mermaid extension Signed-off-by: Haytham Abuelfutuh <[email protected]> * Tab titles Signed-off-by: Haytham Abuelfutuh <[email protected]> * Update link Signed-off-by: Haytham Abuelfutuh <[email protected]> * more cleanup Signed-off-by: Haytham Abuelfutuh <[email protected]> * demystify RP Signed-off-by: Haytham Abuelfutuh <[email protected]> * Feedback Signed-off-by: Haytham Abuelfutuh <[email protected]> Co-authored-by: SandraGH5 <[email protected]>
eapolinario · Jun 23, 2021 · a9b4071 · a9b4071
1 parent fc3b015
commit a9b4071
Show file tree

Hide file tree

Showing 13 changed files with 576 additions and 427 deletions.
diff --git a/.github/workflows/ghcr_push.yml b/.github/workflows/ghcr_push.yml
@@ -2,6 +2,9 @@ name: Build & Push Docker Images
 
 on:
   pull_request:
+    paths-ignore:
+      - 'cookbook/docs/**'
+      - 'cookbook/deployment/**'
   create:
     tags:
       - v*

diff --git a/.github/workflows/sandbox_register.yml b/.github/workflows/sandbox_register.yml
@@ -2,6 +2,9 @@ name: Sandbox Register
 
 on:
   pull_request:
+    paths-ignore:
+      - 'cookbook/docs/**'
+      - 'cookbook/deployment/**'
     branches:
       - master
 

diff --git a/.github/workflows/serialize_examples.yml b/.github/workflows/serialize_examples.yml
@@ -2,6 +2,9 @@ name: Serialize Examples
 
 on:
   pull_request:
+    paths-ignore:
+      - 'cookbook/docs/**'
+      - 'cookbook/deployment/**'
 
 jobs:
   core-build:

diff --git a/.goreleaser.yml b/.goreleaser.yml
@@ -7,7 +7,7 @@ release:
     name: flytesnacks
   draft: false
   prerelease: auto
-  name_template: "{{.ProjectName}}-v{{.Version}} {{.Env.USER}}"
+  name_template: "{{.ProjectName}} v{{.Version}}"
   disable: false
   extra_files:
     - glob: ./cookbook/release-snacks/*.tgz

diff --git a/cookbook/Makefile b/cookbook/Makefile
@@ -3,6 +3,10 @@
 include common/Makefile
 include common/parent.mk
 
+define PIP_COMPILE
+pip-compile $(1) ${PIP_ARGS} --upgrade --verbose
+endef
+
 dev-requirements.txt: export CUSTOM_COMPILE_COMMAND := $(MAKE) dev-requirements.txt
 dev-requirements.txt: dev-requirements.in install-piptools
 	$(call PIP_COMPILE,dev-requirements.in)

diff --git a/cookbook/deployment/cluster/auth_migration.py b/cookbook/deployment/cluster/auth_migration.py
@@ -0,0 +1,164 @@
+"""
+.. _migrating-auth-config:
+
+####################################
+Migrating Your Authentication Config
+####################################
+
+Using Okta as an example, you would have previously seen something like the following:
+
+Okta Config changes
+===================
+
+* An Application (OpenID Connect Web) for Flyte Admin itself (e.g. **0oal5rch46pVhCGF45d6**).
+* An Application (OpenID Native app) for Flyte-cli/flytectl (e.g. **0oal62nxuD6OSFSRq5d6**).
+  These two applications would be assigned to the relevant users.
+* An Application (Web) for Flyte Propeller (e.g. **0abc5rch46pVhCGF9876**).
+  This application would either use the default Authorization server, or you would create a new one.
+
+Admin Config Changes
+====================
+
+.. code-block:: yaml
+
+    server:
+    # ... other settings
+    security:
+        secure: false
+        useAuth: true
+        allowCors: true
+        allowedOrigins:
+        - "*"
+        allowedHeaders:
+        - "Content-Type"
+        oauth:
+        baseUrl: https://dev-62129345.okta.com/oauth2/default/
+        scopes:
+            - profile
+            - openid
+            - email
+        claims:
+            iss: https://dev-62129345.okta.com/oauth2/default
+            aud: 0oal5rch46pVhCGF45d6
+        clientId: 0oal5rch46pVhCGF45d6
+        clientSecretFile: "/Users/ytong/etc/secrets/oauth/secret"
+        authorizeUrl: "https://dev-62129345.okta.com/oauth2/default/v1/authorize"
+        tokenUrl: "https://dev-62129345.okta.com/oauth2/default/v1/token"
+        callbackUrl: "http://localhost:8088/callback"
+        cookieHashKeyFile: "/Users/ytong/etc/secrets/hashkey/hashkey"
+        cookieBlockKeyFile: "/Users/ytong/etc/secrets/blockkey/blockkey"
+        redirectUrl: "/api/v1/projects"
+        thirdPartyConfig:
+            flyteClient:
+            clientId: 0oal62nxuD6OSFSRq5d6
+            redirectUri: http://localhost:12345/callback
+
+After version `0.13.0 <https://github.com/flyteorg/flyte/tree/v0.13.0>`__ of the platform, you can still use the IdP as
+the Authorization Server if you wish. That configuration would now become:
+
+.. code-block:: yaml
+
+    server:
+    # ... other settings
+    security:
+        secure: false
+        useAuth: true
+        allowCors: true
+        allowedOrigins:
+        - "*"
+        allowedHeaders:
+        - "Content-Type"
+    auth:
+        authorizedUris:
+            # This should point at your public http Uri.
+            - https://flyte.mycompany.com
+            # This will be used by internal services in the same namespace as flyteadmin
+            - http://flyteadmin:80
+            # This will be used by internal services in the same cluster but different namespaces
+            - http://flyteadmin.flyte.svc.cluster.local:80
+        userAuth:
+            openId:
+                # Put the URL of the OpenID Connect provider.
+                baseUrl: https://dev-62129345.okta.com/oauth2/default # Okta with a custom Authorization Server
+                scopes:
+                    - profile
+                    - openid
+                    - offline_access # Uncomment if OIdC supports issuing refresh tokens.
+                # Replace with the client id created for Flyte.
+                clientId: 0oal5rch46pVhCGF45d6
+        appAuth:
+            # External delegates app auth responsibilities to an external authorization server, Internal means Flyte Admin does it itself
+            authServerType: External
+            thirdPartyConfig:
+                flyteClient:
+                    clientId: 0oal62nxuD6OSFSRq5d6
+                    redirectUri: http://localhost:12345/callback
+                    scopes:
+                    - all
+                    - offline
+
+Specifically,
+
+* The original **oauth** section has been moved two levels higher into its own section and renamed **auth** but enabling/disabling of authentication remains in the old location.
+* Secrets by default will now be looked up in **/etc/secrets**. Use the following command to generate them:
+
+  .. prompt:: bash
+
+     flyteadmin secrets init -p /etc/secrets
+
+  This will generate the new cookie hash/block keys, as well as other secrets Admin needs to run the Authorization server.
+
+* The **clientSecretFile** has been moved to **/etc/secrets/oidc_client_secret** so move that there.
+* **claims** has been removed, just delete that.
+* **authorizeUrl** and **tokenUrl** are no longer necessary.
+* The **baseUrl** for the external Authorization Server is now in the **appAuth** section.
+* The **thirdPartyConfig** has been moved to **appAuth** as well.
+* **redirectUrl** has been defaulted to **/console**. If that's the value you want, then you no longer need this setting.
+
+Propeller Config Changes
+========================
+
+You might have a configuration section that looks like this:
+
+.. code-block:: yaml
+
+    admin:
+      endpoint: dns:///mycompany.domain.com
+      useAuth: true
+      clientId: flytepropeller
+      clientSecretLocation: /etc/secrets/client_secret
+      tokenUrl: https://demo.nuclyde.io/oauth2/token
+      scopes:
+      - all
+
+This can now be simplified to:
+
+.. code-block:: yaml
+
+    admin:
+      endpoint: dns:///mycompany.domain.com
+      # If you are using the built-in authorization server, you can delete the following two lines:
+      clientId: flytepropeller
+      clientSecretLocation: /etc/secrets/client_secret
+
+Specifically,
+
+* **useAuth** is deprecated and will be removed in a future version. Auth requirement will be discovered through an anonymous admin discovery call.
+* **tokenUrl** and **scopes** will also be discovered through a metadata call.
+* **clientId** and **clientSecretLocation** have defaults that work out of the box with the built-in authorization server (e.g. if you setup Google OpenID Connect).
+
+Flyte-cli Config Changes
+========================
+
+You need these two configs set:
+
+.. prompt:: bash
+
+    FLYTE_PLATFORM_HTTP_URL=http://localhost:8088 FLYTE_CREDENTIALS_CLIENT_ID=0oal62nxuD6OSFSRq5d6 flyte-cli ...
+
+**FLYTE_PLATFORM_HTTP_URL** is used because **flyte-cli** uses only gRPC to communicate with Admin. It needs to know the
+HTTP port (which Admin hosts on a different port because of limitations of the grpc-gateway library). **flyte-cli** uses
+this setting to talk to **/.well-known/oauth-authorization-server** to retrieve information regarding the auth endpoints
+. Previously this redirected to the Okta Authorization Server's metadata endpoint. With this change, Admin now hosts its
+own (even if still using the external Authorization Service).
+"""