Update Flyte components

CHANGELOG/CHANGELOG-v1.4.0.md

@@ -0,0 +1,67 @@
+# Flyte 1.4 release
+
+The main features of the 1.4 release are:
+- Suport for `PodTemplate` at the task-level
+- Revamped auth system in flytekit
+
+As python 3.7 [reached](https://endoflife.date/python) EOL support in December of 2022, we dropped support for that version on this release.
+
+## Platform
+
+### Support for `PodTemplate` at the task-level. 
+Users can now define [PodTemplate](https://docs.flyte.org/en/latest/deployment/configuration/general.html#using-default-k8s-podtemplates) as part of the definition of a task. For example, note how we have access a full [V1PodSpec](https://github.com/kubernetes-client/python/blob/master/kubernetes/docs/V1PodSpec.md) as part of the task definition:
+
+```python=
+@task(
+    pod_template=PodTemplate(
+        primary_container_name="primary",
+        labels={"lKeyA": "lValA", "lKeyB": "lValB"},
+        annotations={"aKeyA": "aValA", "aKeyB": "aValB"},
+        pod_spec=V1PodSpec(
+            containers=[
+                V1Container(
+                    name="primary",
+                    image="repo/placeholderImage:0.0.0",
+                    command="echo",
+                    args=["wow"],
+                    resources=V1ResourceRequirements(limits={"cpu": "999", "gpu": "999"}),
+                    env=[V1EnvVar(name="eKeyC", value="eValC"), V1EnvVar(name="eKeyD", value="eValD")],
+                ),
+            ],
+            volumes=[V1Volume(name="volume")],
+            tolerations=[
+                V1Toleration(
+                    key="num-gpus",
+                    operator="Equal",
+                    value=1,
+                    effect="NoSchedule",
+                ),
+            ],
+        )
+    )
+)
+def t1(i: str):
+    ...
+```
+
+We are working on more examples in our documentation. Stay tuned!
+
+## Flytekit
+
+As promised in https://github.com/flyteorg/flytekit/releases/tag/v1.3.0, we're backporting important changes to the 1.2.x release branch. In the past month we had 2 releases: https://github.com/flyteorg/flytekit/releases/tag/v1.2.8 and https://github.com/flyteorg/flytekit/releases/tag/v1.2.9.
+
+Here's some of the highlights of this release. For a full changelog please visit https://github.com/flyteorg/flytekit/releases/tag/v1.4.0.
+
+### Revamped auth system
+In https://github.com/flyteorg/flytekit/pull/1458 we introduced a new OAuth2 handling system based on [client-side grpc interceptors](https://grpc.github.io/grpc/python/grpc.html#client-side-interceptor). 
+
+## New sandbox features
+In this new release `flytectl demo` brings the following new features:
+- Support for specifying extra configuration for Flyte
+- Support for specifying extra cluster resource templates for boostrapping new namespaces
+- Sandbox state (DB, buckets) is now persistent across restarts and upgrades
+
+## Flyteconsole
+- [Added domain settings for project dashboard](https://github.com/flyteorg/flyteconsole/pull/689)
+- [Added support for ApprovedCondition for GateNodes](https://github.com/flyteorg/flyteconsole/pull/688)
+- [Performance refactor for viewing dynamic nodes](https://github.com/flyteorg/flyteconsole/pull/680)

Dockerfile

@@ -12,7 +12,7 @@ WORKDIR /flyteorg/build
 COPY go.mod go.sum ./
 RUN go mod download
 COPY cmd cmd
-COPY --from=flyteconsole /app/dist cmd/single/dist
+COPY --from=flyteconsole /app/ cmd/single/dist
 RUN --mount=type=cache,target=/root/.cache/go-build --mount=type=cache,target=/root/go/pkg/mod \
     go build -tags console -v -o dist/flyte cmd/main.go
 

charts/flyte-core/README.md

@@ -151,7 +151,7 @@ helm install gateway bitnami/contour -n flyte
 | flyteadmin.extraArgs | object | `{}` | Appends extra command line arguments to the serve command |
 | flyteadmin.image.pullPolicy | string | `"IfNotPresent"` |  |
 | flyteadmin.image.repository | string | `"cr.flyte.org/flyteorg/flyteadmin"` | Docker image for Flyteadmin deployment |
-| flyteadmin.image.tag | string | `"v1.1.72"` |  |
+| flyteadmin.image.tag | string | `"v1.1.77"` |  |
 | flyteadmin.initialProjects | list | `["flytesnacks","flytetester","flyteexamples"]` | Initial projects to create |
 | flyteadmin.nodeSelector | object | `{}` | nodeSelector for Flyteadmin deployment |
 | flyteadmin.podAnnotations | object | `{}` | Annotations for Flyteadmin pods |
@@ -173,7 +173,7 @@ helm install gateway bitnami/contour -n flyte
 | flyteconsole.ga.tracking_id | string | `"G-0QW4DJWJ20"` |  |
 | flyteconsole.image.pullPolicy | string | `"IfNotPresent"` |  |
 | flyteconsole.image.repository | string | `"cr.flyte.org/flyteorg/flyteconsole"` | Docker image for Flyteconsole deployment |
-| flyteconsole.image.tag | string | `"v1.4.1"` |  |
+| flyteconsole.image.tag | string | `"v1.4.3"` |  |
 | flyteconsole.nodeSelector | object | `{}` | nodeSelector for Flyteconsole deployment |
 | flyteconsole.podAnnotations | object | `{}` | Annotations for Flyteconsole pods |
 | flyteconsole.priorityClassName | string | `""` | Sets priorityClassName for flyte console pod(s). |
@@ -193,7 +193,7 @@ helm install gateway bitnami/contour -n flyte
 | flytepropeller.extraArgs | object | `{}` | Appends extra command line arguments to the main command |
 | flytepropeller.image.pullPolicy | string | `"IfNotPresent"` |  |
 | flytepropeller.image.repository | string | `"cr.flyte.org/flyteorg/flytepropeller"` | Docker image for Flytepropeller deployment |
-| flytepropeller.image.tag | string | `"v1.1.64"` |  |
+| flytepropeller.image.tag | string | `"v1.1.69"` |  |
 | flytepropeller.manager | bool | `false` |  |
 | flytepropeller.nodeSelector | object | `{}` | nodeSelector for Flytepropeller deployment |
 | flytepropeller.podAnnotations | object | `{}` | Annotations for Flytepropeller pods |
@@ -213,7 +213,7 @@ helm install gateway bitnami/contour -n flyte
 | flytescheduler.configPath | string | `"/etc/flyte/config/*.yaml"` | Default regex string for searching configuration files |
 | flytescheduler.image.pullPolicy | string | `"IfNotPresent"` | Docker image pull policy |
 | flytescheduler.image.repository | string | `"cr.flyte.org/flyteorg/flytescheduler"` | Docker image for Flytescheduler deployment |
-| flytescheduler.image.tag | string | `"v1.1.72"` | Docker image tag |
+| flytescheduler.image.tag | string | `"v1.1.77"` | Docker image tag |
 | flytescheduler.nodeSelector | object | `{}` | nodeSelector for Flytescheduler deployment |
 | flytescheduler.podAnnotations | object | `{}` | Annotations for Flytescheduler pods |
 | flytescheduler.priorityClassName | string | `""` | Sets priorityClassName for flyte scheduler pod(s). |

charts/flyte-core/values.yaml

@@ -16,7 +16,7 @@ flyteadmin:
   image:
     # -- Docker image for Flyteadmin deployment
     repository: cr.flyte.org/flyteorg/flyteadmin # FLYTEADMIN_IMAGE
-    tag: v1.1.72 # FLYTEADMIN_TAG
+    tag: v1.1.77 # FLYTEADMIN_TAG
     pullPolicy: IfNotPresent
   # -- Additional flyteadmin container environment variables
   #
@@ -97,7 +97,7 @@ flytescheduler:
     # -- Docker image for Flytescheduler deployment
     repository: cr.flyte.org/flyteorg/flytescheduler # FLYTESCHEDULER_IMAGE
     # -- Docker image tag
-    tag: v1.1.72 # FLYTESCHEDULER_TAG
+    tag: v1.1.77 # FLYTESCHEDULER_TAG
     # -- Docker image pull policy
     pullPolicy: IfNotPresent
   # -- Default resources requests and limits for Flytescheduler deployment
@@ -212,7 +212,7 @@ flytepropeller:
   image:
     # -- Docker image for Flytepropeller deployment
     repository: cr.flyte.org/flyteorg/flytepropeller # FLYTEPROPELLER_IMAGE
-    tag: v1.1.64 # FLYTEPROPELLER_TAG
+    tag: v1.1.69 # FLYTEPROPELLER_TAG
     pullPolicy: IfNotPresent
   # -- Default resources requests and limits for Flytepropeller deployment
   resources:
@@ -270,7 +270,7 @@ flyteconsole:
   image:
     # -- Docker image for Flyteconsole deployment
     repository: cr.flyte.org/flyteorg/flyteconsole # FLYTECONSOLE_IMAGE
-    tag: v1.4.1 # FLYTECONSOLE_TAG
+    tag: v1.4.3 # FLYTECONSOLE_TAG
     pullPolicy: IfNotPresent
   # -- Default resources requests and limits for Flyteconsole deployment
   resources:

charts/flyte/values.yaml

@@ -16,7 +16,7 @@ flyte:
       # -- Docker image for Flyteadmin deployment
       repository: cr.flyte.org/flyteorg/flyteadmin # FLYTEADMIN_IMAGE
       # -- Docker image tag
-      tag: v1.1.72 # FLYTEADMIN_TAG
+      tag: v1.1.77 # FLYTEADMIN_TAG
       # -- Docker image pull policy
       pullPolicy: IfNotPresent
     # -- Additional flyteadmin container environment variables
@@ -84,7 +84,7 @@ flyte:
       # -- Docker image for Flytescheduler deployment
       repository: cr.flyte.org/flyteorg/flytescheduler # FLYTESCHEDULER_IMAGE
       # -- Docker image tag
-      tag: v1.1.72 # FLYTESCHEDULER_TAG
+      tag: v1.1.77 # FLYTESCHEDULER_TAG
       # -- Docker image pull policy
       pullPolicy: IfNotPresent
     # -- Default resources requests and limits for Flytescheduler deployment
@@ -178,7 +178,7 @@ flyte:
       # -- Docker image for Flytepropeller deployment
       repository: cr.flyte.org/flyteorg/flytepropeller # FLYTEPROPELLER_IMAGE
       # -- Docker image tag
-      tag: v1.1.64 # FLYTEPROPELLER_TAG
+      tag: v1.1.69 # FLYTEPROPELLER_TAG
       # -- Docker image pull policy
       pullPolicy: IfNotPresent
     # -- Default resources requests and limits for Flytepropeller deployment
@@ -223,7 +223,7 @@ flyte:
       # -- Docker image for Flyteconsole deployment
       repository: cr.flyte.org/flyteorg/flyteconsole # FLYTECONSOLE_IMAGE
       # -- Docker image tag
-      tag: v1.4.1 # FLYTECONSOLE_TAG
+      tag: v1.4.3 # FLYTECONSOLE_TAG
       # -- Docker image pull policy
       pullPolicy: IfNotPresent
     # -- Default resources requests and limits for Flyteconsole deployment

deployment/eks/flyte_aws_scheduler_helm_generated.yaml

@@ -868,7 +868,7 @@ spec:
           - /etc/flyte/config/*.yaml
           - migrate
           - run
-          image: "cr.flyte.org/flyteorg/flyteadmin:v1.1.72"
+          image: "cr.flyte.org/flyteorg/flyteadmin:v1.1.77"
           imagePullPolicy: "IfNotPresent"
           name: run-migrations
           volumeMounts:
@@ -885,7 +885,7 @@ spec:
           - flytesnacks
           - flytetester
           - flyteexamples
-          image: "cr.flyte.org/flyteorg/flyteadmin:v1.1.72"
+          image: "cr.flyte.org/flyteorg/flyteadmin:v1.1.77"
           imagePullPolicy: "IfNotPresent"
           name: seed-projects
           volumeMounts:
@@ -899,7 +899,7 @@ spec:
           - /etc/flyte/config/*.yaml
           - clusterresource
           - sync
-          image: "cr.flyte.org/flyteorg/flyteadmin:v1.1.72"
+          image: "cr.flyte.org/flyteorg/flyteadmin:v1.1.77"
           imagePullPolicy: "IfNotPresent"
           name: sync-cluster-resources
           volumeMounts:
@@ -912,7 +912,7 @@ spec:
           - mountPath: /etc/secrets/
             name: admin-secrets
         - name: generate-secrets
-          image: "cr.flyte.org/flyteorg/flyteadmin:v1.1.72"
+          image: "cr.flyte.org/flyteorg/flyteadmin:v1.1.77"
           imagePullPolicy: "IfNotPresent"
           command: ["/bin/sh", "-c"]
           args:
@@ -935,7 +935,7 @@ spec:
         - --config
         - /etc/flyte/config/*.yaml
         - serve
-        image: "cr.flyte.org/flyteorg/flyteadmin:v1.1.72"
+        image: "cr.flyte.org/flyteorg/flyteadmin:v1.1.77"
         imagePullPolicy: "IfNotPresent"
         name: flyteadmin
         ports:
@@ -1036,7 +1036,7 @@ spec:
             - /etc/flyte/config/*.yaml
             - clusterresource
             - run
-          image: "cr.flyte.org/flyteorg/flyteadmin:v1.1.72"
+          image: "cr.flyte.org/flyteorg/flyteadmin:v1.1.77"
           imagePullPolicy: "IfNotPresent"
           name: sync-cluster-resources
           volumeMounts:
@@ -1089,7 +1089,7 @@ spec:
         runAsUser: 1000
         fsGroupChangePolicy: "OnRootMismatch"
       containers:
-      - image: "cr.flyte.org/flyteorg/flyteconsole:v1.4.1"
+      - image: "cr.flyte.org/flyteorg/flyteconsole:v1.4.3"
         imagePullPolicy: "IfNotPresent"
         name: flyteconsole
         envFrom:
@@ -1255,7 +1255,7 @@ spec:
           valueFrom:
             fieldRef:
               fieldPath: metadata.namespace
-        image: "cr.flyte.org/flyteorg/flytepropeller:v1.1.64"
+        image: "cr.flyte.org/flyteorg/flytepropeller:v1.1.69"
         imagePullPolicy: "IfNotPresent"
         name: flytepropeller
         ports:
@@ -1309,7 +1309,7 @@ spec:
       labels:
         app: flyte-pod-webhook
         app.kubernetes.io/name: flyte-pod-webhook
-        app.kubernetes.io/version: v1.1.64
+        app.kubernetes.io/version: v1.1.69
       annotations:
         configChecksum: "ca9c48eb21423b6a8433aaa4a1451213cbbf3c1c5d856225b64f28e9a44cdb6"
     spec:
@@ -1320,7 +1320,7 @@ spec:
       serviceAccountName: flyte-pod-webhook
       initContainers:
       - name: generate-secrets
-        image: "cr.flyte.org/flyteorg/flytepropeller:v1.1.64"
+        image: "cr.flyte.org/flyteorg/flytepropeller:v1.1.69"
         imagePullPolicy: "IfNotPresent"
         command:
           - flytepropeller
@@ -1343,7 +1343,7 @@ spec:
             mountPath: /etc/flyte/config
       containers:
         - name: webhook
-          image: "cr.flyte.org/flyteorg/flytepropeller:v1.1.64"
+          image: "cr.flyte.org/flyteorg/flytepropeller:v1.1.69"
           imagePullPolicy: "IfNotPresent"
           command:
             - flytepropeller

deployment/eks/flyte_generated.yaml

@@ -8724,7 +8724,7 @@ spec:
           valueFrom:
             fieldRef:
               fieldPath: metadata.namespace
-        image: cr.flyte.org/flyteorg/flytepropeller:v1.1.64
+        image: cr.flyte.org/flyteorg/flytepropeller:v1.1.69
         imagePullPolicy: IfNotPresent
         name: webhook
         volumeMounts:
@@ -8751,7 +8751,7 @@ spec:
           valueFrom:
             fieldRef:
               fieldPath: metadata.namespace
-        image: cr.flyte.org/flyteorg/flytepropeller:v1.1.64
+        image: cr.flyte.org/flyteorg/flytepropeller:v1.1.69
         imagePullPolicy: IfNotPresent
         name: generate-secrets
         volumeMounts:
@@ -8799,7 +8799,7 @@ spec:
         - --config
         - /etc/flyte/config/*.yaml
         - serve
-        image: cr.flyte.org/flyteorg/flyteadmin:v1.1.72
+        image: cr.flyte.org/flyteorg/flyteadmin:v1.1.77
         imagePullPolicy: IfNotPresent
         name: flyteadmin
         ports:
@@ -8846,7 +8846,7 @@ spec:
         - /etc/flyte/config/*.yaml
         - migrate
         - run
-        image: cr.flyte.org/flyteorg/flyteadmin:v1.1.72
+        image: cr.flyte.org/flyteorg/flyteadmin:v1.1.77
         imagePullPolicy: IfNotPresent
         name: run-migrations
         volumeMounts:
@@ -8863,7 +8863,7 @@ spec:
         - flytesnacks
         - flytetester
         - flyteexamples
-        image: cr.flyte.org/flyteorg/flyteadmin:v1.1.72
+        image: cr.flyte.org/flyteorg/flyteadmin:v1.1.77
         imagePullPolicy: IfNotPresent
         name: seed-projects
         volumeMounts:
@@ -8877,7 +8877,7 @@ spec:
         - /etc/flyte/config/*.yaml
         - clusterresource
         - sync
-        image: cr.flyte.org/flyteorg/flyteadmin:v1.1.72
+        image: cr.flyte.org/flyteorg/flyteadmin:v1.1.77
         imagePullPolicy: IfNotPresent
         name: sync-cluster-resources
         volumeMounts:
@@ -8897,7 +8897,7 @@ spec:
           valueFrom:
             fieldRef:
               fieldPath: metadata.namespace
-        image: cr.flyte.org/flyteorg/flyteadmin:v1.1.72
+        image: cr.flyte.org/flyteorg/flyteadmin:v1.1.77
         imagePullPolicy: IfNotPresent
         name: generate-secrets
         volumeMounts:
@@ -8951,7 +8951,7 @@ spec:
       - envFrom:
         - configMapRef:
             name: flyte-console-config
-        image: cr.flyte.org/flyteorg/flyteconsole:v1.4.1
+        image: cr.flyte.org/flyteorg/flyteconsole:v1.4.3
         name: flyteconsole
         ports:
         - containerPort: 8080
@@ -9002,7 +9002,7 @@ spec:
           valueFrom:
             fieldRef:
               fieldPath: metadata.namespace
-        image: cr.flyte.org/flyteorg/flytepropeller:v1.1.64
+        image: cr.flyte.org/flyteorg/flytepropeller:v1.1.69
         imagePullPolicy: IfNotPresent
         name: flytepropeller
         ports:
@@ -9270,7 +9270,7 @@ spec:
             - /etc/flyte/config/*.yaml
             - clusterresource
             - sync
-            image: cr.flyte.org/flyteorg/flyteadmin:v1.1.72
+            image: cr.flyte.org/flyteorg/flyteadmin:v1.1.77
             imagePullPolicy: IfNotPresent
             name: sync-cluster-resources
             volumeMounts: