Release 0.7.0 (#473)

* bump * 0.12.0b0 * set admin to 0.3.3 * kustomize * console 0.9 * bump propeller * kustomize * Contributors * updated * updated * updated * updated * Updated milestones * Changelog updated * Update end2end/tests/endtoend.yaml * Update versions Flytepropeller: v0.4.5 FlyteAdmin: v0.3.5 DataCatalog: v0.2.2 FlyteConsole: v0.11.0 * Updated Flytepropeller * updated * update image * up * fixed changelog * updated flytepropeller * docs updated * end to end updated * version updated Co-authored-by: Ketan Umare <[email protected]>
flyteorg · Sep 4, 2020 · 4a084bb · 4a084bb
1 parent e0de967
commit 4a084bb
Show file tree

Hide file tree

Showing 19 changed files with 140 additions and 85 deletions.
diff --git a/CHANGELOG/CHANGELOG-v0.7.0.md b/CHANGELOG/CHANGELOG-v0.7.0.md
@@ -0,0 +1,18 @@
+# Flyte v0.7.0
+
+## Core Platform
+- Getting ready for node-node relationship. This will make it possible to visualize and correctly model subworkflows etc
+- Project level labels and annotations to attribute costs
+- Full data available from FlyteAdmin endpoint ( no need to use signed URLs)
+- Fixed pesky performance problems in some API's at scale
+- Backend support for custom images on sagemaker
+- Large steps towards intracloud workflow portability
+
+## Console
+ - Ability to track lineage and caching information directly in the UI. On a cache hit - possible to jump to the orginating execution.
+ - Ability to clone an execution
+ - bug fixes
+
+## Flytekit
+ - Resiliency with S3 usage
+ - Portable Workflows - possible to run a flytekit native workflow on AWS/GCP/Sandbox
diff --git a/Makefile b/Makefile
@@ -24,9 +24,9 @@ generate-local-docs:
 # Builds the entire doc tree. Assumes update_ref_docs has run and that all externals rsts are in _rsts/ dir
 .PHONY: generate-docs
 generate-docs: generate-dependent-repo-docs
-	@FLYTEKIT_VERSION=0.11.3 ./script/generate_docs.sh
+	@FLYTEKIT_VERSION=0.12.6 ./script/generate_docs.sh
 
 # updates referenced docs from other repositories (e.g. flyteidl, flytekit)
 .PHONY: generate-dependent-repo-docs
 generate-dependent-repo-docs:
-	@FLYTEKIT_VERSION=0.11.3 FLYTEIDL_VERSION=0.18.0 ./script/update_ref_docs.sh
+	@FLYTEKIT_VERSION=0.12.6 FLYTEIDL_VERSION=0.18.3 ./script/update_ref_docs.sh
diff --git a/README.md b/README.md
@@ -6,7 +6,7 @@
 ![Commit activity](https://img.shields.io/github/commit-activity/w/lyft/flyte.svg?style=plastic)
 ![Commit since last release](https://img.shields.io/github/commits-since/lyft/flyte/latest.svg?style=plastic)
 ![GitHub milestones Completed](https://img.shields.io/github/milestones/closed/lyft/flyte?style=plastic)
-![GitHub next milestone percentage](https://img.shields.io/github/milestones/progress-percent/lyft/flyte/7?style=plastic)
+![GitHub next milestone percentage](https://img.shields.io/github/milestones/progress-percent/lyft/flyte/8?style=plastic)
 ![Twitter Follow](https://img.shields.io/twitter/follow/flyteorg?label=Follow&style=social)
 [![Slack Status](https://img.shields.io/badge/slack-join_chat-white.svg?logo=slack&style=social)](https://docs.google.com/forms/d/e/1FAIpQLSf8bNuyhy7rkm77cOXPHIzCm3ApfL7Tdo7NUs6Ej2NOGQ1PYw/viewform?pli=1)
 
@@ -121,22 +121,59 @@ Resources that would help you get a better understanding of Flyte.
 
 
 # Component Repos 
-Repo | Language | Purpose
---- | --- | ---
-[flyte](https://github.com/lyft/flyte) | Kustomize,RST | deployment, documentation, issues
-[flyteidl](https://github.com/lyft/flyteidl) | Protobuf | interface definitions
-[flytepropeller](https://github.com/lyft/flytepropeller) | Go | execution engine
-[flyteadmin](https://github.com/lyft/flyteadmin) | Go | control plane
-[flytekit](https://github.com/lyft/flytekit) | Python | python SDK and tools
-[flyteconsole](https://github.com/lyft/flyteconsole) | Typescript | admin console
-[datacatalog](https://github.com/lyft/datacatalog) | Go  | manage input & output artifacts
-[flyteplugins](https://github.com/lyft/flyteplugins) | Go  | flyte plugins
-[flytestdlib](https://github.com/lyft/flytestdlib) |  Go | standard library
-[flytesnacks](https://github.com/lyft/flytesnacks) | Python | examples, tips, and tricks
+Repo | Language | Purpose | Status
+--- | --- | --- | ---
+[flyte](https://github.com/lyft/flyte) | Kustomize,RST | deployment, documentation, issues | Production-grade
+[flyteidl](https://github.com/lyft/flyteidl) | Protobuf | interface definitions | Production-grade
+[flytepropeller](https://github.com/lyft/flytepropeller) | Go | execution engine | Production-grade
+[flyteadmin](https://github.com/lyft/flyteadmin) | Go | control plane | Production-grade
+[flytekit](https://github.com/lyft/flytekit) | Python | python SDK and tools | Production-grade
+[flyteconsole](https://github.com/lyft/flyteconsole) | Typescript | admin console | Production-grade
+[datacatalog](https://github.com/lyft/datacatalog) | Go  | manage input & output artifacts | Production-grade
+[flyteplugins](https://github.com/lyft/flyteplugins) | Go  | flyte plugins | Production-grade
+[flytestdlib](https://github.com/lyft/flytestdlib) |  Go | standard library | Production-grade
+[flytesnacks](https://github.com/lyft/flytesnacks) | Python | examples, tips, and tricks | Incubating
+[flytekit-java](https://github.com/spotify/flytekit-java) | Java/Scala | Java & scala SDK for authoring Flyte workflows | Incubating
+[flytectl](https://github.com/lyft/flytectl) | Java/Scala | Java & scala SDK for authoring Flyte workflows | Incomplete
+
 
 # Production K8s Operators
 
 Repo | Language | Purpose
 --- | --- | ---
 [Spark](https://github.com/GoogleCloudPlatform/spark-on-k8s-operator) | Go | Apache Spark batch
 [Flink](https://github.com/lyft/flinkk8soperator) | Go | Apache Flink streaming
+
+# Top Contributors
+Thank you to the community for making Flyte possible.
+ - [@wild-endeavor](https://github.com/wild-endeavor)
+ - [@katrogan](https://github.com/katrogan)
+ - [@EngHabu](https://github.com/EngHabu)
+ - [@akhurana001](https://github.com/akhurana001)
+ - [@anandswaminathan](https://github.com/anandswaminathan)
+ - [@kanterov](https://github.com/kanterov)
+ - [@honnix](https://github.com/honnix)
+ - [@jeevb](https://github.com/jeevb)
+ - [@jonathanburns](https://github.com/jonathanburns)
+ - [@migueltol22](https://github.com/migueltol22)
+ - [@varshaparthay](https://github.com/varshaparthay)
+ - [@pingsutw](https://github.com/pingsutw)
+ - [@narape](https://github.com/narape)
+ - [@lu4nm3](https://github.com/lu4nm3)
+ - [@bnsblue](https://github.com/bnsblue)
+ - [@RubenBarragan](https://github.com/RubenBarragan)
+ - [@schottra](https://github.com/schottra)
+ - [@evalsocket](https://github.com/evalsocket)
+ - [@matthewphsmith](https://github.com/matthewphsmith)
+ - [@slai](https://github.com/slai)
+ - [@derwiki](https://github.com/derwiki)
+ - [@tnsetting](https://github.com/tnsetting)
+ - [@jbrambleDC](https://github.com/jbrambleDC)
+ - [@igorvalko](https://github.com/igorvalko)
+ - [@chanadian](https://github.com/chanadian)
+ - [@surindersinghp](https://github.com/surindersinghp)
+ - [@vsbus](https://github.com/vsbus)
+ - [@catalinii](https://github.com/catalinii)
+ - [@kumare3](https://github.com/kumare3)
+
+
diff --git a/deployment/eks/flyte_generated.yaml b/deployment/eks/flyte_generated.yaml
@@ -524,7 +524,7 @@ spec:
       labels:
         app: datacatalog
         app.kubernetes.io/name: datacatalog
-        app.kubernetes.io/version: 0.2.1
+        app.kubernetes.io/version: 0.2.2
     spec:
       containers:
       - command:
@@ -533,7 +533,7 @@ spec:
         - --config
         - /etc/datacatalog/config/datacatalog_config.yaml
         - serve
-        image: docker.io/lyft/datacatalog:0.2.1
+        image: docker.io/lyft/datacatalog:v0.2.2
         imagePullPolicy: IfNotPresent
         name: datacatalog
         ports:
@@ -555,7 +555,7 @@ spec:
         - /etc/datacatalog/config/datacatalog_config.yaml
         - migrate
         - run
-        image: docker.io/lyft/datacatalog:0.2.1
+        image: docker.io/lyft/datacatalog:v0.2.2
         imagePullPolicy: IfNotPresent
         name: run-migrations
         volumeMounts:
@@ -590,7 +590,7 @@ spec:
       labels:
         app: flyteadmin
         app.kubernetes.io/name: flyteadmin
-        app.kubernetes.io/version: 0.3.0
+        app.kubernetes.io/version: 0.3.4
     spec:
       containers:
       - command:
@@ -599,7 +599,7 @@ spec:
         - --config
         - /etc/flyte/config/flyteadmin_config.yaml
         - serve
-        image: docker.io/lyft/flyteadmin:v0.3.0
+        image: docker.io/lyft/flyteadmin:v0.3.5
         imagePullPolicy: IfNotPresent
         name: flyteadmin
         ports:
@@ -643,7 +643,7 @@ spec:
         - /etc/flyte/config/flyteadmin_config.yaml
         - migrate
         - run
-        image: docker.io/lyft/flyteadmin:v0.3.0
+        image: docker.io/lyft/flyteadmin:v0.3.5
         imagePullPolicy: IfNotPresent
         name: run-migrations
         volumeMounts:
@@ -659,7 +659,7 @@ spec:
         - flytesnacks
         - flytetester
         - flyteexamples
-        image: docker.io/lyft/flyteadmin:v0.3.0
+        image: docker.io/lyft/flyteadmin:v0.3.5
         imagePullPolicy: IfNotPresent
         name: seed-projects
         volumeMounts:
@@ -672,7 +672,7 @@ spec:
         - /etc/flyte/config/flyteadmin_config.yaml
         - clusterresource
         - sync
-        image: docker.io/lyft/flyteadmin:v0.3.0
+        image: docker.io/lyft/flyteadmin:v0.3.5
         imagePullPolicy: IfNotPresent
         name: sync-cluster-resources
         volumeMounts:
@@ -708,13 +708,13 @@ spec:
       labels:
         app: flyteconsole
         app.kubernetes.io/name: flyteconsole
-        app.kubernetes.io/version: 0.9.0
+        app.kubernetes.io/version: 0.11.0
     spec:
       containers:
       - envFrom:
         - configMapRef:
             name: flyte-console-config
-        image: docker.io/lyft/flyteconsole:v0.9.0
+        image: docker.io/lyft/flyteconsole:v0.11.0
         name: flyteconsole
         ports:
         - containerPort: 8080
@@ -750,7 +750,7 @@ spec:
       labels:
         app: flytepropeller
         app.kubernetes.io/name: flytepropeller
-        app.kubernetes.io/version: 0.3.2
+        app.kubernetes.io/version: 0.3.12
     spec:
       containers:
       - args:
@@ -763,7 +763,7 @@ spec:
           valueFrom:
             fieldRef:
               fieldPath: metadata.name
-        image: docker.io/lyft/flytepropeller:v0.3.2
+        image: docker.io/lyft/flytepropeller:v0.3.12
         imagePullPolicy: IfNotPresent
         name: flytepropeller
         ports:
@@ -870,7 +870,7 @@ spec:
             - /etc/flyte/config/flyteadmin_config.yaml
             - clusterresource
             - sync
-            image: docker.io/lyft/flyteadmin:v0.3.0
+            image: docker.io/lyft/flyteadmin:v0.3.5
             imagePullPolicy: IfNotPresent
             name: sync-cluster-resources
             volumeMounts:

diff --git a/deployment/gcp/flyte_generated.yaml b/deployment/gcp/flyte_generated.yaml
@@ -537,7 +537,7 @@ spec:
       labels:
         app: datacatalog
         app.kubernetes.io/name: datacatalog
-        app.kubernetes.io/version: 0.2.1
+        app.kubernetes.io/version: 0.2.2
     spec:
       containers:
       - command:
@@ -546,7 +546,7 @@ spec:
         - --config
         - /etc/datacatalog/config/datacatalog_config.yaml
         - serve
-        image: docker.io/lyft/datacatalog:0.2.1
+        image: docker.io/lyft/datacatalog:v0.2.2
         imagePullPolicy: IfNotPresent
         name: datacatalog
         ports:
@@ -578,7 +578,7 @@ spec:
         - /etc/datacatalog/config/datacatalog_config.yaml
         - migrate
         - run
-        image: docker.io/lyft/datacatalog:0.2.1
+        image: docker.io/lyft/datacatalog:v0.2.2
         imagePullPolicy: IfNotPresent
         name: run-migrations
         volumeMounts:
@@ -618,7 +618,7 @@ spec:
       labels:
         app: flyteadmin
         app.kubernetes.io/name: flyteadmin
-        app.kubernetes.io/version: 0.3.0
+        app.kubernetes.io/version: 0.3.4
     spec:
       containers:
       - command:
@@ -627,7 +627,7 @@ spec:
         - --config
         - /etc/flyte/config/flyteadmin_config.yaml
         - serve
-        image: docker.io/lyft/flyteadmin:v0.3.0
+        image: docker.io/lyft/flyteadmin:v0.3.5
         imagePullPolicy: IfNotPresent
         name: flyteadmin
         ports:
@@ -681,7 +681,7 @@ spec:
         - /etc/flyte/config/flyteadmin_config.yaml
         - migrate
         - run
-        image: docker.io/lyft/flyteadmin:v0.3.0
+        image: docker.io/lyft/flyteadmin:v0.3.5
         imagePullPolicy: IfNotPresent
         name: run-migrations
         volumeMounts:
@@ -699,7 +699,7 @@ spec:
         - flytesnacks
         - flytetester
         - flyteexamples
-        image: docker.io/lyft/flyteadmin:v0.3.0
+        image: docker.io/lyft/flyteadmin:v0.3.5
         imagePullPolicy: IfNotPresent
         name: seed-projects
         volumeMounts:
@@ -714,7 +714,7 @@ spec:
         - /etc/flyte/config/flyteadmin_config.yaml
         - clusterresource
         - sync
-        image: docker.io/lyft/flyteadmin:v0.3.0
+        image: docker.io/lyft/flyteadmin:v0.3.5
         imagePullPolicy: IfNotPresent
         name: sync-cluster-resources
         volumeMounts:
@@ -755,13 +755,13 @@ spec:
       labels:
         app: flyteconsole
         app.kubernetes.io/name: flyteconsole
-        app.kubernetes.io/version: 0.9.0
+        app.kubernetes.io/version: 0.11.0
     spec:
       containers:
       - envFrom:
         - configMapRef:
             name: flyte-console-config
-        image: docker.io/lyft/flyteconsole:v0.9.0
+        image: docker.io/lyft/flyteconsole:v0.11.0
         name: flyteconsole
         ports:
         - containerPort: 8080
@@ -797,7 +797,7 @@ spec:
       labels:
         app: flytepropeller
         app.kubernetes.io/name: flytepropeller
-        app.kubernetes.io/version: 0.3.2
+        app.kubernetes.io/version: 0.3.12
     spec:
       containers:
       - args:
@@ -810,7 +810,7 @@ spec:
           valueFrom:
             fieldRef:
               fieldPath: metadata.name
-        image: docker.io/lyft/flytepropeller:v0.3.2
+        image: docker.io/lyft/flytepropeller:v0.3.12
         imagePullPolicy: IfNotPresent
         name: flytepropeller
         ports:
@@ -852,7 +852,7 @@ spec:
             - /etc/flyte/config/flyteadmin_config.yaml
             - clusterresource
             - sync
-            image: docker.io/lyft/flyteadmin:v0.3.0
+            image: docker.io/lyft/flyteadmin:v0.3.5
             imagePullPolicy: IfNotPresent
             name: sync-cluster-resources
             volumeMounts: