Flyte v0.17.0 milestone release

Flyte v0.17.0

Platform

1. Recovery Mode: Executions that fail due to external system failures (e.g. external system being down) can now be rerun in recovery mode (flytectl --recover docs). It's also available in the UI:

   

Flytekit

1. Great Expectations Integration (docs).
2. Access to durable blob stores (AWS/GCS/etc) are now pluggable.
3. Local task execution has been updated to also trigger the type engine.
4. Tasks that have cache=True should now be cached when running locally as well (docs).

Please see the flytekit release for the full list and more details.

UI

1. Shiny new Graph UX. The graph rendering has been revamped to be more functional and accessible. More updates are coming for better visualization for nested executions and branches.

   

2. JSON Validation for json-based types in the UI.

Before	After

1. Enum support in UI

   

FlyteCtl

1. flytectl upgrade to automatically upgrade itself (docs).
2. --dryRun is available in most commands with server-side-effects to simulate the operations before committing any changes.

And various stabilization fixes!

Fix flyte sandbox and dind docker images startup

refactor dir structure for helm (#1360)

* refactor dir structure for helm

Signed-off-by: Yuvraj <[email protected]>

Signed-off-by: Haytham Abuelfutuh <[email protected]>

Co-authored-by: Haytham Abuelfutuh <[email protected]>

Flyte v0.16.0 milestone release

Flyte v0.16.0

FlyteKit

• Redesigned Flyte Remote
• Parse out descriptions of inputs/outputs for workflows and tasks (issue)
• Improvements to exception handling
• Reference launch plan support in dynamic tasks

Please see the flytekit release for the full list and more details.

Platform

• Allow download of unlimited size in copilot (issue)
• Allow overriding of various configs in flytectl command line (e.g. --storage.type=s3) (issue)
• Allow further customization of the namespaces in which workflows run (issue)

Documentation

• Improvements to getting started.
• Contributing to Flyte guide to help understand the components and their responsibilities (available here).

And various stabilization fixes!

Flyte v0.15.1 milestone release

Flyte v0.15.1

Platform

• Updated helm chart to include missing ingress paths

Flyte v0.15.0 milestone release

Flyte v0.15.0

Platform

• Enum type support throughout the system
• Stabilization improvements for conditionals
• Support AWS Secrets Manager as a source for secrets injector
• Support Map tasks over Pod tasks
• Support max parallelism to limit how many nodes can be allowed to run in parallel.
• Add Athena flytekit plugin and examples
• Add BigQuery plugin

Flytekit

• Support Schema of Dataclasses
• Support node resource overrides

Please see the flytekit release for the full list and more details.

flytectl

• flytectl sandbox start to start a sandbox cluster locally.
• flytectl get workflow .... -o dot to visualize a workflow graph locally.
• Add Bash completion support

Update flyte docker images

v0.14.2

Update Flyte Components (#1206)

Add hack for cgroup v2 to sandbox entrypoints

Admin secrets (#1087)

* add stringdata

Signed-off-by: wild-endeavor <[email protected]>

* extra dash

Signed-off-by: wild-endeavor <[email protected]>

* wrong file

Signed-off-by: wild-endeavor <[email protected]>

* make helm

Signed-off-by: wild-endeavor <[email protected]>

Co-authored-by: Haytham Abuelfutuh <[email protected]>

Flyte v0.14.0 milestone release

Flyte v0.14.0

Platform

• Update to the Flyte Compiler, to supported nested branches and more. Better
  regression tests
• support for iam roles and k8s serviceaccounts
• customizable pod specs for pod tasks (add labels and annotations)
• copilot improvements
• Bug-fixes, visibility improvements
• support for non Cloud provider emailers - like Sendgrid
• performance improvement for dynamic workflows

Flyteconsole

• Bug fixes
• More updates coming soon

Flytekit

• Support for nested conditionals
• DoltTable and Dolt plugins and integration
• Better context management (ready for more work)
• Support for pre-built container plugins in flytekit. this makes it possible
  to library plugins and users do not need to build containers
• More control plane class features
• See full release notes
  Coming soon:
  • Great Expectations integration
  • More use case driven examples in flytesnacks

flytectl

• flytectl is ready for BETA. check it out - https://docs.flyte.org/projects/flytectl/en/latest/

Please see the flytekit release for the full list and more details.

Wait for deployments to be created using rollout before checking for availability

Fixes a race condition with checking availability status on flyte pods

Wait for deployments to be created using rollout before checking for availability

Signed-off-by: Prafulla Mahindrakar [email protected]

Flyte v0.13.0 milestone release

#979

Platform

• Oauth2 support with SingleSignOn and configuration examples for popular IDP's now available in Flyte.
  Please see the updated information and description of the feature, and the setup information
  Attention: If using Auth already - this is a BREAKING change. refer to the migration guide to update configuration to ensure Admin continues to work. (No migration needed if auth is not turned on.)

• Backend improvements to support dynamic workflow visualization (in future releases).
• Lot of features added to flytectl .
• Documentation site overhaul and redesign (more in progress)

Flytekit

The first two features should be considered beta and subject to change

• First cut of the control plane classes to replace the old Sdk... classes. These classes provide programmatic access to a Flyte backend.
• New paradigm for flytekit-only plugins
• SqlAlchemy/Dolt plugins.

Please see the flytekit release for the full list and more details.