Flyte v0.18.1 milestone release

0.18.1 Release ChangeLog

Closed Issues

UX

FlyteKit

See the flytekit 0.24.0 release for the full list of changes. Here are some of the highlights:

1. We've added the following models to the top-level of the flytekit package. The classes will not move from their original locations, but importing from the top level will always be safer as we promise not to break these.

   - Annotations, AuthRole, Labels
   - WorkflowExecutionPhase
   - Literal, LiteralType, Scalar, BlobType, Blob, BlobMetadata

   Instead of from flytekit.models.common import Labels, Annotations, please now do

   from flytekit import Labels, Annotations

2. Support for python pickle. Starting in this release, flytekit is going to pickle inputs and outputs for types which it doesn't have a specific transformer for. This brings a lot of more freedom in porting over code to flyte's model, since it won't force users to write a type transformer in order to use existing code. Keep in mind that all the caveats around pickling code apply in this case.

3. We added a cookiecutter template, simplifying the Getting started docs and also unlocking the path to cookiecutter templates for specific use-cases, e.g. pytorch-enabled samples, etc.

4. Faster installation in Apple M1 Macs. We're now requiring pyarrow 6.0, which contains prebuilt wheels for the M1.

FlyteConsole

1. Added new UI for Workflow details including execution bar chart.
2. Added new bar chart user-selected filter for workflow executions.
3. Added new launch form controls ("Advanced Options")
4. Minor bug fixes

System

1. Performance improvements for executions.
   1. Smaller workflow CRDs,
   2. Better handling of partial failures in large fanout scenarios,
2. All flyte containers now run as non-root users. Docs (Thanks @frsann)
3. Stability and bug fixes

Documentation

1. Plugin Setup Docs (e.g. MPI, Tensorflow, Spark Operators, AWS Batch & Athena, Snowflake and Google BigQuery)

Sandbox

1. Fixed race condition in minio pod initialization.

Flyte v0.18.0 milestone release

Flyte v0.18.0

Platform

1. Flyte Native scheduler
2. Support for Snowflake including backend plugin and flytekit tasks
3. Expose default MaxParallelism in flyteadmin configuration
4. Support custom resource cleanup policy in backend plugins
5. Improved error message in the case of images with invalid names
6. Added default buckets in sandbox helm
7. Fixed bug in helm chart propeller config. logs url will not update correctly
8. Added cluster-admin read only roles for kubernetes dashboard
9. Updated helm chart to include minio & kubernetes Dashboard nodePort

flytekit

1. Continued changes to the FlyteRemote interface, including:
   • remote.sync now operates on the object in-place.
   • A sync_nodes argument has been added to the remote.sync call - by default it's True but set it to False if you want to only render inputs/outputs at the top level (rather than at every node within the execution).
   • The sync call on FlyteWorkflowExecution objects was removed (it was already deprecated and a noop but if you were calling it, you'll need to remove it.)
   • gRPC credentials to FlyteRemote
2. Improved typing errors when scanning user code. Added better type inference to some of the transformers.
3. Plugin tests have been moved into each individual plugin's test folder.
4. Snowflake task has been added
5. Fixes to SQLAlchemy task secrets handling
6. Modin schema transformer

Please see the flytekit release for the full list and more details.

UI

1. Additional information when a Task in a non-terminal state
2. Support for workflow versions

flytectl

1. Sandbox docker images can now be provided as a parameter
2. Bug fixes:
   • panics in calls to get execution details and launchplans.
   • datetime format generated in execFile are now valid

Flyte v0.17.3 milestone release

Flyte v0.17.3

Platform

1. Flyte Native scheduler
2. Support for Snowflake including backend plugin and flytekit tasks
3. Expose default MaxParallelism in flyteadmin configuration
4. Support custom resource cleanup policy in backend plugins
5. Improved error message in the case of images with invalid names

flytekit

1. Continued changes to the FlyteRemote interface, including:
   • remote.sync now operates on the object in-place.
   • A sync_nodes argument has been added to the remote.sync call - by default it's True but set it to False if you want to only render inputs/outputs at the top level (rather than at every node within the execution).
   • The sync call on FlyteWorkflowExecution objects was removed (it was already deprecated and a noop but if you were calling it, you'll need to remove it.)
   • gRPC credentials to FlyteRemote
2. Improved typing errors when scanning user code. Added better type inference to some of the transformers.
3. Plugin tests have been moved into each individual plugin's test folder.
4. Snowflake task has been added
5. Fixes to SQLAlchemy task secrets handling
6. Modin schema transformer

Please see the flytekit release for the full list and more details.

UI

1. Additional information when a Task in a non-terminal state
2. Support for workflow versions

flytectl

1. Sandbox docker images can now be provided as a parameter
2. Bug fixes:
   • panics in calls to get execution details and launchplans.
   • datetime format generated in execFile are now valid

Flyte v0.17.3-b4 milestone release

Flyte v0.17.3-b4

Final test before releasing 0.17.3.

Flyte v0.17.3-b3 milestone release

Flyte v0.17.3-b3

Change Admin event version to 2

Flyte v0.17.3-b2 milestone release

Flyte v0.17.3-b2

Change additional Admin volumes and volume mounts to list

Flyte v0.17.3-b1 milestone release

Flyte v0.17.3-b1

Just picking up some minor changes around the admin deployment.yaml file

Update sandbox image with latest versions

Update Flyte Components (#1473)

Signed-off-by: Flyte-Bot <[email protected]>

Co-authored-by: EngHabu <[email protected]>

Flyte v0.17.2 milestone release

Flyte v0.17.2

Platform

• Added default buckets in sandbox helm
• Fixed bug in helm chart propeller config. logs url will not update correctly
• Added cluster-admin read only roles for kubernetes dashboard

Flyte v0.17.1 milestone release

Flyte v0.17.1

Platform

• Updated helm chart to include minio & kubernetes Dashboard nodePort