diff --git a/README.md b/README.md index 775419d602..cf6f28f36d 100644 --- a/README.md +++ b/README.md @@ -61,13 +61,13 @@ ## 💥 Introduction -Flyte is a structured programming and distributed processing platform that enables highly concurrent, scalable and maintainable workflows for `Machine Learning` and `Data Processing`. It is a fabric that connects disparate computation backends using a type safe data dependency graph. It records all changes to a pipeline, making it possible to rewind time. It also stores -a history of all executions and provides an intuitive UI, CLI and REST/gRPC API to interact with the computation. +Flyte is a structured programming and distributed processing platform that enables highly concurrent, scalable, and maintainable workflows for `Machine Learning` and `Data Processing`. It is a fabric that connects disparate computation backends using a type-safe data dependency graph. It records all changes to a pipeline, making it possible to rewind time. It also stores +a history of all executions and provides an intuitive UI, CLI, and REST/gRPC API to interact with the computation. -Flyte is more than a workflow engine -- it uses a `workflow` as a core concept and a `task` (a single unit of execution) as a top level concept. Multiple tasks arranged in a data -producer-consumer order create a workflow. +Flyte is more than a workflow engine -- it uses `workflow` as a core concept and `task` (a single unit of execution) as a top-level concept. Multiple tasks arranged in a data +producer-consumer order creates a workflow. -`Workflows` and `Tasks` can be written in any language, with out of the box support for [Python](https://github.com/flyteorg/flytekit), [Java and Scala](https://github.com/spotify/flytekit-java). +`Workflows` and `Tasks` can be written in any language, with out-of-the-box support for [Python](https://github.com/flyteorg/flytekit), [Java and Scala](https://github.com/spotify/flytekit-java). ## ⏳ Five Reasons to Use Flyte @@ -120,7 +120,7 @@ To dig deeper into Flyte, refer to the [Documentation](https://docs.flyte.org/en - Used at _Scale_ in production by **500+** users at Lyft with more than **1 million** executions and **40+ million** container executions per month -- A data aware platform +- A data-aware platform - Enables **collaboration across your organization** by: - Executing distributed data pipelines/workflows - Reusing tasks across projects, users, and workflows @@ -130,9 +130,9 @@ To dig deeper into Flyte, refer to the [Documentation](https://docs.flyte.org/en - Sharing workflows and tasks across your teams - Simplifying the complexity of multi-step, multi-owner workflows - **[Quick registration](https://docs.flyte.org/en/latest/getting_started.html)** -- start locally and scale to the cloud instantly -- **Centralized Inventory** constituting Tasks, Workflows and Executions +- **Centralized Inventory** constituting Tasks, Workflows, and Executions - **gRPC / REST** interface to define and execute tasks and workflows -- **Type safe** construction of pipelines -- each task has an interface which is characterized by its input and output, so illegal construction of pipelines fails during declaration rather than at runtime +- **Type safe** construction of pipelines -- each task has an interface that is characterized by its input and output, so illegal construction of pipelines fails during declaration rather than at runtime - Supports multiple **[data types](https://docs.flyte.org/projects/cookbook/en/latest/auto/type_system/index.html)** for machine learning and data processing pipelines, such as Blobs (images, arbitrary files), Directories, Schema (columnar structured data), collections, maps, etc. - Memoization and Lineage tracking - Provides logging and observability @@ -141,17 +141,17 @@ To dig deeper into Flyte, refer to the [Documentation](https://docs.flyte.org/en - Parallel step execution - Extensible backend to add **[customized plugin](https://docs.flyte.org/projects/cookbook/en/latest/auto/core/extend_flyte/custom_task_plugin.html)** experience (with simplified user experience) - **[Branching](https://docs.flyte.org/projects/cookbook/en/latest/auto/core/control_flow/run_conditions.html)** - - Inline **[subworkflows](https://docs.flyte.org/projects/cookbook/en/latest/auto/core/control_flow/subworkflows.html)** (a workflow can be embeded within one node of the top level workflow) + - Inline **[subworkflows](https://docs.flyte.org/projects/cookbook/en/latest/auto/core/control_flow/subworkflows.html)** (a workflow can be embedded within one node of the top-level workflow) - Distributed **remote child workflows** (a remote workflow can be triggered and statically verified at compile time) - **[Array Tasks](https://docs.flyte.org/projects/cookbook/en/latest/auto/core/control_flow/map_task.html)** (map a function over a large dataset -- ensures controlled execution of thousands of containers) - **[Dynamic workflow](https://docs.flyte.org/projects/cookbook/en/latest/auto/core/control_flow/dynamics.html)** creation and execution with runtime type safety - - Container side [plugins](https://docs.flyte.org/projects/cookbook/en/latest/plugins.html) with first class support in Python + - Container side [plugins](https://docs.flyte.org/projects/cookbook/en/latest/plugins.html) with first-class support in Python - _PreAlpha_: Arbitrary flytekit-less containers supported ([RawContainer](https://docs.flyte.org/projects/cookbook/en/latest/auto/core/containerization/raw_container.html)) - Guaranteed **[reproducibility](https://docs.flyte.org/projects/cookbook/en/latest/auto/core/flyte_basics/task_cache.html)** of pipelines via: - - Versioned data, code and models + - Versioned data, code, and models - Automatically tracked executions - Declarative pipelines -- **Multi cloud support** (AWS, GCP and others) +- **Multi-cloud support** (AWS, GCP, and others) - Extensible core, modularized, and deep observability - No single point of failure and is resilient by design - Automated notifications to Slack, Email, and Pagerduty @@ -236,8 +236,8 @@ Here are some resources to help you learn more about Flyte. ### Biweekly Community Sync -- 📣 **Flyte OSS Community Sync** Every other Tuesday, 9am-10am PDT. Checkout the [calendar](https://www.addevent.com/calendar/kE355955) and register to stay up-to-date with our meeting times. Or simply join us on [Zoom](https://us04web.zoom.us/j/71298741279?pwd=TDR1RUppQmxGaDRFdzBOa2lHN1dsZz09). -- Upcoming meeting agenda, previous meeting notes and a backlog of topics are captured in this [document](https://docs.google.com/document/d/1Jb6eOPOzvTaHjtPEVy7OR2O5qK1MhEs3vv56DX2dacM/edit#heading=h.c5ha25xc546e). +- 📣 **Flyte OSS Community Sync** Every other Tuesday, 9am-10am PDT. Check out the [calendar](https://www.addevent.com/calendar/kE355955) and register to stay up-to-date with our meeting times. Or simply join us on [Zoom](https://us04web.zoom.us/j/71298741279?pwd=TDR1RUppQmxGaDRFdzBOa2lHN1dsZz09). +- Upcoming meeting agenda, previous meeting notes, and a backlog of topics are captured in this [document](https://docs.google.com/document/d/1Jb6eOPOzvTaHjtPEVy7OR2O5qK1MhEs3vv56DX2dacM/edit#heading=h.c5ha25xc546e). - If you'd like to revisit any previous community sync meetings, you can access the video recordings on [Flyte's YouTube channel](https://www.youtube.com/channel/UCNduEoLOToNo3nFVly-vUTQ). ### Conference Talks