Plugin READMEs (flyteorg#723)

Signed-off-by: Robert Everson <[email protected]>

plugins/flytekit-aws-athena/README.md

@@ -0,0 +1,11 @@
+# Flytekit AWS Athena Plugin
+
+Flyte backend can be connected with Athena. Once enabled, it allows you to query AWS Athena service (Presto + ANSI SQL Support) and retrieve typed schema (optionally).
+
+To install the plugin, run the following command:
+
+```bash
+pip install flytekitplugins-athena
+```
+
+An [example](https://docs.flyte.org/projects/cookbook/en/latest/auto/integrations/aws/athena/athena.html#sphx-glr-auto-integrations-aws-athena-athena-py) can be found in the documentation.

plugins/flytekit-aws-sagemaker/README.md

@@ -0,0 +1,13 @@
+# Flytekit AWS Sagemaker Plugin
+
+Amazon SageMaker provides several built-in machine learning algorithms that you can use for a variety of problem types. Flyte Sagemaker plugin intends to greatly simplify using Sagemaker for training. We have tried to distill the API into a meaningful subset that makes it easier for users to adopt and run with Sagemaker.
+
+To install the plugin, run the following command:
+
+```bash
+pip install flytekitplugins-awssagemaker
+```
+
+To install Sagemaker in the Flyte deployment's backend, go through the [prerequisites](https://docs.flyte.org/projects/cookbook/en/latest/auto/integrations/aws/sagemaker_training/index.html#prerequisites).
+
+[Built-in sagemaker](https://docs.flyte.org/projects/cookbook/en/latest/auto/integrations/aws/sagemaker_training/sagemaker_builtin_algo_training.html#sphx-glr-auto-integrations-aws-sagemaker-training-sagemaker-builtin-algo-training-py) and [custom sagemaker](https://docs.flyte.org/projects/cookbook/en/latest/auto/integrations/aws/sagemaker_training/sagemaker_custom_training.html#sphx-glr-auto-integrations-aws-sagemaker-training-sagemaker-custom-training-py) training models can be found in the documentation.

plugins/flytekit-data-fsspec/README.md

@@ -1,6 +1,5 @@
-fsspec data plugin for flytekit - Experimental
-=================================================
+# fsspec data plugin for Flytekit — Experimental
 
-This plugin provides an implementation of the data persistence layer in flytekit, that uses fsspec. Once this plugin
-is installed, it overrides all default implementation of dataplugins and provides ones supported by fsspec. this plugin
-will only install the fsspec core. To actually install all fsspec plugins, please follow fsspec documentation.
+This plugin provides an implementation of the data persistence layer in Flytekit that uses fsspec. Once this plugin
+is installed, it overrides all default implementations of the data plugins and provides the ones supported by fsspec. This plugin
+will only install the fsspec core. To install all fsspec plugins, please follow the [fsspec documentation](https://filesystem-spec.readthedocs.io/en/latest/).

plugins/flytekit-dolt/README.md

@@ -0,0 +1,19 @@
+# Flytekit Dolt Plugin
+
+The DoltTable plugin is a wrapper that uses [Dolt](https://github.com/dolthub/dolt) to move data between pandas.DataFrame’s at execution time and database tables at rest.
+
+The Dolt plugin and Dolt command-line tool can be installed as follows:
+
+```bash
+pip install flytekitplugins.dolt
+sudo bash -c 'curl -L https://github.com/dolthub/dolt/releases/latest/download/install.sh | sudo bash'
+```
+
+Dolt requires a user configuration to run init:
+
+```
+dolt config --global --add user.email <email>
+dolt config --global --add user.name <name>
+```
+
+All the [examples](https://docs.flyte.org/projects/cookbook/en/latest/auto/integrations/flytekit_plugins/dolt/index.html) can be found in the documentation.

plugins/flytekit-greatexpectations/README.md

@@ -1,6 +1,6 @@
 # Flytekit Great Expectations Plugin
 
-Great Expectations' helps enforce data quality. The plugin supports the usage of Great Expectations as task and type.
+Great Expectations helps enforce data quality. The plugin supports the usage of Great Expectations as task and type.
 
 To install the plugin, run the following command:
 
@@ -72,3 +72,5 @@ def simple_task(
 def simple_wf(directory: str = "my_assets") -> str:
     return simple_task(directory=directory)
 ```
+
+[More examples](https://docs.flyte.org/projects/cookbook/en/latest/auto/integrations/flytekit_plugins/greatexpectations/index.html) can be found in the documentation.

plugins/flytekit-hive/README.md

@@ -0,0 +1,11 @@
+# Flytekit Hive Plugin
+
+Flyte backend can be connected with various Hive services. Once enabled, it allows you to query a Hive service (e.g., Qubole) and retrieve typed schema (optionally).
+
+To install the plugin, run the following command:
+
+```bash
+pip install flytekitplugins-hive
+```
+
+An [example](https://docs.flyte.org/projects/cookbook/en/latest/auto/integrations/external_services/hive/hive.html#sphx-glr-auto-integrations-external-services-hive-hive-py) can be found in the documentation.

plugins/flytekit-k8s-pod/README.md

@@ -0,0 +1,13 @@
+# Flytekit Kubernetes Pod Plugin
+
+By default, Flyte tasks decorated with `@task` are essentially single functions that are loaded in one container. But often, there is a need to run a job with more than one container.
+
+In this case, a regular task is not enough. Hence, Flyte provides a Kubernetes pod abstraction to execute multiple containers, which can be accomplished using Pod's `task_config`. The `task_config` can be leveraged to fully customize the pod spec used to run the task.
+
+To install the plugin, run the following command:
+
+```bash
+pip install flytekitplugins-pod
+```
+
+An [example](https://docs.flyte.org/projects/cookbook/en/latest/auto/integrations/kubernetes/pod/pod.html) can be found in the documentation.

plugins/flytekit-kf-pytorch/README.md

@@ -0,0 +1,13 @@
+# Flytekit Kubeflow PyTorch Plugin
+
+This plugin uses the Kubeflow PyTorch Operator and provides an extremely simplified interface for executing distributed training using various PyTorch backends.
+
+To install the plugin, run the following command:
+
+```bash
+pip install flytekitplugins-kfpytorch
+```
+
+To set up PyTorch operator in the Flyte deployment's backend, follow the [PyTorch Operator Setup](https://docs.flyte.org/en/latest/deployment/plugin_setup/pytorch_operator.html) guide.
+
+An [example](https://docs.flyte.org/projects/cookbook/en/latest/auto/integrations/kubernetes/kfpytorch/pytorch_mnist.html#sphx-glr-auto-integrations-kubernetes-kfpytorch-pytorch-mnist-py) showcasing PyTorch operator can be found in the documentation.

plugins/flytekit-kf-tensorflow/README.md

@@ -0,0 +1,11 @@
+# Flytekit Kubeflow TensorFlow Plugin
+
+This plugin uses the Kubeflow TensorFlow Operator and provides an extremely simplified interface for executing distributed training using various TensorFlow backends.
+
+To install the plugin, run the following command:
+
+```bash
+pip install flytekitplugins-kftensorflow
+```
+
+_Example coming soon!_

plugins/flytekit-modin/README.md

@@ -0,0 +1,11 @@
+# Flytekit Modin Plugin
+
+Modin is an emerging drop-in replacement or rather extension of Pandas. This plugin could be helpful to use Modin as a data type.
+
+To install the plugin, run the following command:
+
+```bash
+pip install flytekitplugins-modin
+```
+
+_Example coming soon!_

plugins/flytekit-pandera/README.md

@@ -0,0 +1,11 @@
+# Flytekit Pandera Plugin
+
+Flytekit python natively supports many data types, including a FlyteSchema type for type-annotating Pandas DataFrames. The Flytekit Pandera plugin provides an alternative for defining DataFrame schemas by integrating with Pandera, which is a runtime data validation tool for Pandas DataFrames.
+
+To install the plugin, run the following command:
+
+```bash
+pip install flytekitplugins-pandera
+```
+
+All [examples](https://docs.flyte.org/projects/cookbook/en/latest/auto/integrations/flytekit_plugins/pandera_examples/index.html) can be found in the documentation.

plugins/flytekit-papermill/README.md

@@ -0,0 +1,11 @@
+# Flytekit Papermill Plugin
+
+It is possible to run a Jupyter notebook as a Flyte task using Papermill. Papermill executes the notebook as a whole, so before using this plugin, it is essential to construct your notebook as recommended by Papermill.
+
+To install the plugin, run the following command:
+
+```bash
+pip install flytekitplugins-papermill
+```
+
+An [example](https://docs.flyte.org/projects/cookbook/en/latest/auto/integrations/flytekit_plugins/papermilltasks/simple.html#sphx-glr-auto-integrations-flytekit-plugins-papermilltasks-simple-py) can be found in the documentation. We also have a [tutorial](https://docs.flyte.org/projects/cookbook/en/latest/auto/case_studies/feature_engineering/eda/index.html) showcasing the various ways in which you can leverage the Papermill plugin.

plugins/flytekit-snowflake/README.md

@@ -0,0 +1,13 @@
+# Flytekit Snowflake Plugin
+
+Snowflake enables us to build data-intensive applications without operational burden. Flyte backend can be connected with the Snowflake service. Once enabled, it can allow you to query a Snowflake service.
+
+To install the plugin, run the following command:
+
+```bash
+pip install flytekitplugins-snowflake
+```
+
+To configure Snowflake in the Flyte deployment's backend, follow the [configuration guide](https://docs.flyte.org/projects/cookbook/en/latest/auto/integrations/external_services/snowflake/index.html#configuring-the-backend-to-get-snowflake-working).
+
+An [example](https://docs.flyte.org/projects/cookbook/en/latest/auto/integrations/external_services/snowflake/snowflake.html#sphx-glr-auto-integrations-external-services-snowflake-snowflake-py) showcasing Snowflake service can be found in the documentation.

plugins/flytekit-spark/README.md

@@ -0,0 +1,13 @@
+# Flytekit Spark Plugin
+
+Flyte can execute Spark jobs natively on a Kubernetes Cluster, which manages a virtual cluster’s lifecycle, spin-up, and tear down. It leverages the open-sourced Spark On K8s Operator and can be enabled without signing up for any service. This is like running a transient spark cluster — a type of cluster spun up for a specific Spark job and torn down after completion.
+
+To install the plugin, run the following command:
+
+```bash
+pip install flytekitplugins-spark
+```
+
+To configure Spark in the Flyte deployment's backend, follow [Step 1](https://docs.flyte.org/projects/cookbook/en/latest/auto/integrations/kubernetes/k8s_spark/index.html#step-1-deploy-spark-plugin-in-the-flyte-backend), [2](https://docs.flyte.org/projects/cookbook/en/latest/auto/integrations/kubernetes/k8s_spark/index.html#step-2-environment-setup), and [3](https://docs.flyte.org/projects/cookbook/en/latest/auto/integrations/kubernetes/k8s_spark/index.html#step-3-optionally-setup-visibility).
+
+All [examples](https://docs.flyte.org/projects/cookbook/en/latest/auto/integrations/kubernetes/k8s_spark/index.html) showcasing execution of Spark jobs using the plugin can be found in the documentation.

plugins/flytekit-sqlalchemy/README.md

@@ -0,0 +1,11 @@
+# Flytekit SQLAlchemy Plugin
+
+SQLAlchemy is the Python SQL toolkit and Object Relational Mapper that gives application developers the full power and flexibility of SQL. Flyte provides an easy-to-use interface to utilize SQLAlchemy to connect to various SQL Databases.
+
+To install the plugin, run the following command:
+
+```bash
+pip install flytekitplugins-sqlalchemy
+```
+
+An [example](https://docs.flyte.org/projects/cookbook/en/latest/auto/integrations/flytekit_plugins/sql/sql-alchemy.html) can be found in the documentation.