Adds tags to doc pages (flyteorg#922)

* adds tags to doc pages Signed-off-by: Samhita Alla <[email protected]> * modify tag page title Signed-off-by: Samhita Alla <[email protected]> * add contribute tag Signed-off-by: Samhita Alla <[email protected]> * remove tags from TOC Signed-off-by: Samhita Alla <[email protected]> * reverify tags Signed-off-by: Samhita Alla <[email protected]> * add toctree Signed-off-by: Samhita Alla <[email protected]> * remove tags Signed-off-by: Samhita Alla <[email protected]> * add all tags link Signed-off-by: Samhita Alla <[email protected]> * all tags link title Signed-off-by: Samhita Alla <[email protected]> * incorporated suggestions Signed-off-by: Samhita Alla <[email protected]> Signed-off-by: Samhita Alla <[email protected]>
eapolinario · Nov 19, 2022 · 8337b64 · 8337b64
1 parent 3e245af
commit 8337b64
Show file tree

Hide file tree

Showing 107 changed files with 257 additions and 57 deletions.
diff --git a/.gitignore b/.gitignore
@@ -30,4 +30,5 @@ cookbook/release-snacks
 download-artifact/
 typescript
 .bash_history
-.venv/
+.venv/
+cookbook/docs/_tags/
diff --git a/cookbook/case_studies/bioinformatics/blast/README.rst b/cookbook/case_studies/bioinformatics/blast/README.rst
@@ -3,6 +3,8 @@
 Nucleotide Sequence Querying with BLASTX
 ----------------------------------------
 
+.. tags:: Advanced
+
 This tutorial shows how computational biology intermixes with Flyte. The problem statement we will be looking at is
 querying a nucleotide sequence against a local protein database, to identify potential homologues.
 This guide will show you how to:

diff --git a/cookbook/case_studies/feature_engineering/eda/README.md b/cookbook/case_studies/feature_engineering/eda/README.md
diff --git a/cookbook/case_studies/feature_engineering/eda/README.rst b/cookbook/case_studies/feature_engineering/eda/README.rst
@@ -1,6 +1,8 @@
 EDA, Feature Engineering, and Modeling With Papermill
 =====================================================
 
+.. tags:: Data, Jupyter, Intermediate
+
 Exploratory Data Analysis (EDA) refers to the critical process of performing initial investigations on data to discover patterns,
 spot anomalies, test hypotheses and check assumptions with the help of summary statistics and graphical representations.
 

diff --git a/cookbook/case_studies/feature_engineering/eda/notebook.py b/cookbook/case_studies/feature_engineering/eda/notebook.py
@@ -1,6 +1,6 @@
 """
 Flyte Pipeline in One Jupyter Notebook
-=======================================
+======================================
 
 In this example, we will implement a simple pipeline that takes hyperparameters, does EDA, feature engineering, and measures the Gradient
 Boosting model's performance using mean absolute error (MAE), all in one notebook.

diff --git a/cookbook/case_studies/feature_engineering/feast_integration/README.rst b/cookbook/case_studies/feature_engineering/feast_integration/README.rst
@@ -1,6 +1,8 @@
 Feast Integration
 -----------------
 
+.. tags:: Data, MachineLearning, Advanced
+
 `Feast <https://feast.dev/>`__ is an operational data system for managing and serving machine learning features to models in production.
 
 Flyte provides a way to train models and perform feature engineering as a single pipeline.

diff --git a/cookbook/case_studies/ml_training/house_price_prediction/README.md b/cookbook/case_studies/ml_training/house_price_prediction/README.md
diff --git a/cookbook/case_studies/ml_training/house_price_prediction/README.rst b/cookbook/case_studies/ml_training/house_price_prediction/README.rst
@@ -1,5 +1,7 @@
 House Price Regression
------------------------
+----------------------
+
+.. tags:: Data, MachineLearning, DataFrame, Intermediate
 
 House Price Regression refers to the prediction of house prices based on various factors, using the XGBoost Regression model (in our case).
 In this example, we will train our data on the XGBoost model to predict house prices in multiple regions.

diff --git a/...book/case_studies/ml_training/house_price_prediction/multiregion_house_price_predictor.py b/...book/case_studies/ml_training/house_price_prediction/multiregion_house_price_predictor.py
@@ -1,7 +1,7 @@
 """
 
 Predicting House Price in Multiple Regions Using XGBoost and Dynamic Workflows
--------------------------------------------------------------------------------
+------------------------------------------------------------------------------
 
 In this tutorial, we will understand how to predict house prices in multiple regions using XGBoost, and :ref:`dynamic workflows <sphx_glr_auto_core_control_flow_dynamics.py>` in Flyte.
 

diff --git a/cookbook/case_studies/ml_training/mnist_classifier/README.rst b/cookbook/case_studies/ml_training/mnist_classifier/README.rst
@@ -3,6 +3,8 @@
 MNIST Classification With PyTorch and W&B
 -----------------------------------------
 
+.. tags:: MachineLearning, GPU, Advanced
+
 PyTorch
 =======
 

diff --git a/cookbook/case_studies/ml_training/mnist_classifier/pytorch_single_node_multi_gpu.py b/cookbook/case_studies/ml_training/mnist_classifier/pytorch_single_node_multi_gpu.py
@@ -1,6 +1,6 @@
 """
 Single Node, Multi GPU Training
---------------------------------
+-------------------------------
 
 When you need to scale up model training in pytorch, you can use the :py:class:`~torch:torch.nn.DataParallel` for
 single node, multi-gpu/cpu training or :py:class:`~torch:torch.nn.parallel.DistributedDataParallel` for multi-node,

diff --git a/cookbook/case_studies/ml_training/nlp_processing/README.rst b/cookbook/case_studies/ml_training/nlp_processing/README.rst
@@ -1,6 +1,8 @@
 NLP Processing
 --------------
 
+.. tags:: MachineLearning, UI, Intermediate
+
 This tutorial will demonstrate how to process text data and generate word embeddings and visualizations
 as part of a Flyte workflow. It's an adaptation of the official Gensim `Word2Vec tutorial <https://radimrehurek.com/gensim/auto_examples/tutorials/run_word2vec.html>`__.
 

diff --git a/cookbook/case_studies/ml_training/pima_diabetes/README.rst b/cookbook/case_studies/ml_training/pima_diabetes/README.rst
@@ -1,5 +1,7 @@
 Diabetes Classification
-------------------------
+-----------------------
+
+.. tags:: MachineLearning, Intermediate
 
 The workflow demonstrates how to train an XGBoost model. The workflow is designed for the `Pima Indian Diabetes dataset <https://github.com/jbrownlee/Datasets/blob/master/pima-indians-diabetes.names>`__.
 

diff --git a/cookbook/case_studies/ml_training/pima_diabetes/diabetes.py b/cookbook/case_studies/ml_training/pima_diabetes/diabetes.py
@@ -1,6 +1,6 @@
 """
 Train and Validate a Diabetes Classification XGBoost Model
------------------------------------------------------------
+----------------------------------------------------------
 
 Watch a demo of sandbox creation and a sample execution of the pima diabetes pipeline below.
 

diff --git a/cookbook/case_studies/ml_training/spark_horovod/README.rst b/cookbook/case_studies/ml_training/spark_horovod/README.rst
@@ -1,7 +1,9 @@
 .. _spark_horovod:
 
 Forecasting Rossman Store Sales with Horovod and Spark
-----------------------------------------------------------------------
+------------------------------------------------------
+
+.. tags:: MachineLearning, Integration, Advanced
 
 The problem statement we will be looking at is forecasting sales using `rossmann store sales <https://www.kaggle.com/c/rossmann-store-sales>`__ data.
 Our example is an adaptation of the `Horovod-Spark example <https://github.com/horovod/horovod/blob/master/examples/spark/keras/keras_spark_rossmann_estimator.py>`__.

diff --git a/cookbook/core/containerization/multi_images.py b/cookbook/core/containerization/multi_images.py
@@ -4,6 +4,8 @@
 Multiple Container Images in a Single Workflow
 ----------------------------------------------
 
+.. tags:: Containerization, Intermediate
+
 When working locally, it is recommended to install all requirements of your project locally (maybe in a single virtual environment). It gets complicated when you want to deploy your code to a remote
 environment since most tasks in Flyte (function tasks) are deployed using a Docker Container.
 

diff --git a/cookbook/core/containerization/raw_container.py b/cookbook/core/containerization/raw_container.py
@@ -2,7 +2,9 @@
 .. _raw_container:
 
 Using Raw Containers
----------------------
+--------------------
+
+.. tags:: Containerization, Advanced
 
 This example demonstrates how to use arbitrary containers in 5 different languages, all orchestrated in flytekit seamlessly.
 Flyte mounts an input data volume where all the data needed by the container is available and an output data volume

diff --git a/cookbook/core/containerization/spot_instances.py b/cookbook/core/containerization/spot_instances.py
@@ -2,6 +2,8 @@
 Using Spot/Preemptible Instances
 --------------------------------
 
+.. tags:: AWS, GCP, Intermediate
+
 """
 # %%
 # What Are Spot/Preemptible Instances?

diff --git a/cookbook/core/containerization/use_secrets.py b/cookbook/core/containerization/use_secrets.py
@@ -6,6 +6,8 @@
 Using Secrets in a Task
 =======================
 
+.. tags:: Kubernetes, Intermediate
+
 Flyte supports running a variety of tasks, from containers to SQL queries and service calls. For Flyte-run
 containers to request and access secrets, Flyte provides a native Secret construct.
 

diff --git a/cookbook/core/containerization/workflow_labels_annotations.py b/cookbook/core/containerization/workflow_labels_annotations.py
@@ -2,6 +2,8 @@
 Adding Workflow Labels and Annotations
 --------------------------------------
 
+.. tags:: Kubernetes, Intermediate
+
 In Flyte, workflow executions are created as Kubernetes resources. These can be extended with
 `labels <https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/>`__ and
 `annotations <https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/>`__.

diff --git a/cookbook/core/control_flow/chain_entities.py b/cookbook/core/control_flow/chain_entities.py
@@ -2,6 +2,8 @@
 Chain Flyte Entities
 --------------------
 
+.. tags:: Basic
+
 Data passing between tasks or workflows need not necessarily happen through parameters.
 In such a case, if you want to explicitly construct the dependency, flytekit provides a mechanism to chain Flyte entities using the ``>>`` operator.
 

diff --git a/cookbook/core/control_flow/checkpoint.py b/cookbook/core/control_flow/checkpoint.py
@@ -1,6 +1,8 @@
 """
 Intratask Checkpoints
-----------------------
+---------------------
+
+.. tags:: MachineLearning, Intermediate
 
 .. note::
 

diff --git a/cookbook/core/control_flow/conditions.py b/cookbook/core/control_flow/conditions.py
@@ -1,6 +1,9 @@
 """
 Conditions
---------------
+----------
+
+.. tags:: Intermediate
+
 Flytekit supports conditions as a first class construct in the language. Conditions offer a way to selectively execute
 branches of a workflow based on static or dynamic data produced by other tasks or come in as workflow inputs.
 Conditions are very performant to be evaluated. However, they are limited to certain binary and logical operators and can

diff --git a/cookbook/core/control_flow/dynamics.py b/cookbook/core/control_flow/dynamics.py
@@ -3,6 +3,8 @@
 Dynamic Workflows
 =================
 
+.. tags:: Intermediate
+
 A workflow is typically static when the directed acyclic graph's (DAG) structure is known at compile-time.
 However, in cases where a run-time parameter (for example, the output of an earlier task) determines the full DAG structure, you can use dynamic workflows by decorating a function with ``@dynamic``.
 

diff --git a/cookbook/core/control_flow/map_task.py b/cookbook/core/control_flow/map_task.py
@@ -2,6 +2,8 @@
 Map Tasks
 ---------
 
+.. tags:: Intermediate
+
 A map task lets you run a pod task or a regular task over a list of inputs within a single workflow node.
 This means you can run thousands of instances of the task without creating a node for every instance, providing valuable performance gains!
 

diff --git a/cookbook/core/control_flow/merge_sort.py b/cookbook/core/control_flow/merge_sort.py
@@ -2,7 +2,9 @@
 .. _advanced_merge_sort:
 
 Implementing Merge Sort
-------------------------
+-----------------------
+
+.. tags:: Intermediate
 
 FlyteIdl (the fundamental building block of the Flyte Language) allows various programming language features:
 conditionals, recursion, custom typing, and more.

diff --git a/cookbook/core/control_flow/subworkflows.py b/cookbook/core/control_flow/subworkflows.py
@@ -4,6 +4,8 @@
 SubWorkflows
 ------------
 
+.. tags:: Intermediate
+
 Subworkflows are similar to :ref:`launch plans <Launch plans>` since they allow users to kick off one workflow from within another.
 
 What's the Difference?

diff --git a/cookbook/core/extend_flyte/backend_plugins.py b/cookbook/core/extend_flyte/backend_plugins.py
@@ -1,9 +1,11 @@
 """
 .. _extend-plugin-flyte-backend:
 
-###############################
+##########################
 Writing Backend Extensions
-###############################
+##########################
+
+.. tags:: Extensibility, Contribute, Intermediate
 
 Now that you have landed here, we can assume that you have exhausted your options of extending and want to extend Flyte in a way that adds new capabilities to the platform.
 

diff --git a/cookbook/core/extend_flyte/container_interface.py b/cookbook/core/extend_flyte/container_interface.py
@@ -4,6 +4,8 @@
 Container Interface
 -------------------
 
+.. tags:: Extensibility, Contribute, Intermediate
+
 Flyte typically interacts with containers in the course of its task execution (since most tasks are container
 tasks). This is what that process looks like:
 

diff --git a/cookbook/core/extend_flyte/custom_types.py b/cookbook/core/extend_flyte/custom_types.py
@@ -4,6 +4,8 @@
 Writing Custom Flyte Types
 --------------------------
 
+.. tags:: Extensibility, Contribute, Intermediate
+
 Flyte is a strongly-typed framework for authoring tasks and workflows. But there are situations when the existing
 types do not directly work. This is true with any programming language!
 

diff --git a/cookbook/core/extend_flyte/prebuilt_container.py b/cookbook/core/extend_flyte/prebuilt_container.py
@@ -4,6 +4,8 @@
 Pre-built Container Task Plugin
 -------------------------------
 
+.. tags:: Extensibility, Contribute, Intermediate
+
 A pre-built container task plugin runs a pre-built container. The following are the advantages of using a pre-built container in comparison to a user-defined container:
 
 - Shifts the burden of writing Dockerfile from the user who uses the task in workflows to the author of the task type.

diff --git a/cookbook/core/extend_flyte/user_container.py b/cookbook/core/extend_flyte/user_container.py
@@ -4,6 +4,8 @@
 User Container Task Plugin
 --------------------------
 
+.. tags:: Extensibility, Contribute, Intermediate
+
 A user container task plugin runs a user-defined container that has the user code.
 
 This tutorial will walk you through writing your own sensor-style plugin that allows users to wait for a file to land

diff --git a/cookbook/core/flyte_basics/basic_workflow.py b/cookbook/core/flyte_basics/basic_workflow.py
@@ -1,6 +1,8 @@
 """
 Workflows
-----------
+---------
+
+.. tags:: Basic
 
 Once you have a handle on :ref:`tasks <basics_of_tasks>`, you can dive into Flyte workflows.
 Together, Flyte tasks and workflows make up the fundamental building blocks of Flyte.

diff --git a/cookbook/core/flyte_basics/deck.py b/cookbook/core/flyte_basics/deck.py
@@ -1,6 +1,8 @@
 """
 Flyte Decks
--------------
+-----------
+
+.. tags:: UI, Basic
 
 Deck enables users to get customizable and default visibility into their tasks.
 

diff --git a/cookbook/core/flyte_basics/decorating_tasks.py b/cookbook/core/flyte_basics/decorating_tasks.py
@@ -2,6 +2,8 @@
 Decorating Tasks
 ----------------
 
+.. tags:: Intermediate
+
 A simple way of modifying the behavior of tasks is by using decorators to wrap your task functions.
 
 In order to make sure that your decorated function contains all the type annotation and docstring

diff --git a/cookbook/core/flyte_basics/decorating_workflows.py b/cookbook/core/flyte_basics/decorating_workflows.py
@@ -2,6 +2,8 @@
 Decorating Workflows
 --------------------
 
+.. tags:: Intermediate
+
 The behavior of workflows can be modified in a light-weight fashion by using the built-in :py:func:`~functools.wraps`
 decorator pattern, similar to using decorators to
 :ref:`customize task behavior <sphx_glr_auto_core_flyte_basics_decorating_tasks.py>`. However, unlike in the case of

diff --git a/cookbook/core/flyte_basics/documented_workflow.py b/cookbook/core/flyte_basics/documented_workflow.py
@@ -2,6 +2,8 @@
 Add Docstrings to Workflows
 ---------------------------
 
+.. tags:: Basic
+
 Documented code helps enhance the readability of the code. Flyte supports docstrings to document your code.
 Docstrings are stored in FlyteAdmin and shown on the UI in the launch form.
 

diff --git a/cookbook/core/flyte_basics/files.py b/cookbook/core/flyte_basics/files.py
@@ -1,6 +1,8 @@
 """
 Working With Files
--------------------
+------------------
+
+.. tags:: Data, Basic
 
 Files are one of the most fundamental entities that users of Python work with,
 and they are fully supported by Flyte. In the IDL, they are known as

diff --git a/cookbook/core/flyte_basics/folders.py b/cookbook/core/flyte_basics/folders.py
@@ -1,6 +1,8 @@
 """
 Working With Folders
----------------------
+--------------------
+
+.. tags:: Data, Basic
 
 In addition to files, folders are another fundamental operating system primitive users often work with. Flyte supports folders
 in the form of `multi-part blobs <https://github.com/flyteorg/flyteidl/blob/237f025a15c0102675ad41d2d1e66869afa80822/protos/flyteidl/core/types.proto#L73>`__.

diff --git a/cookbook/core/flyte_basics/hello_world.py b/cookbook/core/flyte_basics/hello_world.py
@@ -1,7 +1,10 @@
 """
 
 Hello World
-------------
+-----------
+
+.. tags:: Basic
+
 This simple workflow calls a task that returns "Hello World" and then just sets that as the final output of the workflow.
 """
 import typing

diff --git a/cookbook/core/flyte_basics/imperative_wf_style.py b/cookbook/core/flyte_basics/imperative_wf_style.py
@@ -4,6 +4,8 @@
 Imperative Workflows
 --------------------
 
+.. tags:: Basic
+
 Workflows are typically created and specified by decorating a function with the ``@workflow`` decorator. This will
 run through the body of the function at compile time, using the subsequent calls of the underlying tasks to determine
 and record the workflow structure. This is the declarative style and makes sense when a human is writing it up by hand.

diff --git a/cookbook/core/flyte_basics/lp.py b/cookbook/core/flyte_basics/lp.py
@@ -2,6 +2,8 @@
 Launch Plans
 ------------
 
+.. tags:: Basic
+
 Launch plans bind a partial or complete list of inputs necessary to launch a workflow, along
 with optional run-time overrides such as notifications, schedules and more.
 Launch plan inputs must only assign inputs already defined in the reference workflow definition.

diff --git a/cookbook/core/flyte_basics/named_outputs.py b/cookbook/core/flyte_basics/named_outputs.py
@@ -2,6 +2,8 @@
 Named Outputs
 -------------
 
+.. tags:: Basic
+
 By default, Flyte names the outputs of a task or workflow using a standardized convention. All the outputs are named
 as ``o1, o2, o3, ... o<n>.`` where ``o`` is the standard prefix and ``1, 2, .. <n>`` is the index position within the return values.
-Original file line number
+Diff line change
@@ Expand Up / @@ -3,6 +3,8 @@ @@
     MNIST Classification With PyTorch and W&B
     -----------------------------------------
+    .. tags:: MachineLearning, GPU, Advanced
     PyTorch
     =======
@@ Expand Down @@