From 42f818a96b7d1cb0d892f09f51343804cdf3cb77 Mon Sep 17 00:00:00 2001 From: pmahindrakar-oss <77798312+pmahindrakar-oss@users.noreply.github.com> Date: Thu, 20 May 2021 08:40:51 +0530 Subject: [PATCH] Added intersphinx links to IDL docs (#223) Signed-off-by: Prafulla Mahindrakar --- cookbook/core/extend_flyte/backend_plugins.rst | 8 ++++---- cookbook/docs/conf.py | 1 + cookbook/docs/extend_flyte_backend_plugins.rst | 8 ++++---- 3 files changed, 9 insertions(+), 8 deletions(-) diff --git a/cookbook/core/extend_flyte/backend_plugins.rst b/cookbook/core/extend_flyte/backend_plugins.rst index 6658779bcf..44f8cd034e 100644 --- a/cookbook/core/extend_flyte/backend_plugins.rst +++ b/cookbook/core/extend_flyte/backend_plugins.rst @@ -23,9 +23,9 @@ We will try to understand the components of a backend plugin using an example pl Interface specification ------------------------ -Usually Flyte extensions need information that is not covered by a :std:ref:`Flyte TaskTemplate `. The TaskTemplate consists of a -the interface, task_type identifier, some metadata and other fields. An important field to note here is - :std:ref:`api_field_flyteidl.core.tasktemplate.custom`. The custom field is essentially an unstructured JSON. -This makes it possible to extend a task-template beyond the default supported targets -- :std:ref:`api_field_flyteidl.core.tasktemplate.container` (WIP, sql etc). +Usually Flyte extensions need information that is not covered by a :std:ref:`Flyte TaskTemplate `. The TaskTemplate consists of a +the interface, task_type identifier, some metadata and other fields. An important field to note here is - :std:ref:`custom `. The custom field is essentially an unstructured JSON. +This makes it possible to extend a task-template beyond the default supported targets -- :std:ref:`container ` (WIP, sql etc). The motivation of the Custom field, is to marshal a JSON structure that specifies information beyond what a regular TaskTemplate can capture. The actual structure of the JSON is known only to the implemented backend-plugin and the SDK components. The core Flyte platform, does not understand of look into the specifics of this structure. It is highly recommended to use an interface definition lanugage like Protobuf, OpenAPISpec etc to declare specify the structure of the JSON. From here, on we refer to this as the ``Plugin Specification``. @@ -72,4 +72,4 @@ Catch-all - Core Plugin .. NOTE:: Coming soon 🛠 - \ No newline at end of file + diff --git a/cookbook/docs/conf.py b/cookbook/docs/conf.py index f2988206d6..1ace1f4634 100644 --- a/cookbook/docs/conf.py +++ b/cookbook/docs/conf.py @@ -318,4 +318,5 @@ def __call__(self, filename): "flyte": ("https://flyte.readthedocs.io/en/latest/", None), # Uncomment for local development and change to your username # "flytekit": ("/Users/ytong/go/src/github.com/lyft/flytekit/docs/build/html", None), + "flyteidl": ("https://docs.flyte.org/projects/flyteidl/en/latest", None), } diff --git a/cookbook/docs/extend_flyte_backend_plugins.rst b/cookbook/docs/extend_flyte_backend_plugins.rst index 6658779bcf..5af8398670 100644 --- a/cookbook/docs/extend_flyte_backend_plugins.rst +++ b/cookbook/docs/extend_flyte_backend_plugins.rst @@ -23,14 +23,14 @@ We will try to understand the components of a backend plugin using an example pl Interface specification ------------------------ -Usually Flyte extensions need information that is not covered by a :std:ref:`Flyte TaskTemplate `. The TaskTemplate consists of a -the interface, task_type identifier, some metadata and other fields. An important field to note here is - :std:ref:`api_field_flyteidl.core.tasktemplate.custom`. The custom field is essentially an unstructured JSON. -This makes it possible to extend a task-template beyond the default supported targets -- :std:ref:`api_field_flyteidl.core.tasktemplate.container` (WIP, sql etc). +Usually Flyte extensions need information that is not covered by a :std:ref:`Flyte TaskTemplate `. The TaskTemplate consists of a +the interface, task_type identifier, some metadata and other fields. An important field to note here is - :std:ref:`custom `. The custom field is essentially an unstructured JSON. +This makes it possible to extend a task-template beyond the default supported targets -- :std:ref:`container ` (WIP, sql etc). The motivation of the Custom field, is to marshal a JSON structure that specifies information beyond what a regular TaskTemplate can capture. The actual structure of the JSON is known only to the implemented backend-plugin and the SDK components. The core Flyte platform, does not understand of look into the specifics of this structure. It is highly recommended to use an interface definition lanugage like Protobuf, OpenAPISpec etc to declare specify the structure of the JSON. From here, on we refer to this as the ``Plugin Specification``. -For Spark we decided to use Protobuf to specify the plugin as can be seen `here `__. Note it is not necessary to have the Plugin structure specified in FlyteIDL, we do it for simplicity, ease of maintenance alongwith the core platform and because of existing tooling to generate code for protobuf. +For Spark we decided to use Protobuf to specify the plugin as can be seen `here `__. Note it is not necessary to have the Plugin structure specified in FlyteIDL, we do it for simplicity, ease of maintenance alongwith the core platform and because of existing tooling to generate code for protobuf. Flytekit Plugin implementation --------------------------------