diff --git a/README.md b/README.md index 3595b5e06..a1be04d74 100644 --- a/README.md +++ b/README.md @@ -1,31 +1,32 @@ -# Flyte IDL +# Flyteidl + +This is one of the core repositories of Flyte. It contains the Specification of the Flyte Language using protobuf messages, the Backend API specification in gRPC, and Swagger REST. The repo contains the generated clients and protocol message structures in multiple languages. Along with the generated code, the repository also contains the Golang clients for Flyte's backend APIs (the services grouped under FlyteAdmin). -This is one of the core repositories of Flyte and contains the Specification of the Flyte Lanugage using protobuf messages, the Backend API specification in gRPC and Swagger REST. The repo contains generated clients and protocol message structures in multiple languages. Along-with the generated code, the repository also contains the Golang clients for Flyte's backend API's (the services grouped under Flyteadmin). [![Slack](https://img.shields.io/badge/slack-join_chat-white.svg?logo=slack&style=social)](https://slack.flyte.org) * [flyte.org](https://flyte.org) * [Flyte Docs](http://docs.flyte.org) -* [FlyteIDL API reference documentation](https://docs.flyte.org/projects/flyteidl/en/stable/index.html) +* [Flyteidl API reference documentation](https://docs.flyte.org/projects/flyteidl/en/stable/index.html) -## Contributing to FlyteIDL +## Contributing to Flyteidl -## Tooling for FlyteIDL +## Tooling for Flyteidl -1. Run ``make download_tooling`` to install generator dependencies +1. Run ``make download_tooling`` to install generator dependencies. ```bash make download_tooling ``` -2. Make sure docker is installed locally. -3. Once installed, run ``make generate`` to generate all the code and mock client for FlyteAdmin Service aswell as the docs for it. +2. Ensure Docker is installed locally. +3. Run ``make generate`` to generate all the code, mock client, and docs for FlyteAdmin Service. ```bash make generate ``` -4. To add new dependencies for documentation generation, modify ``doc-requirements.in`` and then +4. To add new dependencies for documentation generation, modify ``doc-requirements.in`` and run ```bash make doc-requirements.txt @@ -33,8 +34,8 @@ This is one of the core repositories of Flyte and contains the Specification of ## Docs structure -The index.rst files for protos are kept in parallel folder structure under the docs folder. -All the proto definitions are within protos/flyteidl and there corresponding docs are kept in protos/docs +The index.rst files for protos are arranged in parallel under the ``docs`` folder. +All the proto definitions are within ``protos/flyteidl`` and their corresponding docs are in ``protos/docs``. ``` docs @@ -58,13 +59,13 @@ docs │   └── service.rst ``` -Each module in protos has same named module under the docs also. -eg : protos/flyteidl/core has same named doc structure placing it index and other documentation files in protos/docs/core +Each module in protos has a module in docs with the same name. +For example: ``protos/flyteidl/core`` has a module ``protos/docs/core`` under the ``docs`` folder which has the corresponding index and documentation files. -## Docs Generation +## Generating Documentation -* If introducing a new module then follow the structure for core files in `generate_protos.sh` file which helps in generating the core documentation from its proto files. +* If a new module is to be introduced, follow the structure for core files in `generate_protos.sh` file which helps generate the core documentation from its proto files. ``` core_proto_files=`ls protos/flyteidl/core/*.proto |xargs` # Remove any currently generated file @@ -72,7 +73,7 @@ eg : protos/flyteidl/core has same named doc structure placing it index and othe protoc --doc_out=protos/docs/core --doc_opt=restructuredtext,core.rst -I=protos `echo $core_proto_files` ``` -* ``make generate`` would have already generated the modified rst files. +* ``make generate`` generates the modified rst files. -* ``make html`` Generate the sphinx documentation from the docs folder to use the modified rst for docs. +* ``make html`` generates the Sphinx documentation from the docs folder that uses the modified rst files. diff --git a/index.rst b/index.rst index 672a91d7b..f06f0e0e5 100644 --- a/index.rst +++ b/index.rst @@ -1,15 +1,16 @@ .. flyteidl documentation master file, created by -FlyteIDL: Flyte's Core Language Specification -=================================================== +Flyteidl: Flyte's Core Language Specification +============================================== -``FlyteIDL`` contains the core language specification of Flyte, using `Google's Protocol Buffers `_. The Specification contains +``Flyteidl`` contains the core language specification of Flyte, using `Google's Protocol Buffers `_. +The Specification contains: -#. The core specification for Flyte Workflows, tasks and the type system +#. The core specification for Flyte workflows, tasks, and the type system #. The specification for FlyteAdmin's `gRPC `_ and ``REST`` endpoints -#. Some of the core plugin API's like - Spark, Sagemaker, etc +#. Some of the core plugin APIs like - Spark, Sagemaker, etc. -This specification is used to generate client stubs for `FlyteKit `_, `FlyteKit Java `_, `Flytectl `_ and the `FlyteAdmin Service `_. +This specification is used to generate client stubs for `Flytekit `_, `Flytekit Java `_, `Flytectl `_ and the `FlyteAdmin Service `_. .. toctree:: @@ -31,6 +32,6 @@ This specification is used to generate client stubs for `FlyteKit + Flyteidl protos/index Contributing Guide diff --git a/protos/docs/admin/index.rst b/protos/docs/admin/index.rst index b08c7e8f7..6518e82db 100644 --- a/protos/docs/admin/index.rst +++ b/protos/docs/admin/index.rst @@ -1,9 +1,8 @@ Flyte Admin Service entities ============================ -These are the control plane entities that can be used to communication with the -Flyte Admin service over gRPC or REST. The endpoint specification is defined in the - +These are the control plane entities that can be used to communicate with the +FlyteAdmin service over gRPC or REST. The endpoint specification is defined in the `Admin raw protos `__ .. toctree:: diff --git a/protos/docs/core/index.rst b/protos/docs/core/index.rst index a20d9f524..7d2ce0661 100644 --- a/protos/docs/core/index.rst +++ b/protos/docs/core/index.rst @@ -1,9 +1,9 @@ Core Flyte language specification ================================= -These set of protocol buffers provides details of some of the core data -structures like Workflows, Tasks, Nodes, Literals. These are the specification -of the various entities in Flyte as well as the type system. +Protocol buffers provide details about core data +structures like :ref:`workflows `, :ref:`tasks `, :ref:`nodes `, and Literals. They are the specifications +of the various entities in Flyte and the type system. `Core raw protos `__ diff --git a/protos/docs/datacatalog/index.rst b/protos/docs/datacatalog/index.rst index 4e203eb58..d64c2ddd9 100644 --- a/protos/docs/datacatalog/index.rst +++ b/protos/docs/datacatalog/index.rst @@ -1,10 +1,10 @@ Flyte Data Catalog Service -================================= +============================ -These protos provide the interface definition for the Data Catalog Service. Data Catalog is a service for -indexing parameterized, strongly-typed data artifacts across revisions. It is used in the Flyte ecosystem -to catalog artifacts generated by task executions. Outputs generated by a task can be stored as artifact -data and tagged by the user so that it can be retrieved later by that tag. +Protos provides the interface definition for the Data Catalog Service. Data Catalog is a service to +index parameterized, strongly-typed data artifacts across revisions. It is used in the Flyte ecosystem +to catalog artifacts generated by the task executions. The output generated by a task can be stored as artifact +data and tagged by the user so as to be retrieved later by that tag. `Datacatalog raw proto `__ diff --git a/protos/docs/event/index.rst b/protos/docs/event/index.rst index a9541ba68..b2c8abe50 100644 --- a/protos/docs/event/index.rst +++ b/protos/docs/event/index.rst @@ -9,14 +9,13 @@ External Eventing system. Flyte Internal Eventing ======================== -This is the interface used by the dataplane (execution engine) to communicate -workflow and task progress back to the **control plane** admin service. +This is the interface used by the dataplane (execution engine) to communicate with the control plane admin service about the workflow and task progress. Flyte External Eventing - Event Egress ======================================= -This refers to the interface for all the event messages exiting from the Flyte -**control plane** and delivered on the configured pubsub channel. +This refers to the interface for all the event messages leaving the Flyte +**control plane** and reaching on the configured pubsub channel. `Event raw proto `__