diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 0000000000..f774b84d05 --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,3 @@ +# Contributing to Flyte + +For information related to contributing to Flyte, please check out the [Contributing to Flyte](https://docs.flyte.org/en/latest/community/contribute.html) section of the documentation at the Flyte site. diff --git a/rsts/community/contribute.rst b/rsts/community/contribute.rst new file mode 100644 index 0000000000..bf1212adc1 --- /dev/null +++ b/rsts/community/contribute.rst @@ -0,0 +1,255 @@ +###################### +Contributing to Flyte +###################### + +Thank you for taking the time to contribute to Flyte! Here are some guidelines for you to follow, which will make your first and follow-up contributions easier. + +.. note:: + Please read our `Code of Conduct `_ before contributing to Flyte. + +Code +==== +An issue tagged with ``good first issue`` is the best place to start for first-time contributors. Look into them `here `_. + +To take a step ahead, check out the repositories available under `flyteorg `_. + +**Appetizer (for every repo): Fork and clone the concerned repository. Create a new branch on your fork and make the required changes. Create a pull request once your work is ready for review.** + +.. note:: + Note: To open a pull request, follow this `guide `_. + +*A piece of good news -- You can be added as a committer to any ``flyteorg`` repo as you become more involved with the project.* + +Example PR for your reference: `GitHub PR `_. A couple of checks are introduced to help in maintaining the robustness of the project. + +#. To get through DCO, sign off on every commit. (`Reference `_) +#. To improve code coverage, write unit tests to test your code. + +.. note:: + Format your Go code with ``golangci-lint`` followed by ``goimports`` (we used the same in the `Makefile `_), and Python code with ``black`` (use ``make fmt`` command which contains both black and isort). + +Environment Setup +***************** + +.. figure:: https://raw.githubusercontent.com/flyteorg/flyte/static-resources/img/contribution_guide/dependency_graph.png + :alt: Dependency Graph between various flyteorg repos + :align: center + :figclass: align-center + + The dependency graph between various flyteorg repos + +#. `flyte `_ + | Purpose: Deployment, Documentation, and Issues + | Languages: Kustomize & RST +#. `flyteidl `_ + | Purpose: The Flyte Workflow specification in `protocol buffers `_ which forms the core of Flyte + | Language: Protobuf + | Setup: Refer to the `README `_ +#. `flytepropeller `_ + | Purpose: Kubernetes native execution engine for Flyte Workflows and Tasks + | Language: Go + + Setup: + * Check for the Makefile in the root repo + * Run the following commands: + * ``make generate`` + * ``make test_unit`` + * ``make link`` + * To compile, run ``make compile`` +#. `flyteadmin `_ + | Purpose: Control Plane + | Language: Go + + Setup: + * Check for the Makefile in the root repo + * If the service code has to be tested, run it locally: + * ``make compile`` + * ``make server`` + * To seed data locally: + * ``make compile`` + * ``make seed_projects`` + * ``make migrate`` + * To run integration tests locally: + * ``make integration`` + * (or, to run in containerized dockernetes): ``make k8s_integration`` +#. `flytekit `_ + | Purpose: Python SDK & Tools + | Language: Python + | Setup: Refer to the `Flytekit Contribution Guide `_ +#. `flyteconsole `_ + | Purpose: Admin Console + | Language: Typescript + | Setup: Refer to the `README `_ +#. `datacatalog `_ + | Purpose: Manage Input & Output Artifacts + | Language: Go +#. `flyteplugins `_ + | Purpose: Flyte Plugins + | Language: Go + + Setup: + * Check for the Makefile in the root repo + * Run the following commands: + * ``make generate`` + * ``make test_unit`` + * ``make link`` +#. `flytestdlib `_ + | Purpose: Standard Library for Shared Components + | Language: Go +#. `flytesnacks `_ + | Purpose: Examples, Tips, and Tricks to use Flytekit SDKs + | Language: Python (In future, Java shall be added) + + Setup: + * If the Python code has to be tested, run it locally + * If the Python code has to be tested in a cluster: + * Run the ``make start`` command in the root directory of the flytesnacks repo + * Visit https://localhost:30081 to view the Flyte console consisting of the examples present in ``flytesnacks/cookbook/core`` directory + * To fetch the new dependencies and rebuild the image, run ``make register`` +#. `flytectl `_ + | Purpose: A Standalone Flyte CLI + | Language: Go + + Setup: + * Check for the Makefile in the root repo + * Run the following commands: + * ``make generate`` + * ``make test_unit`` + * ``make link`` + +Issues +====== +`GitHub Issues `_ is used for issue tracking. There are a variety of issue types available that you could use while filing an issue. + +* `Plugin Request `_ +* `Bug Report `_ +* `Documentation Bug/Update Request `_ +* `Core Feature Request `_ +* `Flytectl Feature Request `_ +* `Housekeeping `_ +* `UI Feature Request `_ + +If none of the above fits your requirements, file a `blank `_ issue. + +Documentation +============= +Flyte uses Sphinx for documentation and ``godocs`` for Golang. ``godocs`` is quite simple -- comment your code and you are good to go! + +Sphinx spans across multiple repositories under the `flyteorg `_ repository. It uses reStructured Text (rst) files to store the documentation content. For both the API and code-related content, it extracts docstrings from the code files. + +To get started, look into `reStructuredText reference `_. + +Environment Setup +***************** +Install all the requirements from the `docs-requirements.txt` file present in the root of a repository. + +.. code-block:: console + + pip install -r docs-requirements.txt + +From the ``docs`` directory present in the repository root (for ``flytesnacks``, ``docs`` is present in ``flytesnacks/cookbook``), run the command: + +.. code-block:: console + + make html + +.. note:: + For implicit targets, run ``make -C docs html``. + +You can then view the HTML pages in the ``docs/_build`` directory. + +.. note:: + For ``flyte`` repo, there is no ``docs`` directory. Instead, consider the ``rsts`` directory. To generate HTML files, run the following command in the root of the repo. + + .. code-block:: console + + make -C rsts html + +For minor edits that don’t require a local setup, you can edit the GitHub page in the documentation to propose the improvements. + +The edit option is found at the bottom of a page, as shown below. + +.. figure:: https://raw.githubusercontent.com/flyteorg/flyte/static-resources/img/contribution_guide/docs_edit.png + :alt: GitHub edit option for Documentation + :align: center + :figclass: align-center + +Intersphinx +*********** +`Intersphinx `_ can generate automatic links to the documentation of objects in other projects. + +To establish a reference to any other documentation from Flyte or within it, use intersphinx. + +To do so, create an ``intersphinx_mapping`` in the ``conf.py`` file present in the ``docs/source`` directory. + +For example: + +.. code-block:: python + + intersphinx_mapping = { + "python": ("https://docs.python.org/3", None), + "flytekit": ("https://flyte.readthedocs.io/projects/flytekit/en/master/", None), + } + +.. note:: + ``docs/source`` is present in the repository root. Click `here `_ to view the intersphinx configuration. + +The key refers to the name used to refer to the file (while referencing the documentation), and the URL denotes the precise location. + +Here are a couple of examples that you can refer to: + +.. code-block:: text + + Task: :std:doc:`generated/flytekit.task` + +Output: + +Task: :std:doc:`generated/flytekit.task` + +.. code-block:: text + + :std:doc:`Using custom words ` + +Output: + +:std:doc:`Using custom words ` + +| + +Linking to Python elements changes based on what you're linking to. Check out this `section `_ to learn more. + +| + +For instance, linking to the `task` decorator in flytekit uses the ``func`` role. + +.. code-block:: text + + Link to flytekit code :py:func:`flytekit:flytekit.task` + +Output: + +Link to flytekit code :py:func:`flytekit:flytekit.task` + +| + +Here are a couple more examples. + +.. code-block:: text + + :py:mod:`Module ` + :py:class:`Class ` + :py:data:`Data ` + :py:func:`Function ` + :py:meth:`Method ` + +Output: + +:py:mod:`Module ` + +:py:class:`Class ` + +:py:data:`Data ` + +:py:func:`Function ` + +:py:meth:`Method ` \ No newline at end of file diff --git a/rsts/community/docs.rst b/rsts/community/docs.rst deleted file mode 100644 index 2e33df8440..0000000000 --- a/rsts/community/docs.rst +++ /dev/null @@ -1,144 +0,0 @@ -.. _contribute-docs: - -###################### -Contributing to Docs -###################### - -************************ -Docs for various repos -************************ -Flyte is a large project and all the docs span multiple repositories. The core of the documention is in the `flyteorg/flyte `_ repository. -Flyte uses `Sphinx `_ to compile it docs. Docs are automatically pushed on merge to master and docs are hosted using `readthedocs.org `_ - -*************** -Sphinx and RST -*************** - -Style -========= - -Headers --------- -Typically, we try to follow these characters in this order for heading separation. - -.. code-block:: text - - # with overline - * with overline - = - - - ^ - -Intersphinx -============= -`Intersphinx `__ is a plugin that all Flyte repos that build Sphinx documentation use for cross-referencing with each other. There's some good background information on it on these `slides `__. - -Inventory File ----------------- -When Sphinx runs, an inventory file gets created and is available alongside each repo's HTML pages. For example at ``https://readthedocs.org/projects/flytecookbook/objects.inv``. This file is a compressed inventory of all the sections, tags, etc. in the flyte cookbook documentation. This inventory file is what allows the intersphinx plugin to cross-link between projects. - -There is an open-source tool called ``sphobjinv`` that has managed to `reverse engineer these files `__, and offers a CLI to help search for things inside them. - -Setup -------- -Installing ``sphobjinv`` and simple usage :: - - $ pip install sphobjinv - - # Using the CLI to query a hosted inventory file, note the -u switch - $ sphobjinv suggest https://flytekit.readthedocs.io/en/latest/ -u task - - No inventory at provided URL. - Attempting "https://flytekit.readthedocs.io/en/latest/objects.inv" ... - Remote inventory found. - - :py:function:`flytekit.task` - :std:doc:`tasks` - :std:doc:`tasks.extend` - :std:label:`tasks:tasks` - - # Using the CLI to query a local file, useful when iterating locally - $ sphobjinv suggest ~/go/src/github.com/flyteorg/flytekit/docs/build/html/objects.inv task - - :py:function:`flytekit.task` - :std:doc:`tasks` - :std:doc:`tasks.extend` - :std:label:`tasks:tasks` - -.. note:: - - Even though the ``sphobjinv`` CLI returns ``:py:function:...``, when actually creating a link you should just use ``:py:func:...``. See `this `__. Here is a quick list of mappings - - .. list-table:: Conversion table for - ``sphobjinv`` - :widths: 50 50 - :header-rows: 1 - - * - What the tool returns? - - What you should use instead? - * - :py:module: - - :py:mod: - * - :py:function: - - :py:func: - * - :std:label: - - :std:ref: - * - :py:method: - - :py:meth: - - - -Linking Examples ------------------- -In the ``conf.py`` file of each repo, there is an intersphinx mapping argument that looks something like this :: - - intersphinx_mapping = { - "python": ("https://docs.python.org/3", None), - "flytekit": ("https://flyte.readthedocs.io/projects/flytekit/en/master/", None), - ... - } - -This file is what tells the plugin where to look for these inventory files, and what project name to refer to each inventory file as. The project name is important because they're used when actually referencing something from the inventory. - -Here are some examples, first the code and then the link - -.. code-block:: text - - Task: :std:doc:`generated/flytekit.task` - -Task: :std:doc:`generated/flytekit.task` - ------ - -.. code-block:: text - - :std:doc:`Using custom words ` - -:std:doc:`Using custom words ` - -Python -^^^^^^^ -Linking to Python elements changes based on what you're linking to. Check out this `section `__. For instance linking to the ``task`` decorator in flytekit uses the ``func`` role. - -.. code-block:: text - - Link to flytekit code :py:func:`flytekit:flytekit.task` - -Link to flytekit code :py:func:`flytekit:flytekit.task` - -Other elements use different Sphinx roles, here are some examples using Python core docs. :: - - :py:mod:`Module ` - :py:class:`Class ` - :py:data:`Data ` - :py:func:`Function ` - :py:meth:`Method ` - - -:py:mod:`Module ` - -:py:class:`Class ` - -:py:data:`Data ` - -:py:func:`Function ` - -:py:meth:`Method ` diff --git a/rsts/index.rst b/rsts/index.rst index 4fee64cdd9..b4b8bae45f 100644 --- a/rsts/index.rst +++ b/rsts/index.rst @@ -34,7 +34,7 @@ :hidden: Join the Community - community/docs + community/contribute community/roadmap community/compare