From 287427e6e68232dc9f0c61afa3845fb5ca62c10e Mon Sep 17 00:00:00 2001 From: Samhita Alla Date: Fri, 1 Oct 2021 16:06:21 +0530 Subject: [PATCH] Update Flytesnacks Contribution Guide (#426) * Update Flytesnacks Contribution Guide Signed-off-by: Samhita Alla * fix README Signed-off-by: Samhita Alla * fix README Signed-off-by: Samhita Alla * fix README Signed-off-by: Samhita Alla * fix README Signed-off-by: Samhita Alla * add docs env setup Signed-off-by: Samhita Alla * add docs env setup Signed-off-by: Samhita Alla --- README.md | 57 +++------------------- cookbook/docs/contribute.rst | 92 ++++++++++++++++++++++++++++++++++++ cookbook/docs/index.rst | 1 + 3 files changed, 100 insertions(+), 50 deletions(-) create mode 100644 cookbook/docs/contribute.rst diff --git a/README.md b/README.md index ffeac66bbb..de0ed2e31a 100644 --- a/README.md +++ b/README.md @@ -6,14 +6,14 @@ Flyte User Guide & Tutorials

- Flytesnacks encompasses code examples built on top of Flytekit Python + Flytesnacks encompasses code examples showcasing Flytekit Python

User Guide ยท Tutorials ยท - Contribution Guide + Contribution Guide

@@ -29,58 +29,15 @@ [Tutorials](https://docs.flyte.org/projects/cookbook/en/latest/tutorials.html) section has real-world examples, ranging from machine learning training, data processing to feature engineering. +> Flytesnacks currently has all examples in Python (Flytekit Python SDK). In the future, Java examples employing Flytekit Java will be added. +

- ๐Ÿ“– Contribution Guide + ๐Ÿ“– How to Contribute to Flytesnacks

-Flytesnacks currently has all examples in Python (Flytekit Python SDK). In the future, Java examples employing Flytekit JAVA will be out. - -Here are the setup instructions to start contributing to `flytesnacks` repo: - -### ๐Ÿ’ป Contribute to examples - -1. Determine where to put your new code: - * [Core](https://github.com/flyteorg/flytesnacks/tree/master/cookbook/core) - - Contains examples that demonstrates functionality available within core flytekit. These examples should be runnable - locally. - - * [Integrations](https://github.com/flyteorg/flytesnacks/tree/master/cookbook/integrations) - - Contains examples that leverage one or more of the available plugins. - - * [Case Studies](https://github.com/flyteorg/flytesnacks/tree/master/cookbook/case_studies) - - Contains examples that demonstrates usage of Flyte to solve real world problems. These are more generally more complex - examples that may require extra setup or that can only run on larger clusters. - -2. Create a directory - After determining where to put your example, create a directory under the appropriate parent directory. Each example - directory should contain: - * Dockerfile - * Makefile - * README.rst - * __init__.py - * requirements.in - * sandbox.config - It might be easier to copy one of the existing examples and modify it to your needs. -3. Add it to CI - Examples are references in [this github workflow](https://github.com/flyteorg/flytesnacks/blob/master/.github/workflows/ghcr_push.yml) - Add a new entry under ``strategy -> matrix -> directory`` with the name of your directory as well as its relative path. -4. Test your code! - * If the Python code can be run locally, just use `python ` to run it. - * If the Python code has to be tested in a cluster: - * Install flytectl by running `brew install flyteorg/homebrew-tap/flytectl`. Learn more about install and configuration of [flytectl](https://docs.flyte.org/projects/flytectl/en/latest/index.html) - * Run the `make start` command in the root directory of the flytesnacks repo - * Visit https://localhost:30081/console to view the Flyte console consisting of the examples present in flytesnacks/cookbook/core directory - * To fetch new dependencies and rebuild the image, run `make serialize` - * If examples from a different directory (other than `core`) have to registered then run `make -C register`. - -### ๐Ÿ“ Contribute to documentation - -`docs` folder in `flytesnacks` houses the user guide and tutorials present in the documentation. Refer to the [documentation contribution guide](https://docs.flyte.org/en/latest/community/contribute.html#documentation) to get acquainted with the guidelines. +You can find the detailed contribution guide [here](cookbook/docs/contribute.rst).

@@ -88,4 +45,4 @@ Here are the setup instructions to start contributing to `flytesnacks` repo:

-Refer to the [issues](https://docs.flyte.org/en/latest/community/contribute.html#issues) section in the contribution guide if you'd like to file an issue relating to `flytesnacks` code or documentation. +Refer to the [issues](https://docs.flyte.org/en/latest/community/contribute.html#issues) section in the contribution guide if you'd like to file an issue. diff --git a/cookbook/docs/contribute.rst b/cookbook/docs/contribute.rst new file mode 100644 index 0000000000..9d5241b862 --- /dev/null +++ b/cookbook/docs/contribute.rst @@ -0,0 +1,92 @@ +############################## +Flytesnacks Contribution Guide +############################## + +First off, thank you for thinking about contributing! +Below youโ€™ll find instructions that will hopefully guide you through how to contribute to and improve Flytesnacks. + +๐Ÿ’ป Contribute to Examples +========================= + +1. Determine where to put your new code + + * `Core `__: Contains examples that demonstrate functionality available within core flytekit. These examples should be runnable locally. + * `Integrations `__: Contains examples that leverage one or more of the available plugins. + * `Case Studies `__: Contains examples that demonstrate the usage of Flyte to solve real-world problems. These are generally more complex examples that may require extra setup or that can only run on larger clusters. + +2. Create a directory (applicable for ``integrations`` and ``case_studies`` directories) + + After determining where to put your example, create a directory under the appropriate parent directory. Each example + directory should contain: + + * Dockerfile + * Makefile + * README.rst + * __init__.py + * requirements.in + * sandbox.config + + It might be easier to copy one of the existing examples and modify it to your needs. +3. Add the example to CI + + Examples are references in `this github workflow `__. + Add a new entry under ``strategy -> matrix -> directory`` with the name of your directory as well as its relative path. + Also, add the example to `flyte_tests_manifest.json `__. +4. Test your code! + * If the Python code can be run locally, just use ``python `` to run it. + * If the Python code has to be tested in a cluster: + * Install flytectl by running ``brew install flyteorg/homebrew-tap/flytectl``. Learn more about installation and configuration of flytectl `here `__. + * Run the ``flytectl sandbox start --source=$(pwd)`` command in the directory that's one level above the directory that has Dockerfile. + For example, to register `house_price_prediction `__ example, run the start command in the ``ml_training`` directory. + To register ``core`` examples, run the start command in the ``cookbook`` directory. So, ``cd`` to the required directory and run all the upcoming commands in there! + + Following are the commands to run if examples in ``core`` directory are to be tested on sandbox: + 1. Build Docker container using the command: ``flytectl sandbox exec -- docker build . --tag "core:v1" -f core/Dockerfile``. + 2. Package the examples by running ``pyflyte --pkgs core package --image core:v1 -f``. + 3. Register the examples by running ``flytectl register files --archive -p flytesnacks -d development --archive flyte-package.tgz --version v1``. + 4. Visit https://localhost:30081/console to view the Flyte console, which consists of the examples present in the ``flytesnacks/cookbook/core`` directory. + 5. To fetch new dependencies and rebuild the image, run + ``flytectl sandbox exec -- docker build . --tag "core:v2" -f core/Dockerfile``, + ``pyflyte --pkgs core package --image core:v2 -f``, and + ``flytectl register files --archive -p flytesnacks -d development --archive flyte-package.tgz --version v2``. + 6. Refer to `Build & Deploy Your Application โ€œFastโ€er! `__ if code in itself is updated and requirements.txt is the same. + +๐Ÿ“ Contribute to Documentation +============================== + +``docs`` folder in ``flytesnacks`` houses the required documentation configuration. All the core, case_studies, and integrations docs are written in the respective README.md and the Python code files. + +1. README.md file needs to capture the *what*, *why*, and *how* + * What is the integration about? Its features, etc. + * Why do we need this integration? How is it going to benefit the Flyte users? + * Showcase the uniqueness of the integration + * How to install the plugin? + + Refer to any repo in the cookbook directory to understand this better. + +2. Explain what the code does +3. Following are some documentation configuration-related changes (imagine you have added ``snowflake`` to the ``integrations/external_services`` folder): + + 1. Update `conf.py `__ + + * Add the Python file names to the ``CUSTOM_FILE_SORT_ORDER`` list + * Add ``../integrations/external_services/snowflake`` to ``example_dirs`` + * Add ``auto/integrations/external_services/snowflake`` to ``gallery_dirs`` + 2. Update `external_services.rst `__ file + + * Add a new panel + * Update the toctree +4. Verify if the code and documentation look as expected + + 1. Learn about the documentation tools `here `__ + 2. Install the requirements by running ``pip install -r docs-requirements.txt`` in the ``cookbook`` folder + 3. Run ``make html`` in the ``docs`` folder + + .. tip:: + For implicit targets, run ``make -C docs html``. + 4. Open HTML pages present in the ``docs/_build`` directory in the browser + +5. After creating the pull request, check if the docs are rendered correctly by clicking on the documentation check + + .. image:: https://raw.githubusercontent.com/flyteorg/flyte/static-resources/img/flytesnacks/contribution-guide/test_docs_link.png + :alt: Docs link in a PR \ No newline at end of file diff --git a/cookbook/docs/index.rst b/cookbook/docs/index.rst index 33fe29c0c9..98c5e55f69 100644 --- a/cookbook/docs/index.rst +++ b/cookbook/docs/index.rst @@ -268,6 +268,7 @@ Steps Configuring Production Features integrations Extending flyte + contribute .. toctree:: :maxdepth: -1