diff --git a/docs/contribute/ci.rst b/docs/contribute/ci.rst new file mode 100644 index 0000000000000..1e78e9139eb50 --- /dev/null +++ b/docs/contribute/ci.rst @@ -0,0 +1,176 @@ +.. Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + +.. http://www.apache.org/licenses/LICENSE-2.0 + +.. Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. + +.. _ci_guide: + +Using TVM's CI +============== + +TVM uses Jenkins for running Linux continuous integration (CI) tests on +`branches `_ and +`pull requests `_ through a +build configuration specified in a `Jenkinsfile `_. +Non-critical jobs run in GitHub Actions for Windows and MacOS jobs. + +A standard CI run looks something like this viewed in `Jenkins' BlueOcean viewer `_. +CI runs usually take several hours to complete and pull requests (PRs) cannot be merged before CI +has successfully completed. To diagnose failing steps, click through to the failing +pipeline stage then to the failing step to see the output logs. + +.. image:: https://github.com/tlc-pack/web-data/raw/main/images/contribute/ci.png + :width: 800 + :alt: The Jenkins UI for a CI run + + +Debugging Failures +****************** + +When CI fails for some reason, there are several methods to diagnose the issue. + +Jenkins Logs +------------ + +.. |pytest| replace:: ``pytest`` +.. _pytest: https://docs.pytest.org/en/6.2.x/ + +The first place to look for a failure is in the CI logs, follow the red Xs on +the failing job to view the logs. Note: + +* Jenkins does not display the full log by default, at the top of the log viewer + is a button "Show complete log" which will take you to a plaintext version of the log +* |pytest|_ failures are summarized at the bottom of the log but you will likely + need to scroll up to view the actual failure. + +Reproduce Failures +------------------ + +Most TVM Python tests run under |pytest|_ and +can be run as described in :ref:`pr-testing`. For a closer environment to the one +than runs in CI you can run the docker images directly, build TVM, and execute +tests inside the container. See :ref:`docker_images` for details. + +Keeping CI Green +**************** + +Developers rely on the TVM CI to get signal on their PRs before merging. +Occasionally breakages slip through and break ``main``, which in turn causes +the same error to show up on an PR that is based on the broken commit(s). Broken +commits can be identified `through GitHub `_ +via the commit status icon or via `Jenkins `_. +In these situations it is possible to either revert the offending commit or +submit a forward fix to address the issue. It is up to the committer and commit +author which option to choose, keeping in mind that a broken CI affects all TVM +developers and should be fixed as soon as possible. + +Skip CI for Reverts +------------------- + +For reverts and trivial forward fixes, adding ``[skip ci]`` to the revert's +commit message will cause CI to shortcut and only run lint. Committers should +take care that they only merge CI-skipped PRs to fix a failure on ``main`` and +not in cases where the submitter wants to shortcut CI to merge a change faster. + +.. code:: bash + + # Revert HEAD commit, make sure to insert '[skip ci]' at the beginning of + # the commit subject + git revert HEAD + git checkout -b my_fix + # After you have pushed your branch, create a PR as usual. + git push my_repo + # Example: Skip CI on a branch with an existing PR + # Adding this commit to an existing branch will cause a new CI run where + # Jenkins is skipped + git commit --allow-empty --message "[skip ci] Trigger skipped CI" + git push my_repo + +Handling Flaky Failures +*********************** + +.. https://stackoverflow.com/questions/4743845/format-text-in-a-link-in-restructuredtext/4836544#4836544 +.. |pytest's @xfail decorator| replace:: pytest's ``@xfail`` decorator +.. _pytest's @xfail decorator: https://docs.pytest.org/en/6.2.x/skipping.html#xfail-mark-test-functions-as-expected-to-fail +.. |strict=True| replace:: ``strict=True`` +.. _strict=True: https://docs.pytest.org/en/6.2.x/skipping.html#strict-parameter + +If you notice a failure on your PR that seems unrelated to your change, you should +search `recent GitHub issues related to flaky tests `_ and +`file a new issue `_ +if you don't see any reports of the failure. If a certain test or class of tests affects +several PRs or commits on ``main`` with flaky failures, the test should be disabled via +|pytest's @xfail decorator|_ with |strict=True|_ and the relevant issue linked in the +disabling PR. + +.. code:: python + + @pytest.mark.xfail(strict=False, reason="Flaky test: https://github.com/apache/tvm/issues/1234 + def test_something_flaky(): + pass + +``ci-docker-staging`` +********************* + +The `ci-docker-staging `_ +branch is used to test updates to Docker images and ``Jenkinsfile`` changes. When +running a build for a normal PR from a forked repository, Jenkins uses the code +from the PR except for the ``Jenkinsfile`` itself, which comes from the base branch. +When branches are built, the ``Jenkinsfile`` in the branch is used, so a committer +with write access must push PRs to a branch in apache/tvm to properly test +``Jenkinsfile`` changes. If your PR makes changes to the ``Jenkinsfile``, make sure +to @ a `committer `_ +and ask them to push your PR as a branch to test the changes. + +.. _docker_images: + +Docker Images +************* + +.. |top_of_the_Jenkinsfile| replace:: top of the ``Jenkinsfile`` +.. _top_of_the_Jenkinsfile: https://github.com/apache/tvm/blob/7481a297740f073b193a3f09b3e27f056e8c7f2e/Jenkinsfile#L48-L54 + +Each CI job runs most of its work inside a Docker container, built from files +in the `docker/ `_ folder. These +files are built nightly in Jenkins via the `docker-images-ci `_ job. +The images for these containers are hosted in the `tlcpack Docker Hub `_ +and referenced at the |top_of_the_Jenkinsfile|_. These can be inspected and run +locally via standard Docker commands. + +.. code:: bash + + # Beware: CI images can be several GB in size + # Get a bare docker shell in the ci-gpu container + docker run -it tlcpack/ci-gpu:v0.78 /bin/bash + +``docker/bash.sh`` will automatically grab the latest image from the ``Jenkinsfile`` +and help in mounting your current directory. + +.. code:: bash + + # Run the ci_cpu image specified in Jenkinsfile + cd tvm + bash docker/bash.sh ci_cpu + # the tvm directory is automatically mounted + # example: build tvm (note: this will overrwrite build/) + $ ./tests/scripts/task_config_build_cpu.sh + $ ./tests/scripts/task_build.sh build -j32 + + +Reporting Issues +**************** + +Issues with CI should be `reported on GitHub `_ +with a link to the relevant jobs, commits, or PRs. diff --git a/docs/contribute/committer_guide.rst b/docs/contribute/committer_guide.rst index 68885b6b927a3..3dc5bf07f3cdd 100644 --- a/docs/contribute/committer_guide.rst +++ b/docs/contribute/committer_guide.rst @@ -92,7 +92,7 @@ when they actively manage outstanding PRs, but watch the community less frequently in the rest of the time. Remember that your merit will never go away, so please -take your time and pace when contributing to the project:) +take your time and pace when contributing to the project :) Broad Collaboration @@ -101,37 +101,3 @@ Sometimes, we tend to only interact with people we know. However, broad collaborations are necessary to the success of the project. Try to keep that in mind, shepherd PRs for, and request code reviews from community members who you do not interact physically. - - -Keeping CI Green ----------------- -Developers rely on the TVM CI to get signal on their PRs before merging. -Occasionally breakges slip through and break ``main``, which in turn causes -the same error to show up on an PR that is based on the broken commit(s). -In these situations it is possible to either revert the offending commit or -submit a forward fix to address the issue. It is up to the committer and commit -author which option to choose, keeping in mind that a broken CI affects all TVM -developers and should be fixed as soon as possible. - -For reverts and trivial forward fixes, adding ``[skip ci]`` to the revert's -commit message will cause CI to shortcut and only run lint. Committers should -take care that they only merge CI-skipped PRs to fix a failure on ``main`` and -not in cases where the submitter wants to shortcut CI to merge a change faster. - -.. code:: bash - - # Example: Skip CI on a revert - # Revert HEAD commit, make sure to insert '[skip ci]' at the beginning of - # the commit subject - git revert HEAD - - git checkout -b my_fix - # After you have pushed your branch, create a PR as usual. - git push my_repo - - # Example: Skip CI on a branch with an existing PR - # Adding this commit to an existing branch will cause a new CI run where - # Jenkins is skipped - git commit --allow-empty --message "[skip ci] Trigger skipped CI" - git push my_repo - diff --git a/docs/contribute/community.rst b/docs/contribute/community.rst index 8867202a674c3..c41c7f394dd50 100644 --- a/docs/contribute/community.rst +++ b/docs/contribute/community.rst @@ -17,8 +17,8 @@ .. _community_guide: -TVM Community Guideline -======================= +TVM Community Guidelines +======================== TVM adopts the Apache style model and governs by merit. We believe that it is important to create an inclusive community where everyone can use, contribute to, and influence the direction of the project. See `CONTRIBUTORS.md `_ for the current list of contributors. @@ -42,7 +42,7 @@ Committers are individuals who are granted the write access to the project. A co - Quality of contributions: High-quality, readable code contributions indicated by pull requests that can be merged without a substantial code review. History of creating clean, maintainable code and including good test cases. Informative code reviews to help other contributors that adhere to a good standard. - Community involvement: active participation in the discussion forum, promote the projects via tutorials, talks and outreach. We encourage committers to collaborate broadly, e.g. do code reviews and discuss designs with community members that they do not interact physically. -The Project Management Committee(PMC) consists group of active committers that moderate the discussion, manage the project release, and proposes new committer/PMC members. Potential candidates are usually proposed via an internal discussion among PMCs, followed by a consensus approval, i.e. least 3 +1 votes, and no vetoes. Any veto must be accompanied by reasoning. PMCs should serve the community by upholding the community practices and guidelines TVM a better community for everyone. PMCs should strive to only nominate new candidates outside of their own organization. +The `Project Management Committee (PMC) `_ consists group of active committers that moderate the discussion, manage the project release, and proposes new committer/PMC members. Potential candidates are usually proposed via an internal discussion among PMCs, followed by a consensus approval, (i.e. at least 3 +1 votes, and no vetoes). Any veto must be accompanied by reasoning. PMCs should serve the community by upholding the community practices and guidelines TVM a better community for everyone. PMCs should strive to only nominate new candidates outside of their own organization. Reviewers diff --git a/docs/contribute/document.rst b/docs/contribute/document.rst index e3d12e83865d6..8658c0fea5062 100644 --- a/docs/contribute/document.rst +++ b/docs/contribute/document.rst @@ -26,7 +26,8 @@ it is a "simple, comprehensive and nearly universally-applicable scheme. It is proven in practice across a wide variety of fields and applications." This document describes the organization of TVM documentation, and how to write -new documentation. +new documentation. See `docs/README.md `_ +for instructions on building the docs. The Four Document Types *********************** @@ -40,8 +41,8 @@ without necessarily explaining why the software works the way it does. Those explanations can be saved for other document types. An introductory tutorial focuses on a successful first experience. These are the most important docs to turning newcomers into new users and developers. A fully end-to-end -tutorial— from installing TVM and supporting ML software, to creating and -training a model, to compiling to different architectures—will give a new +tutorial — from installing TVM and supporting ML software, to creating and +training a model, to compiling to different architectures — will give a new user the opportunity to use TVM in the most efficient way possible. A tutorial teaches a beginner something they need to know. This is in contrast with a how-to, which is meant to be an answer to a question that a user with some @@ -92,7 +93,7 @@ Within these documents you can explore contradictory and conflicting position, and help the reader make sense of how and why the software was built the way it is. It's not the place for how-tos and descriptions on how to accomplish tasks. They instead focus on higher level concepts that help with the understanding of -the project. Generally these are written by the architects and developers of +the project. Generally these are written by the architects and developers of the project, but can useful to help both users and developers to have a deeper understanding of why the software works the way it does, and how to contribute to it in ways that are consistent with the underlying design principles. @@ -124,18 +125,22 @@ Technical Details ***************** We use the `Sphinx `_ for the main documentation. -Sphinx support both the reStructuredText and markdown. When possible, we -encourage to use reStructuredText as it has richer features. Note that the -python doc-string and tutorials allow you to embed reStructuredText syntax. +Sphinx supports both reStructuredText and markdown. When possible, we +encourage reStructuredText as it has richer features. Note that the +Python doc-string and tutorials allow you to embed reStructuredText syntax. + +See +`docs/README.md `_ +for instructions on building the docs. Python Reference Documentation ------------------------------ -We use `numpydoc `_ format to -document the function and classes. The following snippet gives an example -docstring. We always document all the public functions, when necessary, -provide an usage example of the features we support(as shown below). +We use the `numpydoc `_ format to +document the function and classes. The following snippet gives an example +docstring. We always document all the public functions, when necessary, +provide an usage example of the features we support (as shown below). .. code:: python @@ -167,19 +172,19 @@ provide an usage example of the features we support(as shown below). """ return rv1 -Be careful to leave blank lines between sections of your documents. In the -above case, there has to be a blank line before `Parameters`, `Returns` and -`Examples` in order for the doc to be built correctly. To add a new function to -the doc, we need to add the `sphinx.autodoc -`_ rules to the -`docs/api/python `_). +Be careful to leave blank lines between sections of your documents. In the +above case, there has to be a blank line before ``Parameters``, ``Returns`` and +``Examples`` in order for the doc to be built correctly. To add a new function to +the docs, we need to add the `sphinx.autodoc +`_ rules to +`docs/reference/api/python `_). You can refer to the existing files under this folder on how to add the functions. C++ Reference Documentation --------------------------- -We use the doxgen format to document c++ functions. The following snippet +We use the doxygen format to document c++ functions. The following snippet shows an example of c++ docstring. .. code:: c++ @@ -200,15 +205,15 @@ add comments about code logics to improve readability. Sphinx Gallery How-Tos ---------------------- -We use the `sphinx-gallery `_ to build many -python how-tos. You can find the source code under `gallery -`_ quite self explanatory. +We use `sphinx-gallery `_ to build many +Python how-tos. You can find the source code under `gallery +`_. One thing that worth noting is that the comment blocks are written in reStructuredText instead of markdown so be aware of the syntax. -The how-to code will run on our build server to generate the document page. So +The how-to code will run on our build server to generate the document page. So we may have a restriction like not being able to access a remote Raspberry Pi, -in such case add a flag variable to the tutorial (e.g. `use_rasp`) and allow +in such case add a flag variable to the tutorial (e.g. ``use_rasp``) and allow users to easily switch to the real device by changing one flag. Then use the existing environment to demonstrate the usage. @@ -218,7 +223,7 @@ If you add a new categorization of how-to, you will need to add references to Refer to Another Location in the Document ----------------------------------------- -Please use sphinx's `:ref:` markup to refer to another location in the same doc. +Please use sphinx's ``:ref:`` markup to refer to another location in the same doc. .. code-block:: rst diff --git a/docs/contribute/git_howto.rst b/docs/contribute/git_howto.rst index 765153be220be..0fa904ff2ef6c 100644 --- a/docs/contribute/git_howto.rst +++ b/docs/contribute/git_howto.rst @@ -23,39 +23,39 @@ Git Usage Tips Here are some tips for git workflow. -How to resolve a conflict with `main` +How to resolve a conflict with ``main`` ------------------------------------- - First rebase to most recent main -.. code:: bash + .. code:: bash - # The first two steps can be skipped after you do it once. - git remote add upstream [url to tvm repo] - git fetch upstream - git rebase upstream/main + # The first two steps can be skipped after you do it once. + git remote add upstream [url to tvm repo] + git fetch upstream + git rebase upstream/main -- The git may show some conflicts it cannot merge, say `conflicted.py`. +- The git may show some conflicts it cannot merge, say ``conflicted.py``. - Manually modify the file to resolve the conflict. - After you resolved the conflict, mark it as resolved by -.. code:: bash + .. code:: bash - git add conflicted.py + git add conflicted.py - Then you can continue rebase by -.. code:: bash + .. code:: bash - git rebase --continue + git rebase --continue - Finally push to your fork, you may need to force push here. -.. code:: bash + .. code:: bash - git push --force + git push --force How to combine multiple commits into one @@ -66,35 +66,36 @@ to create a PR with set of meaningful commits. You can do it by following steps. - Before doing so, configure the default editor of git if you haven't done so before. -.. code:: bash + .. code:: bash - git config core.editor the-editor-you-like + git config core.editor the-editor-you-like - Assume we want to merge last 3 commits, type the following commands -.. code:: bash + .. code:: bash - git rebase -i HEAD~3 + git rebase -i HEAD~3 -- It will pop up an text editor. Set the first commit as `pick`, and change later ones to `squash`. +- It will pop up an text editor. Set the first commit as ``pick``, and change later ones to ``squash``. - After you saved the file, it will pop up another text editor to ask you modify the combined commit message. - Push the changes to your fork, you need to force push. -.. code:: bash + .. code:: bash - git push --force + git push --force Reset to the most recent main branch ------------------------------------ You can always use git reset to reset your version to the most recent main. -Note that all your ***local changes will get lost***. +Note that **all your local changes will get lost**. So only do it when you do not have local changes or when your pull request just get merged. .. code:: bash - git reset --hard [hash tag of main] + git fetch origin main + git reset --hard FETCH_HEAD Recover a Previous Commit after Reset diff --git a/docs/contribute/index.rst b/docs/contribute/index.rst index acacfdc8a6e26..aa893dbccb72b 100644 --- a/docs/contribute/index.rst +++ b/docs/contribute/index.rst @@ -48,4 +48,5 @@ Here are guidelines for contributing to various aspect of the project: error_handling pull_request git_howto - release_process + ci + release_process \ No newline at end of file diff --git a/docs/contribute/pull_request.rst b/docs/contribute/pull_request.rst index 23d2b1441ce8f..226e693e2c724 100644 --- a/docs/contribute/pull_request.rst +++ b/docs/contribute/pull_request.rst @@ -86,6 +86,8 @@ Here is the protocol to update CI image: - Tag the new version as the latest. - Periodically cleanup the old versions on local workers +.. _pr-testing: + Testing ------- Even though we have hooks to run unit tests automatically for each pull request, it's always recommended to run unit tests