Docs: Refactor "Environment variables" into 3 articles (Diátaxis) (#9966

) * Splits environment variable articles in explanation (custom variables) and reference (pre-defined) * Split out further contents to a how-to, refine all the text, adding more detail on using secret envs * Clarification * Expand section "alternative approaches" and improve intro * Clarification of environment expansion * "Build process" not "build environment" to make it easier * Merge together two paragraphs and clarify that "two layers" as "two sets of environment variables" * Elaborate the scenario for an alternative approach * Real sentence structure for how-to steps * Put the note before the "Save" instruction. Add a bit more message glue between sections * Add a reference and improve env variable reference intro * a-z order env vars * New title for explanation and some more content on sphinx-multiproject * Phrase a bit more general as "patterns" * Re-order env vars, remove Git conflict artifact, add seealsos * Apply suggestions from @ericholscher's code review Co-authored-by: Eric Holscher <[email protected]> * Fix a :ref: * seealso texts updated * Have one `note::` box less --------- Co-authored-by: Eric Holscher <[email protected]>
readthedocs · Feb 10, 2023 · b1b877c · b1b877c
1 parent 28da459
commit b1b877c
Show file tree

Hide file tree

Showing 8 changed files with 236 additions and 77 deletions.
diff --git a/docs/user/build-customization.rst b/docs/user/build-customization.rst
@@ -70,7 +70,7 @@ In that case, a config file similar to this one can be used:
 There are some caveats to knowing when using user-defined jobs:
 
 * The current working directory is at the root of your project's cloned repository
-* Environment variables are expanded in the commands (see :doc:`environment-variables`)
+* Environment variables are expanded for each individual command (see :doc:`/reference/environment-variables`)
 * Each command is executed in a new shell process, so modifications done to the shell environment do not persist between commands
 * Any command returning non-zero exit code will cause the build to fail immediately
   (note there is a special exit code to `cancel the build <cancel-build-based-on-a-condition>`_)

diff --git a/docs/user/builds.rst b/docs/user/builds.rst
@@ -16,7 +16,7 @@ It also allows you customize the steps of the build process.
 .. note::
 
    All the steps are run inside a Docker container with the image the project defines in :ref:`config-file/v2:build.os`,
-   and all the :doc:`/environment-variables` defined are exposed to them.
+   and all the :doc:`pre-defined environment variables </reference/environment-variables>` and :doc:`custom environment variables </environment-variables>` defined are exposed to them.
 
 
 The following are the pre-defined jobs executed by Read the Docs:

diff --git a/docs/user/environment-variables.rst b/docs/user/environment-variables.rst
@@ -1,116 +1,135 @@
-Environment Variables
-=====================
+.. _Environment Variables:
 
-Read the Docs supports two types of environment variables in project builds:
+Understanding environment variables
+===================================
 
-* `Default environment variables`_
-* `User-defined environment variables`_
+Read the Docs allows you to define your own environment variables to be used in the build process.
+It also defines a set of :doc:`default environment variables </reference/environment-variables>` with information about your build.
+These are useful for different purposes:
 
-Both are merged together during the build process and are exposed to all of the executed commands. There are two exceptions for user-defined environment variables however:
+* Custom environment variables are useful for adding build secrets such as API tokens.
+* Default environment variables are useful for varying your build specifically for Read the Docs or specific types of builds on Read the Docs.
 
-* User-defined variables are not available during the checkout step of the :doc:`build process </builds>`
-* User-defined variables that are not marked as public will not be available in :doc:`pull request builds </pull-requests>`
+.. The following introduction is difficult to balance.
+.. We should ideally support environment variables in the Config File,
+.. but as long as it's not supported then people can add environment variables in different ways.
+.. Using the Dashboard is a good approach
+.. but adding an environment variable with ``ENV=123 command --flag`` in the build process is possibly better.
 
-Default environment variables
------------------------------
+Custom environment variables are defined in the :term:`dashboard` interface in :menuselection:`Admin --> Environment variables`.
+Environment variables are defined for a project's entire build process,
+:ref:`with 2 important exceptions <custom_env_var_exceptions>`.
 
-Read the Docs builders set the following environment variables automatically for each documentation build:
+Aside from storing secrets,
+there are :ref:`other patterns <environment-variables:Patterns of using environment variables>` that take advantage of environment variables,
+like reusing the same *monorepo* configuration in multiple documentation projects.
+In cases where the environment variable isn't a secret,
+like a build tool flag,
+you should also be aware of the :ref:`alternatives to environment variables <environment-variables:Alternatives to environment variables>`.
 
-.. envvar:: READTHEDOCS
+.. seealso::
 
-    Whether the build is running inside Read the Docs.
+   :doc:`/guides/environment-variables`
+     A practical example of adding and accessing custom environment variables.
 
-    :Default: ``True``
+   :doc:`/reference/environment-variables`
+     Reference to all pre-defined environment variables for your build environments.
 
-.. envvar:: READTHEDOCS_VERSION
+   :ref:`Public API reference: Environment variables <api/v3:Environment Variables>`
+     Reference for managing custom environments via Read the Docs' API.
 
-    The :term:`slug` of the version being built, such as ``latest``, ``stable``,
-    or a branch name like ``feature-1234``. For :doc:`pull request builds </pull-requests>`,
-    the value will be the pull request number.
+Environment variables and build process
+---------------------------------------
 
-.. envvar:: READTHEDOCS_VERSION_NAME
+When a :doc:`build process </builds>` is started,
+:doc:`pre-defined environment variables </reference/environment-variables>` and custom environment variables are added *at each step* of the build process.
+The two sets of environment variables are merged together during the build process and are exposed to all of the executed commands,
+with pre-defined variables taking precedence over custom environment variables.
 
-    The verbose name of the version being built, such as ``latest``, ``stable``,
-    or a branch name like ``feature/1234``.
+.. _custom_env_var_exceptions:
 
-.. envvar:: READTHEDOCS_VERSION_TYPE
+There are two noteworthy exceptions for *custom environment variables*:
 
-    The type of the version being built.
+Build checkout step
+  Custom environment variables are **not** available during the checkout step of the :doc:`build process </builds>`
+Pull Request builds
+  Custom environment variables that are not marked as :guilabel:`Public` will not be available in :doc:`pull request builds </pull-requests>`
 
-    :Values: ``branch``, ``tag``, ``external`` (for :doc:`pull request builds </pull-requests>`), or ``unknown``
+.. the presence of this section is intended to evolve into a better explanation
+.. with a few more scenarios,
+.. once there is better options for environment variables in config files
 
-.. envvar:: READTHEDOCS_PROJECT
+Patterns of using environment variables
+---------------------------------------
 
-    The :term:`slug` of the project being built. For example, ``my-example-project``.
+Aside from storing secrets,
+environment variables are also useful if you need to make either your :doc:`.readthedocs.yaml </config-file/v2>` or the commands called in the :doc:`build process </builds>`
+behave depending on :doc:`pre-defined environment variables </reference/environment-variables>` or your own custom environment variables.
 
-.. envvar:: READTHEDOCS_LANGUAGE
+Example: Multiple projects from the same Git repo
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-    The locale name, or the identifier for the locale, for the project being built.
-    This value comes from the project's configured language.
+If you have the need to build multiple documentation websites from the same Git repository,
+you can use an environment variable to configure the behavior of your :doc:`build commands </build-customization>`
+or Sphinx ``conf.py`` file.
 
-    :Examples: ``en``, ``it``, ``de_AT``, ``es``, ``pt_BR``
+An example of this is found in *the documentation project that you are looking at now*.
+Using the Sphinx extension `sphinx-multiproject`_,
+the following configuration code decides whether to build the *user* or *developer* documentation.
+This is defined by the ``PROJECT`` environment variable:
 
-.. envvar:: READTHEDOCS_VIRTUALENV_PATH
-
-    Path for the :ref:`virtualenv that was created for this build <builds:Understanding what's going on>`.
-    Only exists for builds using Virtualenv and not Conda.
-
-    :Example: ``/home/docs/checkouts/readthedocs.org/user_builds/project/envs/version``
-
-User-defined environment variables
-----------------------------------
-
-If extra environment variables are needed in the build process (like an API token),
-you can define them from the project's settings page:
+.. code-block:: python
+   :caption: Read the Docs' conf.py [1]_ is used to build 2 documentation projects.
 
-#. Go to your project's :guilabel:`Admin` > :guilabel:`Environment Variables`
-#. Click on :guilabel:`Add Environment Variable`
-#. Fill the ``Name`` and ``Value``
-#. Check the :guilabel:`Public` option if you want to expose this environment variable
-   to :doc:`builds from pull requests </pull-requests>`.
+   from multiproject.utils import get_project
 
-   .. warning::
+   # (...)
 
-      If you mark this option, any user that can create a pull request
-      on your repository will be able to see the value of this environment variable.
+   multiproject_projects = {
+       "user": {
+           "use_config_file": False,
+           "config": {
+               "project": "Read the Docs user documentation",
+           },
+       },
+       "dev": {
+           "use_config_file": False,
+           "config": {
+               "project": "Read the Docs developer documentation",
+           },
+       },
+   }
 
-#. Click on :guilabel:`Save`
 
-.. note::
+   docset = get_project(multiproject_projects)
 
-   Once you create an environment variable,
-   you won't be able to see its value anymore.
+.. _sphinx-multiproject: https://sphinx-multiproject.readthedocs.io/
+.. [1] https://github.com/readthedocs/readthedocs.org/blob/main/docs/conf.py
 
-After adding an environment variable,
-you can read it from your build process,
-for example in your Sphinx's configuration file:
+Alternatives to environment variables
+-------------------------------------
 
-.. code-block:: python
-   :caption: conf.py
+In some scenarios, it's more feasible to define your build's environment variables using the ``.readthedocs.yaml`` :doc:`configuration file </config-file/index>`.
+Using the :term:`dashboard` for administering environment variables may not be the right fit if you already know that you want to manage environment variables *as code*.
 
-   import os
-   import requests
+Consider the following scenario:
 
-   # Access to our custom environment variables
-   username = os.environ.get("USERNAME")
-   password = os.environ.get("PASSWORD")
+* The environment variable **is not** a secret.
 
-   # Request a username/password protected URL
-   response = requests.get(
-       "https://httpbin.org/basic-auth/username/password",
-       auth=(username, password),
-   )
+  **and**
+* The environment variable is used just once for a custom command.
 
-You can also use any of these variables from :term:`user-defined build jobs` in your project's configuration file:
+In this case, you can define the environment variable *as code* using :doc:`/build-customization`.
+The following example shows how a non-secret single-purpose environment variable can also be used.
 
 .. code-block:: yaml
    :caption: .readthedocs.yaml
 
    version: 2
    build:
-     os: ubuntu-22.04
+     os: "ubuntu-22.04"
      tools:
-       python: 3.10
+       python: "3.11"
      jobs:
-       post_install:
-         - curl -u ${USERNAME}:${PASSWORD} https://httpbin.org/basic-auth/username/password
+       post_build:
+         - EXAMPLE_ENVIRONMENT_VARIABLE=foobar command --flag
diff --git a/docs/user/guides/administrators.rst b/docs/user/guides/administrators.rst
@@ -28,6 +28,7 @@ have a look at our :doc:`/tutorial/index`.
    setup-single-sign-on-github-gitlab-bitbucket
    setup-single-sign-on-google-email
    manage-read-the-docs-teams
+   Use custom environment variables <environment-variables>
    automation-rules
    redirects
    Managing your Read the Docs for Business subscription </commercial/subscriptions>
diff --git a/docs/user/guides/environment-variables.rst b/docs/user/guides/environment-variables.rst
@@ -0,0 +1,84 @@
+How to use custom environment variables
+=======================================
+
+If extra environment variables are needed in the build process,
+you can define them from the project's :term:`dashboard`.
+
+.. seealso::
+
+   :doc:`/environment-variables`
+     Learn more about how Read the Docs applies environment variables in your builds.
+
+Go to your project's :menuselection:`Admin --> Environment Variables` and click on :guilabel:`Add Environment Variable`.
+You will then see the form for adding an environment variable:
+
+.. image:: /img/screenshot_environment_variables.png
+   :alt: Screenshot of the form for adding an environment variable
+
+#. Fill in the :guilabel:`Name` field, this is the name of your variable, for instance ``SECRET_TOKEN`` or ``PIP_EXTRA_INDEX_URL``.
+#. Fill in the :guilabel:`Value` field with the environment variable's value, for instance a secret token or a build configuration.
+#. Check the :guilabel:`Public` option if you want to expose this environment variable
+   to :doc:`builds from pull requests </pull-requests>`.
+
+   .. warning::
+
+      If you make your environment variable **public**, any user that can create a pull request
+      on your repository will be able to see the value of this environment variable.
+      In other words,
+      **do not use this option if your environment variable is a secret.**
+
+Finally, click on :guilabel:`Save`.
+Your custom environment variable is immediately active for all future builds
+and you are ready to start using it.
+
+Note that once you create an environment variable,
+you won't be able to edit or view its value.
+The only way to edit an environment variable is to delete and create it again.
+
+Keep reading for a few examples of using environment variables. ⬇️
+
+Accessing environment variables in code
+---------------------------------------
+
+After adding an environment variable,
+you can read it from your build process,
+for example in your Sphinx's configuration file:
+
+.. code-block:: python
+   :caption: conf.py
+
+   import os
+   import requests
+
+   # Access to our custom environment variables
+   username = os.environ.get("USERNAME")
+   password = os.environ.get("PASSWORD")
+
+   # Request a username/password protected URL
+   response = requests.get(
+       "https://httpbin.org/basic-auth/username/password",
+       auth=(username, password),
+   )
+
+Accessing environment variables in build commands
+-------------------------------------------------
+
+You can also use any of these variables from :term:`user-defined build jobs` in your project's configuration file:
+
+.. code-block:: yaml
+   :caption: .readthedocs.yaml
+
+   version: 2
+   build:
+     os: ubuntu-22.04
+     tools:
+       python: 3.10
+     jobs:
+       post_install:
+         - curl -u ${USERNAME}:${PASSWORD} https://httpbin.org/basic-auth/username/password
+
+.. note::
+
+   If you use ``${SECRET_ENV}`` in a command in ``.readthedocs.yaml``,
+   the private value of the environment variable is not substituted in log entries of the command.
+   It will also be logged as ``${SECRET_ENV}``.
diff --git a/docs/user/img/screenshot_environment_variables.png b/docs/user/img/screenshot_environment_variables.png
diff --git a/docs/user/index.rst b/docs/user/index.rst
@@ -80,6 +80,7 @@ to help you create fantastic documentation for your project.
    /build-notifications
    /integrations
    /custom-domains
+   /environment-variables
    /pull-requests
    /connected-accounts
    /downloadable-documentation
@@ -113,6 +114,7 @@ to help you create fantastic documentation for your project.
    /reference/features
    /server-side-search/syntax
    /reference/policies
+   /reference/environment-variables
 
 
 Read the Docs feature overview
@@ -142,6 +144,7 @@ and some of the core features of Read the Docs.
   :doc:`Build process </builds>` |
   :doc:`Build customization </build-customization>` |
   :doc:`/environment-variables` |
+  :doc:`/reference/environment-variables` |
   :doc:`/badges`
 
 * **Troubleshooting**:
@@ -160,7 +163,6 @@ and some of the core features of Read the Docs.
 
    /builds
    /build-customization
-   /environment-variables
 
    /support