Docs rli bug (#6008)

* add readthedocs env vars to conf.py

Signed-off-by: Niels Bantilan <[email protected]>

* use literalinclude instead of rli

Readthedocs is having issues with the remoteliteralinclude directive.
At build time it gets 403 errors when trying to fetch remote files from
github.

Signed-off-by: Niels Bantilan <[email protected]>

---------

Signed-off-by: Niels Bantilan <[email protected]>

docs/README.md

@@ -24,6 +24,11 @@ This creates a new environment called `monodocs-env` with all the dependencies n
 
 In the `flyteorg/flyte` root directory make sure you have activated the `monodocs-env` (or whatever you called it) environment and do:
 
+```bash
+# need to set this to a fake value to build the docs locally
+$ export DOCSEARCH_API_KEY=fake-api-key
+```
+
 ```bash
 $ make docs
 ```

docs/community/contribute/contribute_docs.md

@@ -82,14 +82,14 @@ The following are some tips to include various content:
 * **Source code references (Embedded format)** <br>
 	`.rst` example:
 	```{code-block}
-  .. rli:: https://raw.githubusercontent.com/flyteorg/<source repo name>/<git sha>/<target file path>
+  .. literalinclude:: /examples/<target file path>
 		:lines: <from line>-<to line>
 	```
 
 	`.md` example:
 	````{code-block}
-	```{rli} https://raw.githubusercontent.com/flyteorg/<source repo name>/<git sha>/<target file path>
-		lines: <from line>-<to line>
+	```{literalinclude} /examples/<target file path>
+		:lines: <from line>-<to line>
 	```
 	````
 

docs/conf.py

@@ -343,6 +343,22 @@
     "flytesnacks/README.md",
     "flytekit/**/README.md",
     "flytekit/_templates/**",
+    "examples/advanced_composition/**",
+    "examples/basics/**",
+    "examples/customizing_dependencies/**",
+    "examples/data_types_and_io/**",
+    "examples/development_lifecycle/**",
+    "examples/extending/**",
+    "examples/productionizing/**",
+    "examples/testing/**",
+    "flytesnacks/examples/advanced_composition/*.md",
+    "flytesnacks/examples/basics/*.md",
+    "flytesnacks/examples/customizing_dependencies/*.md",
+    "flytesnacks/examples/data_types_and_io/*.md",
+    "flytesnacks/examples/development_lifecycle/*.md",
+    "flytesnacks/examples/extending/*.md",
+    "flytesnacks/examples/productionizing/*.md",
+    "flytesnacks/examples/testing/*.md",
     "api/flytectl/index.rst",
     "protos/boilerplate/**",
     "protos/tmp/**",
@@ -622,14 +638,6 @@
                 "flytesnacks/_build",
                 "flytesnacks/_tags",
                 "flytesnacks/index.md",
-                "examples/advanced_composition",
-                "examples/basics",
-                "examples/customizing_dependencies",
-                "examples/data_types_and_io",
-                "examples/development_lifecycle",
-                "examples/extending",
-                "examples/productionizing",
-                "examples/testing"
             ]
         ],
         "local": flytesnacks_local_path is not None,
@@ -690,6 +698,15 @@
 # Disable warnings from tensorflow
 os.environ["TF_CPP_MIN_LOG_LEVEL"] = "3"
 
+# Define the canonical URL if you are using a custom domain on Read the Docs
+html_baseurl = os.environ.get("READTHEDOCS_CANONICAL_URL", "")
+
+# Tell Jinja2 templates the build is running on Read the Docs
+if os.environ.get("READTHEDOCS", "") == "True":
+    if "html_context" not in globals():
+        html_context = {}
+    html_context["READTHEDOCS"] = True
+
 
 class CustomWarningSuppressor(logging.Filter):
     """Filter logs by `suppress_warnings`."""

docs/user_guide/advanced_composition/decorating_workflows.md

@@ -23,7 +23,7 @@ To clone and run the example code on this page, see the [Flytesnacks repo][flyte
 
 To begin, import the necessary libraries.
 
-```{rli} https://raw.githubusercontent.com/flyteorg/flytesnacks/69dbe4840031a85d79d9ded25f80397c6834752d/examples/advanced_composition/advanced_composition/decorating_workflows.py
+```{literalinclude} /examples/advanced_composition/advanced_composition/decorating_workflows.py
 :caption: advanced_composition/decorating_workflows.py
 :lines: 1-6
 ```
@@ -32,7 +32,7 @@ Let's define the tasks we need for setup and teardown. In this example, we use t
 {py:class}`unittest.mock.MagicMock` class to create a fake external service that we want to initialize at the
 beginning of our workflow and finish at the end.
 
-```{rli} https://raw.githubusercontent.com/flyteorg/flytesnacks/69dbe4840031a85d79d9ded25f80397c6834752d/examples/advanced_composition/advanced_composition/decorating_workflows.py
+```{literalinclude} /examples/advanced_composition/advanced_composition/decorating_workflows.py
 :caption: advanced_composition/decorating_workflows.py
 :lines: 9-21
 ```
@@ -45,7 +45,7 @@ external service and Flyte.
 
 We create a decorator that we want to use to wrap our workflow function.
 
-```{rli} https://raw.githubusercontent.com/flyteorg/flytesnacks/69dbe4840031a85d79d9ded25f80397c6834752d/examples/advanced_composition/advanced_composition/decorating_workflows.py
+```{literalinclude} /examples/advanced_composition/advanced_composition/decorating_workflows.py
 :caption: advanced_composition/decorating_workflows.py
 :pyobject: setup_teardown
 ```
@@ -66,14 +66,14 @@ There are a few key pieces to note in the `setup_teardown` decorator above:
 
 We define two tasks that will constitute the workflow.
 
-```{rli} https://raw.githubusercontent.com/flyteorg/flytesnacks/69dbe4840031a85d79d9ded25f80397c6834752d/examples/advanced_composition/advanced_composition/decorating_workflows.py
+```{literalinclude} /examples/advanced_composition/advanced_composition/decorating_workflows.py
 :caption: advanced_composition/decorating_workflows.py
 :lines: 63-70
 ```
 
 And then create our decorated workflow:
 
-```{rli} https://raw.githubusercontent.com/flyteorg/flytesnacks/69dbe4840031a85d79d9ded25f80397c6834752d/examples/advanced_composition/advanced_composition/decorating_workflows.py
+```{literalinclude} /examples/advanced_composition/advanced_composition/decorating_workflows.py
 :caption: advanced_composition/decorating_workflows.py
 :lines: 74-82
 ```

docs/user_guide/advanced_composition/eager_workflows.md

@@ -45,7 +45,7 @@ using the `@eager` decorator.
 To clone and run the example code on this page, see the [Flytesnacks repo][flytesnacks].
 ```
 
-```{rli} https://raw.githubusercontent.com/flyteorg/flytesnacks/69dbe4840031a85d79d9ded25f80397c6834752d/examples/advanced_composition/advanced_composition/eager_workflows.py
+```{literalinclude} /examples/advanced_composition/advanced_composition/eager_workflows.py
 :caption: advanced_composition/eager_workflows.py
 :lines: 1-21
 ```
@@ -116,7 +116,7 @@ One of the biggest benefits of eager workflows is that you can now materialize
 task and subworkflow outputs as Python values and do operations on them just
 like you would in any other Python function. Let's look at another example:
 
-```{rli} https://raw.githubusercontent.com/flyteorg/flytesnacks/69dbe4840031a85d79d9ded25f80397c6834752d/examples/advanced_composition/advanced_composition/eager_workflows.py
+```{literalinclude} /examples/advanced_composition/advanced_composition/eager_workflows.py
 :caption: advanced_composition/eager_workflows.py
 :pyobject: another_eager_workflow
 ```
@@ -131,7 +131,7 @@ As you saw in the `simple_eager_workflow` workflow above, you can use regular
 Python conditionals in your eager workflows. Let's look at a more complicated
 example:
 
-```{rli} https://raw.githubusercontent.com/flyteorg/flytesnacks/69dbe4840031a85d79d9ded25f80397c6834752d/examples/advanced_composition/advanced_composition/eager_workflows.py
+```{literalinclude} /examples/advanced_composition/advanced_composition/eager_workflows.py
 :caption: advanced_composition/eager_workflows.py
 :lines: 36-53
 ```
@@ -144,7 +144,7 @@ to check if `out` is negative, but we're also using the `gt_100` task in the
 
 You can also gather the outputs of multiple tasks or subworkflows into a list:
 
-```{rli} https://raw.githubusercontent.com/flyteorg/flytesnacks/69dbe4840031a85d79d9ded25f80397c6834752d/examples/advanced_composition/advanced_composition/eager_workflows.py
+```{literalinclude} /examples/advanced_composition/advanced_composition/eager_workflows.py
 :caption: advanced_composition/eager_workflows.py
 :lines: 58-69
 ```
@@ -153,7 +153,7 @@ You can also gather the outputs of multiple tasks or subworkflows into a list:
 
 You can also invoke static workflows from within an eager workflow:
 
-```{rli} https://raw.githubusercontent.com/flyteorg/flytesnacks/69dbe4840031a85d79d9ded25f80397c6834752d/examples/advanced_composition/advanced_composition/eager_workflows.py
+```{literalinclude} /examples/advanced_composition/advanced_composition/eager_workflows.py
 :caption: advanced_composition/eager_workflows.py
 :lines: 74-84
 ```
@@ -162,7 +162,7 @@ You can also invoke static workflows from within an eager workflow:
 
 You can have nest eager subworkflows inside a parent eager workflow:
 
-```{rli} https://raw.githubusercontent.com/flyteorg/flytesnacks/69dbe4840031a85d79d9ded25f80397c6834752d/examples/advanced_composition/advanced_composition/eager_workflows.py
+```{literalinclude} /examples/advanced_composition/advanced_composition/eager_workflows.py
 :caption: advanced_composition/eager_workflows.py
 :lines: 89-97
 ```
@@ -171,7 +171,7 @@ You can have nest eager subworkflows inside a parent eager workflow:
 
 You can also catch exceptions in eager workflows through `EagerException`:
 
-```{rli} https://raw.githubusercontent.com/flyteorg/flytesnacks/69dbe4840031a85d79d9ded25f80397c6834752d/examples/advanced_composition/advanced_composition/eager_workflows.py
+```{literalinclude} /examples/advanced_composition/advanced_composition/eager_workflows.py
 :caption: advanced_composition/eager_workflows.py
 :lines: 102-117
 ```
@@ -195,7 +195,7 @@ and remotely.
 You can execute eager workflows locally by simply calling them like a regular
 `async` function:
 
-```{rli} https://raw.githubusercontent.com/flyteorg/flytesnacks/69dbe4840031a85d79d9ded25f80397c6834752d/examples/advanced_composition/advanced_composition/eager_workflows.py
+```{literalinclude} /examples/advanced_composition/advanced_composition/eager_workflows.py
 :caption: advanced_composition/eager_workflows.py
 :lines: 123-125
 ```
@@ -244,7 +244,7 @@ When using a sandbox cluster started with `flytectl demo start`, however, the
 `client_secret_group` and `client_secret_key` are not required, since the
 default sandbox configuration does not require key-based authentication.
 
-```{rli} https://raw.githubusercontent.com/flyteorg/flytesnacks/69dbe4840031a85d79d9ded25f80397c6834752d/examples/advanced_composition/advanced_composition/eager_workflows.py
+```{literalinclude} /examples/advanced_composition/advanced_composition/eager_workflows.py
 :caption: advanced_composition/eager_workflows.py
 :lines: 130-145
 ```

docs/user_guide/advanced_composition/intratask_checkpoints.md

@@ -51,14 +51,14 @@ To clone and run the example code on this page, see the [Flytesnacks repo][flyte
 
 To begin, import the necessary libraries and set the number of task retries to `3`:
 
-```{rli} https://raw.githubusercontent.com/flyteorg/flytesnacks/69dbe4840031a85d79d9ded25f80397c6834752d/examples/advanced_composition/advanced_composition/checkpoint.py
+```{literalinclude} /examples/advanced_composition/advanced_composition/checkpoint.py
 :caption: advanced_composition/checkpoint.py
 :lines: 1-4
 ```
 
 We define a task to iterate precisely `n_iterations`, checkpoint its state, and recover from simulated failures:
 
-```{rli} https://raw.githubusercontent.com/flyteorg/flytesnacks/69dbe4840031a85d79d9ded25f80397c6834752d/examples/advanced_composition/advanced_composition/checkpoint.py
+```{literalinclude} /examples/advanced_composition/advanced_composition/checkpoint.py
 :caption: advanced_composition/checkpoint.py
 :pyobject: use_checkpoint
 ```
@@ -69,14 +69,14 @@ The checkpoint system offers additional APIs, documented in the code accessible
 Create a workflow that invokes the task:
 The task will automatically undergo retries in the event of a  {ref}`FlyteRecoverableException <flytekit:exception_handling>`.
 
-```{rli} https://raw.githubusercontent.com/flyteorg/flytesnacks/69dbe4840031a85d79d9ded25f80397c6834752d/examples/advanced_composition/advanced_composition/checkpoint.py
+```{literalinclude} /examples/advanced_composition/advanced_composition/checkpoint.py
 :caption: advanced_composition/checkpoint.py
 :pyobject: checkpointing_example
 ```
 
 The local checkpoint is not utilized here because retries are not supported:
 
-```{rli} https://raw.githubusercontent.com/flyteorg/flytesnacks/69dbe4840031a85d79d9ded25f80397c6834752d/examples/advanced_composition/advanced_composition/checkpoint.py
+```{literalinclude} /examples/advanced_composition/advanced_composition/checkpoint.py
 :caption: advanced_composition/checkpoint.py
 :lines: 37-42
 ```

docs/user_guide/advanced_composition/map_tasks.md

@@ -23,14 +23,14 @@ To clone and run the example code on this page, see the [Flytesnacks repo][flyte
 
 To begin, import the required libraries:
 
-```{rli} https://raw.githubusercontent.com/flyteorg/flytesnacks/69dbe4840031a85d79d9ded25f80397c6834752d/examples/advanced_composition/advanced_composition/map_task.py
+```{literalinclude} /examples/advanced_composition/advanced_composition/map_task.py
 :caption: advanced_composition/map_task.py
 :lines: 1
 ```
 
 Here's a simple workflow that uses {py:func}`map_task <flytekit:flytekit.map_task>`:
 
-```{rli} https://raw.githubusercontent.com/flyteorg/flytesnacks/69dbe4840031a85d79d9ded25f80397c6834752d/examples/advanced_composition/advanced_composition/map_task.py
+```{literalinclude} /examples/advanced_composition/advanced_composition/map_task.py
 :caption: advanced_composition/map_task.py
 :lines: 4-19
 ```
@@ -82,7 +82,7 @@ When defining a map task, avoid calling other tasks in it. Flyte can't accuratel
 
 In this example, the map task `suboptimal_mappable_task` would not give you the best performance:
 
-```{rli} https://raw.githubusercontent.com/flyteorg/flytesnacks/69dbe4840031a85d79d9ded25f80397c6834752d/examples/advanced_composition/advanced_composition/map_task.py
+```{literalinclude} /examples/advanced_composition/advanced_composition/map_task.py
 :caption: advanced_composition/map_task.py
 :lines: 31-40
 ```
@@ -98,7 +98,7 @@ You might need to map a task with multiple inputs.
 
 For instance, consider a task that requires three inputs:
 
-```{rli} https://raw.githubusercontent.com/flyteorg/flytesnacks/69dbe4840031a85d79d9ded25f80397c6834752d/examples/advanced_composition/advanced_composition/map_task.py
+```{literalinclude} /examples/advanced_composition/advanced_composition/map_task.py
 :caption: advanced_composition/map_task.py
 :pyobject: multi_input_task
 ```
@@ -107,21 +107,21 @@ You may want to map this task with only the ``quantity`` input, while keeping th
 Since a map task accepts only one input, you can achieve this by partially binding values to the map task.
 This can be done using the {py:func}`functools.partial` function:
 
-```{rli} https://raw.githubusercontent.com/flyteorg/flytesnacks/69dbe4840031a85d79d9ded25f80397c6834752d/examples/advanced_composition/advanced_composition/map_task.py
+```{literalinclude} /examples/advanced_composition/advanced_composition/map_task.py
 :caption: advanced_composition/map_task.py
 :lines: 52-58
 ```
 
 Another possibility is to bind the outputs of a task to partials:
 
-```{rli} https://raw.githubusercontent.com/flyteorg/flytesnacks/69dbe4840031a85d79d9ded25f80397c6834752d/examples/advanced_composition/advanced_composition/map_task.py
+```{literalinclude} /examples/advanced_composition/advanced_composition/map_task.py
 :caption: advanced_composition/map_task.py
 :lines: 63-72
 ```
 
 You can also provide multiple lists as input to a `map_task`:
 
-```{rli} https://raw.githubusercontent.com/flyteorg/flytesnacks/69dbe4840031a85d79d9ded25f80397c6834752d/examples/advanced_composition/advanced_composition/map_task.py
+```{literalinclude} /examples/advanced_composition/advanced_composition/map_task.py
 :caption: advanced_composition/map_task.py
 :pyobject: map_workflow_with_lists
 ```

docs/user_guide/advanced_composition/subworkflows.md

@@ -24,7 +24,7 @@ To clone and run the example code on this page, see the [Flytesnacks repo][flyte
 
 Here's an example illustrating the calculation of slope, intercept and the corresponding y-value:
 
-```{rli} https://raw.githubusercontent.com/flyteorg/flytesnacks/69dbe4840031a85d79d9ded25f80397c6834752d/examples/advanced_composition/advanced_composition/subworkflow.py
+```{literalinclude} /examples/advanced_composition/advanced_composition/subworkflow.py
 :caption: advanced_composition/subworkflow.py
 :lines: 1-35
 ```
@@ -34,7 +34,7 @@ Subsequently, the `regression_line_wf` triggers `slope_intercept_wf` and then co
 
 To execute the workflow locally, use the following:
 
-```{rli} https://raw.githubusercontent.com/flyteorg/flytesnacks/69dbe4840031a85d79d9ded25f80397c6834752d/examples/advanced_composition/advanced_composition/subworkflow.py
+```{literalinclude} /examples/advanced_composition/advanced_composition/subworkflow.py
 :caption: advanced_composition/subworkflow.py
 :lines: 39-40
 ```
@@ -43,14 +43,14 @@ It's possible to nest a workflow that contains a subworkflow within another work
 Workflows can be easily constructed from other workflows, even if they function as standalone entities.
 Each workflow in this module has the capability to exist and run independently:
 
-```{rli} https://raw.githubusercontent.com/flyteorg/flytesnacks/69dbe4840031a85d79d9ded25f80397c6834752d/examples/advanced_composition/advanced_composition/subworkflow.py
+```{literalinclude} /examples/advanced_composition/advanced_composition/subworkflow.py
 :caption: advanced_composition/subworkflow.py
 :pyobject: nested_regression_line_wf
 ```
 
 You can run the nested workflow locally as well:
 
-```{rli} https://raw.githubusercontent.com/flyteorg/flytesnacks/69dbe4840031a85d79d9ded25f80397c6834752d/examples/advanced_composition/advanced_composition/subworkflow.py
+```{literalinclude} /examples/advanced_composition/advanced_composition/subworkflow.py
 :caption: advanced_composition/subworkflow.py
 :lines: 52-53
 ```
@@ -71,7 +71,7 @@ external workflows may offer a way to distribute the workload of a workflow acro
 
 Here's an example that illustrates the concept of external workflows:
 
-```{rli} https://raw.githubusercontent.com/flyteorg/flytesnacks/69dbe4840031a85d79d9ded25f80397c6834752d/examples/advanced_composition/advanced_composition/subworkflow.py
+```{literalinclude} /examples/advanced_composition/advanced_composition/subworkflow.py
 :caption: advanced_composition/subworkflow.py
 :lines: 61-71
 ```
@@ -85,7 +85,7 @@ In the console screenshot above, note that the launch plan execution ID differs
 
 You can run a workflow containing an external workflow locally as follows:
 
-```{rli} https://raw.githubusercontent.com/flyteorg/flytesnacks/69dbe4840031a85d79d9ded25f80397c6834752d/examples/advanced_composition/advanced_composition/subworkflow.py
+```{literalinclude} /examples/advanced_composition/advanced_composition/subworkflow.py
 :caption: advanced_composition/subworkflow.py
 :lines: 75-76
 ```