From c6af674985fa19256e3d9c5320e95739ae5a5d3c Mon Sep 17 00:00:00 2001
From: C Freeman <cecille@google.com>
Date: Wed, 10 Jul 2024 04:39:09 -0400
Subject: [PATCH] Docs: move CI python testing args into CI section (#34264)

---
 docs/testing/python.md | 114 ++++++++++++++++++++++-------------------
 1 file changed, 62 insertions(+), 52 deletions(-)

diff --git a/docs/testing/python.md b/docs/testing/python.md
index 06051fee4fb33b..5745a301535fc7 100644
--- a/docs/testing/python.md
+++ b/docs/testing/python.md
@@ -85,58 +85,8 @@ The default_matter_test_main() function is used to run the test on the command
 line. These two lines should appear verbatim at the bottom of every python test
 file.
 
-## Defining the test arguments
-
-Below is the format:
-
-```
-# test-runner-runs: <run_identifier>
-# test-runner-run/<run_identifier>/app: ${TYPE_OF_APP}
-# test-runner-run/<run_identifier>/factoryreset: <True|False>
-# test-runner-run/<run_identifier>/quiet: <True|False>
-# test-runner-run/<run_identifier>/app-args: <app_arguments>
-# test-runner-run/<run_identifier>/script-args: <script_arguments>
-```
-
-### Description of Parameters
-
--   test-runner-runs: Specifies the identifier for the run. This can be any
-    unique identifier.
-
-    -   Example: run1
-
--   test-runner-run/<run_identifier>/app: Indicates the application to be used
-    in the test. Different app types as needed could be referenced from section
-    [name: Generate an argument environment file ] of the file
-    [.github/workflows/tests.yaml](https://github.com/project-chip/connectedhomeip/blob/master/.github/workflows/tests.yaml)
-
-        -   Example: \${TYPE_OF_APP}
-
--   test-runner-run/<run_identifier>/factoryreset: Determines whether a factory
-    reset should be performed before the test.
-
-    -   Example: True
-
--   test-runner-run/<run_identifier>/quiet: Sets the verbosity level of the test
-    run. When set to True, the test run will be quieter.
-
-    -   Example: True
-
--   test-runner-run/<run_identifier>/app-args: Specifies the arguments to be
-    passed to the application during the test.
-
-    -   Example: --discriminator 1234 --KVS kvs1 --trace-to
-        json:\${TRACE_APP}.json
-
--   test-runner-run/<run_identifier>/script-args: Specifies the arguments to be
-    passed to the test script.
-    -   Example: --storage-path admin_storage.json --commissioning-method
-        on-network --discriminator 1234 --passcode 20202021 --trace-to
-        json:${TRACE_TEST_JSON}.json --trace-to perfetto:${TRACE_TEST_PERFETTO}.perfetto
-
-This structured format ensures that all necessary configurations are clearly
-defined and easily understood, allowing for consistent and reliable test
-execution.
+The structured comments above the class definition are used to set up the CI for
+the tests. Please see [Running tests in CI](#running-tests-in-ci).
 
 ## Cluster Codegen
 
@@ -610,3 +560,63 @@ example DUT on the host and includes factory reset support
 -   if there are things in your test that will fail on CI (ex. test vendor
     checks), gate them on the PICS_SDK_CI_ONLY
     -   is_ci = self.check_pics('PICS_SDK_CI_ONLY')
+
+The CI test runner uses a structured environment setup that can be declared
+using structured comments at the top of the test file. To use this structured
+format, use the --load-from-env flag with the run_python_tests.py runner.
+
+Ex:
+`scripts/run_in_python_env.sh out/venv './scripts/tests/run_python_test.py --load-from-env /tmp/test_env.yaml --script src/python_testing/TC_ICDM_2_1.py'`
+
+## Defining the CI test arguments
+
+Below is the format:
+
+```
+# test-runner-runs: <run_identifier>
+# test-runner-run/<run_identifier>/app: ${TYPE_OF_APP}
+# test-runner-run/<run_identifier>/factoryreset: <True|False>
+# test-runner-run/<run_identifier>/quiet: <True|False>
+# test-runner-run/<run_identifier>/app-args: <app_arguments>
+# test-runner-run/<run_identifier>/script-args: <script_arguments>
+```
+
+### Description of Parameters
+
+-   test-runner-runs: Specifies the identifier for the run. This can be any
+    unique identifier.
+
+    -   Example: run1
+
+-   test-runner-run/<run_identifier>/app: Indicates the application to be used
+    in the test. Different app types as needed could be referenced from section
+    [name: Generate an argument environment file ] of the file
+    [.github/workflows/tests.yaml](https://github.com/project-chip/connectedhomeip/blob/master/.github/workflows/tests.yaml)
+
+        -   Example: \${TYPE_OF_APP}
+
+-   test-runner-run/<run_identifier>/factoryreset: Determines whether a factory
+    reset should be performed before the test.
+
+    -   Example: True
+
+-   test-runner-run/<run_identifier>/quiet: Sets the verbosity level of the test
+    run. When set to True, the test run will be quieter.
+
+    -   Example: True
+
+-   test-runner-run/<run_identifier>/app-args: Specifies the arguments to be
+    passed to the application during the test.
+
+    -   Example: --discriminator 1234 --KVS kvs1 --trace-to
+        json:\${TRACE_APP}.json
+
+-   test-runner-run/<run_identifier>/script-args: Specifies the arguments to be
+    passed to the test script.
+    -   Example: --storage-path admin_storage.json --commissioning-method
+        on-network --discriminator 1234 --passcode 20202021 --trace-to
+        json:${TRACE_TEST_JSON}.json --trace-to perfetto:${TRACE_TEST_PERFETTO}.perfetto
+
+This structured format ensures that all necessary configurations are clearly
+defined and easily understood, allowing for consistent and reliable test
+execution.