Merge remote-tracking branch 'upstream/main' into autoapi

* upstream/main: Quieter datum warning (SciTools#6050) Allow MeshCoord to have a coord-system (SciTools#6016) Bump scitools/workflows from 2024.07.1 to 2024.07.2 (SciTools#6053) Faster concatenation of cubes with `AuxCoordFactory`s (SciTools#6038) Shorten cube iterator tests (SciTools#6041) Bump scitools/workflows from 2024.07.0 to 2024.07.1 (SciTools#6045) Bump scitools/workflows from 2024.06.5 to 2024.07.0 (SciTools#6034) Update test_Saver__ugrid.py (SciTools#6017) NEP29 and NumPy v2 pins (SciTools#6039) Adapt setup.py for pypa/setuptools@2db55275f. (SciTools#6036) Replace DelegatedConda with Delegated (SciTools#5963) Enable type hint checking (SciTools#5956) Bump scitools/workflows from 2024.06.4 to 2024.06.5 (SciTools#6026) Do not realize cell measures and ancillary variables in concatenate (SciTools#6010) [pre-commit.ci] pre-commit autoupdate (SciTools#6022) Bump scitools/workflows from 2024.06.3 to 2024.06.4 (SciTools#6018)
tkknight · Jul 18, 2024 · d6df9a2 · d6df9a2
2 parents 8591583 + c66703a
commit d6df9a2
Show file tree

Hide file tree

Showing 68 changed files with 1,070 additions and 587 deletions.
diff --git a/.github/workflows/ci-manifest.yml b/.github/workflows/ci-manifest.yml
@@ -23,4 +23,4 @@ concurrency:
 jobs:
   manifest:
     name: "check-manifest"
-    uses: scitools/workflows/.github/workflows/ci-manifest.yml@2024.06.3
+    uses: scitools/workflows/.github/workflows/ci-manifest.yml@2024.07.2
diff --git a/.github/workflows/refresh-lockfiles.yml b/.github/workflows/refresh-lockfiles.yml
@@ -14,5 +14,5 @@ on:
 
 jobs:
   refresh_lockfiles:
-    uses: scitools/workflows/.github/workflows/refresh-lockfiles.yml@2024.06.3
+    uses: scitools/workflows/.github/workflows/refresh-lockfiles.yml@2024.07.2
     secrets: inherit
diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
@@ -29,7 +29,7 @@ repos:
     -   id: no-commit-to-branch
 
 -   repo: https://github.com/astral-sh/ruff-pre-commit
-    rev: "v0.4.9"
+    rev: "v0.4.10"
     hooks:
     -   id: ruff
         types: [file, python]
@@ -62,6 +62,14 @@ repos:
     -   id: sort-all
         types: [file, python]
 
+-   repo: https://github.com/pre-commit/mirrors-mypy
+    rev: 'v1.9.0'
+    hooks:
+    -   id: mypy
+        additional_dependencies:
+            - 'types-requests'
+        exclude: 'noxfile\.py|docs/src/conf\.py'
+
 -   repo: https://github.com/numpy/numpydoc
     rev: v1.7.0
     hooks:

diff --git a/benchmarks/README.md b/benchmarks/README.md
@@ -34,9 +34,11 @@ your
 [Conda package cache](https://conda.io/projects/conda/en/latest/user-guide/configuration/use-condarc.html#specify-pkg-directories),
 if it is not already. You can achieve this by either:
 
-- Temporarily reconfiguring `delegated_env_commands` and `delegated_env_parent` 
+- Temporarily reconfiguring `ENV_PARENT` in `delegated_env_commands` 
   in [asv.conf.json](asv.conf.json) to reference a location on the same file
   system as the Conda package cache.
+- Using an alternative Conda package cache location during the benchmark run,
+  e.g. via the `$CONDA_PKGS_DIRS` environment variable.
 - Moving your Iris repo to the same file system as the Conda package cache.
 
 ### Environment variables
@@ -63,6 +65,15 @@ plan accordingly.
 decorated with `@on_demand_benchmark` are included in the ASV run. Usually
 coupled with the ASV `--bench` argument to only run the benchmark(s) of
 interest. Is set during the benchmark runner `cperf` and `sperf` sub-commands.
+* `ASV_COMMIT_ENVS` - optional - instruct the 
+[delegated environment management](#benchmark-environments) to create a
+dedicated environment for each commit being benchmarked when set (to any 
+value). This means that benchmarking commits with different environment 
+requirements will not be delayed by repeated environment setup - especially 
+relevant given the [benchmark runner](bm_runner.py)'s use of
+[--interleave-rounds](https://asv.readthedocs.io/en/stable/commands.html?highlight=interleave-rounds#asv-run),
+or any time you know you will repeatedly benchmark the same commit. **NOTE:**
+Iris environments are large so this option can consume a lot of disk space.
 
 ## Writing benchmarks
 

diff --git a/benchmarks/asv.conf.json b/benchmarks/asv.conf.json
@@ -3,29 +3,59 @@
     "project": "scitools-iris",
     "project_url": "https://github.com/SciTools/iris",
     "repo": "..",
-    "environment_type": "conda-delegated",
+    "environment_type": "delegated",
     "show_commit_url": "https://github.com/scitools/iris/commit/",
     "branches": ["upstream/main"],
-    "build_command": [
-        "python setup.py build",
-        "python -mpip wheel --no-deps -w {build_cache_dir} {build_dir}"
-    ],
 
     "benchmark_dir": "./benchmarks",
     "env_dir": ".asv/env",
     "results_dir": ".asv/results",
     "html_dir": ".asv/html",
-    "plugins": [".asv_delegated_conda"],
+    "plugins": [".asv_delegated"],
+
+    "delegated_env_commands_comment": [
+        "The command(s) that create/update an environment correctly for the",
+        "checked-out commit. Command(s) format follows `build_command`:",
+        "  https://asv.readthedocs.io/en/stable/asv.conf.json.html#build-command-install-command-uninstall-command",
+
+        "The commit key indicates the earliest commit where the command(s)",
+        "will work.",
+
+        "Differences from `build_command`:",
+        "  * See: https://asv.readthedocs.io/en/stable/asv.conf.json.html#build-command-install-command-uninstall-command",
+        "  * Env vars limited to those set outside build time.",
+        "    (e.g. `{conf_dir}` available but `{build_dir}` not)",
+        "  * Run in the same environment as the ASV install itself.",
+
+        "Mandatory format for the first 'command' within each commit:",
+        "  * `ENV_PARENT=path/to/parent/directory/of/env-directory`",
+        "  * Can contain env vars (e.g. `{conf_dir}`)",
+        "  * `ENV_PARENT` available as `{env_parent}` in subsequent commands",
+        "  * The environment will be detected as the most recently updated",
+        "    environment in `{env_parent}`."
+
+    ],
+    "delegated_env_commands": {
+        "c8a663a0": [
+            "ENV_PARENT={conf_dir}/.asv/env/nox312",
+            "PY_VER=3.12 nox --envdir={env_parent} --session=tests --install-only --no-error-on-external-run --verbose"
+        ],
+        "d58fca7e": [
+            "ENV_PARENT={conf_dir}/.asv/env/nox311",
+            "PY_VER=3.11 nox --envdir={env_parent} --session=tests --install-only --no-error-on-external-run --verbose"
+        ],
+        "44fae030": [
+            "ENV_PARENT={conf_dir}/.asv/env/nox310",
+            "PY_VER=3.10 nox --envdir={env_parent} --session=tests --install-only --no-error-on-external-run --verbose"
+        ]
+    },
 
-    // The command(s) that create/update an environment correctly for the
-    //  checked-out commit.
-    // Interpreted the same as build_command, with following exceptions:
-    //  * No build-time environment variables.
-    //  * Is run in the same environment as the ASV install itself.
-    "delegated_env_commands": [
-        "PY_VER=3.12 nox --envdir={conf_dir}/.asv/env/nox01 --session=tests --install-only --no-error-on-external-run --verbose"
+    "command_comment": [
+        "We know that the Nox command takes care of installation in each",
+        "environment, and in the case of Iris no specialised uninstall or",
+        "build commands are needed to get it working."
     ],
-    // The parent directory of the above environment.
-    //  The most recently modified environment in the directory will be used.
-    "delegated_env_parent": "{conf_dir}/.asv/env/nox01"
+    "install_command": [],
+    "uninstall_command": [],
+    "build_command": []
 }