allow Python bundles and packages to specify a maximum Python version for the system toolchain

[casparvl]

We implement support for specifying a maximum python version for EasyConfigs that use system python (through the max_py_majver, max_py_minver keywords). This is is required for easybuilders/easybuild-easyconfigs#21307 which has one EasyConfig that will only work on the python 3.6 - 3.11 range (3.12 will fail because the setuptools is too old, but newer setuptools do not support python3.6).

This PR is built on top of #3430 . That one should probably be reviewed & merged first, after which this PR should be synced with develop.

[bedroge]

Test report by @bedroge

Overview of tested easyconfigs (in order)

• SUCCESS pyBigWig-0.3.22-gfbf-2023a.eb
• SUCCESS PyYAML-6.0-GCCcore-12.2.0.eb
• SUCCESS sktime-0.25.0-gfbf-2023a.eb
• SUCCESS NiBabel-4.0.2-foss-2022a.eb

Build succeeded for 4 out of 4 (4 easyconfigs in total)
bob-Latitude-5300 - Linux Ubuntu 24.04.1 LTS (Noble Numbat), x86_64, Intel(R) Core(TM) i7-8665U CPU @ 1.90GHz, Python 3.12.3
See https://gist.github.com/bedroge/3235d0ff53051a7f7472f287c85e5194 for a full test report.

[bedroge]

Test report by @bedroge

Overview of tested easyconfigs (in order)

• FAIL (build issue) ReFrame-4.6.2-Python-3.6.eb (partial log available at https://gist.github.com/bedroge/bdacc658fc5a6a96e373ad8f5505b7c2)

Build succeeded for 0 out of 1 (1 easyconfigs in total)
bob-Latitude-5300 - Linux Ubuntu 24.04.1 LTS (Noble Numbat), x86_64, Intel(R) Core(TM) i7-8665U CPU @ 1.90GHz, Python 3.12.3
See https://gist.github.com/bedroge/c5bc91e9bf2dab40ee1d2eec64f6eba2 for a full test report.

Expected failure that confirms that the new functionality works. I used Python 3.12 for a ReFrame version (from easybuilders/easybuild-easyconfigs#21307) that only supports Python <= 3.11.

== FAILED: Installation ended unsuccessfully (build directory: /data/eb/build/ReFrame/4.6.2/system-system-Python-3.6): build failed (first 300 chars): Failed to pick Python command that satisfies requirements in the easyconfig (req_py_majver = 3, req_py_minver = 6, max_py_majver = 3, max_py_minver = 11) (took 0 secs)

[bedroge]

Test report by @bedroge

Overview of tested easyconfigs (in order)

• SUCCESS ReFrame-4.6.2.eb

Build succeeded for 1 out of 1 (1 easyconfigs in total)
bob-Latitude-5300 - Linux Ubuntu 24.04.1 LTS (Noble Numbat), x86_64, Intel(R) Core(TM) i7-8665U CPU @ 1.90GHz, Python 3.12.3
See https://gist.github.com/bedroge/0a9eb4ce1c0a45c11a91de2ef7e08122 for a full test report.

[bedroge]

Test report by @bedroge

Overview of tested easyconfigs (in order)

• FAIL (build issue) ReFrame-4.6.2-Python-3.6.eb (partial log available at https://gist.github.com/bedroge/78dc999c06ebd0f46f3bfe56f69a5a70)

Build succeeded for 0 out of 1 (1 easyconfigs in total)
bob-Latitude-5300 - Linux Ubuntu 24.04.1 LTS (Noble Numbat), x86_64, Intel(R) Core(TM) i7-8665U CPU @ 1.90GHz, Python 3.12.3
See https://gist.github.com/bedroge/5459719d39a8ec3d367525b0a0eb14cc for a full test report.

[bedroge]

Redoing all test builds to make sure things still work.

[bedroge]

Test report by @bedroge

Overview of tested easyconfigs (in order)

• SUCCESS pyBigWig-0.3.22-gfbf-2023a.eb
• SUCCESS PyYAML-6.0-GCCcore-12.2.0.eb
• SUCCESS sktime-0.25.0-gfbf-2023a.eb
• SUCCESS NiBabel-4.0.2-foss-2022a.eb
• SUCCESS ReFrame-4.6.2.eb

Build succeeded for 5 out of 5 (5 easyconfigs in total)
bob-Latitude-5300 - Linux Ubuntu 24.04.1 LTS (Noble Numbat), x86_64, Intel(R) Core(TM) i7-8665U CPU @ 1.90GHz, Python 3.12.3
See https://gist.github.com/bedroge/9e6d8b842a542e42333cb212c2c2b18b for a full test report.

[bedroge]

Test report by @bedroge

Overview of tested easyconfigs (in order)

• FAIL (build issue) ReFrame-4.6.2-Python-3.6.eb (partial log available at https://gist.github.com/bedroge/79e8d70e09d85997faee4b54764edf73)

Build succeeded for 0 out of 1 (1 easyconfigs in total)
bob-Latitude-5300 - Linux Ubuntu 24.04.1 LTS (Noble Numbat), x86_64, Intel(R) Core(TM) i7-8665U CPU @ 1.90GHz, Python 3.12.3
See https://gist.github.com/bedroge/62e41ef10b452bb177eeb4c9d67484c7 for a full test report.

Reworded the debug message a bit:

Python version (3.12.3) on the system is newer than the maximum supported Python version specified in the easyconfig (3.11)

[bedroge]

Test report by @bedroge

Overview of tested easyconfigs (in order)

• SUCCESS ReFrame-4.6.2.eb

Build succeeded for 1 out of 1 (1 easyconfigs in total)
bob-Latitude-5300 - Linux Ubuntu 24.04.1 LTS (Noble Numbat), x86_64, Intel(R) Core(TM) i7-8665U CPU @ 1.90GHz, Python 3.12.3
See https://gist.github.com/bedroge/1a18889f623416471276207030f5d75f for a full test report.

[bedroge]

@casparvl I did a search for pick_python_cmd, and found that there are unit tests for this function. So it would be good to add some additional ones here https://github.com/easybuilders/easybuild-easyblocks/blob/develop/test/easyblocks/module.py#L227.

edit: added two tests in casparvl#4.

[casparvl]

@bedroge I merged your unit test. Is there anything else that needs to be done for this PR to be merged? :)

[bedroge]

Lgtm

[bedroge]

Test report by @bedroge

Overview of tested easyconfigs (in order)

• SUCCESS pyBigWig-0.3.22-gfbf-2023a.eb
• SUCCESS PyYAML-6.0-GCCcore-12.2.0.eb
• SUCCESS sktime-0.25.0-gfbf-2023a.eb
• SUCCESS NiBabel-4.0.2-foss-2022a.eb
• SUCCESS ReFrame-4.6.2.eb

Build succeeded for 5 out of 5 (5 easyconfigs in total)
bob-Latitude-5300 - Linux Ubuntu 24.04.1 LTS (Noble Numbat), x86_64, Intel(R) Core(TM) i7-8665U CPU @ 1.90GHz, Python 3.12.3
See https://gist.github.com/bedroge/e084b7ecccf94c2730b4362c9e8887d3 for a full test report.

[Flamefire]

@casparvl What is the intended semantic for specifying only a major version? I would expect that max_py_majver = 3 would accept any Python 3 version. Or maybe more sensible: max_py_majver = 2 would only exclude Python 3. Currently the implementation seems to check for < max_py_majver, which means Python 3.1 and Python 2.1 respectively would be excluded.