{bio}[foss/2020b] QIIME2 v2022.8 w/ Python 3.8.6 (WIP)

[migueldiascosta]

(created using eb --new-pr)

requires:

• {lib}[GCCcore/10.2.0] alsa-lib v1.2.8 #16941
• {tools}[foss/2020b] umap-learn v0.4.6 w/ Python 3.8.6 #16942 {tools}[foss/2020b] umap-learn v0.5.3 w/ Python 3.8.6 #17161
• {bio}[foss/2020b] scikit-bio v0.5.7 w/ Python 3.8.6 #16943

(as mentioned in the linked PRs, the reason to use 2020b is that QIIME v2022.8 w/ Python >= 3.9 fails simple tasks, related to the Python ast module)

[migueldiascosta]

hm, using the distributed pretrained classifiers requires scikit-learn 0.24.1 specifically (https://github.com/qiime2/docs/blob/master/source/data-resources.rst)

not sure that's worth the dependency variant exception (would also affect scikit-bio and umap-learn)

[lexming]

@migueldiascosta tests fail because two different versions of numba and LLVM get pulled in the dependency tree:

• R > Mesa > LLVM-11
• umap-learn > numba-0.52 > LLVM-10

The easiest solution seems to be to upgrade umap-learn to v0.5.3 which is compatible with numba v0.53 and LLVM 11. I made a PR for it in #17161

[boegelbot]

@migueldiascosta: Tests failed in GitHub Actions, see https://github.com/easybuilders/easybuild-easyconfigs/actions/runs/3963144339
Output from first failing test suite run:

FAIL: test_dep_versions_per_toolchain_generation (test.easyconfigs.easyconfigs.EasyConfigTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/easyconfigs/easyconfigs.py", line 883, in test_dep_versions_per_toolchain_generation
    self.assertFalse(multi_dep_vars, error_msg)
AssertionError: No multi-variant deps found for '^.*-(?P<tc_gen>20(1[89]|[2-9][0-9])[ab]).*\.eb$' easyconfigs:

found 2 variants of 'umap-learn' dependency in easyconfigs using '2020b' toolchain generation
* version: 0.4.6; versionsuffix:  as dep for set(['cell2location-0.05-alpha-fosscuda-2020b.eb'])
* version: 0.5.3; versionsuffix:  as dep for set(['QIIME2-2022.8-foss-2020b.eb'])


----------------------------------------------------------------------
Ran 16244 tests in 938.616s

FAILED (failures=1)
ERROR: Not all tests were successful

bleep, bloop, I'm just a bot (boegelbot v20200716.01)
Please talk to my owner @boegel if you notice me acting stupid),
or submit a pull request to https://github.com/boegel/boegelbot fix the problem.

[lexming]

QIIME2 also depends on MAFFT. See https://docs.qiime2.org/2023.2/tutorials/moving-pictures/

[boegelbot]

@migueldiascosta: Tests failed in GitHub Actions, see https://github.com/easybuilders/easybuild-easyconfigs/actions/runs/5186087489
Output from first failing test suite run:

FAIL: test_dep_versions_per_toolchain_generation (test.easyconfigs.easyconfigs.EasyConfigTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/runner/work/easybuild-easyconfigs/easybuild-easyconfigs/test/easyconfigs/easyconfigs.py", line 884, in test_dep_versions_per_toolchain_generation
    self.assertFalse(multi_dep_vars, error_msg)
AssertionError: ['umap-learn'] is not false : No multi-variant deps found for '^.*-(?P<tc_gen>20(1[89]|[2-9][0-9])[ab]).*\.eb$' easyconfigs:

found 2 variants of 'umap-learn' dependency in easyconfigs using '2020b' toolchain generation
* version: 0.4.6; versionsuffix:  as dep for {'cell2location-0.05-alpha-fosscuda-2020b.eb'}
* version: 0.5.3; versionsuffix:  as dep for {'QIIME2-2022.8-foss-2020b.eb'}


----------------------------------------------------------------------
Ran 17206 tests in 1085.219s

FAILED (failures=1)
ERROR: Not all tests were successful

bleep, bloop, I'm just a bot (boegelbot v20200716.01)
Please talk to my owner @boegel if you notice me acting stupid),
or submit a pull request to https://github.com/boegel/boegelbot fix the problem.

[migueldiascosta]

this is still WIP, I'll check if newer versions of QIIME2 still have problems with Python > 3.8, if they don't I'll submit a PR with a more recent toolchain

[migueldiascosta]

Nope, with QIIME2 2023.2 and Python 3.9.6 I still get e.g.

$ qiime tools validate $EBROOTQIIME2/lib/python3.9/site-packages/q2_sample_classifier/tests/data/vaw.qza
Traceback (most recent call last):
  File "$EBROOTQIIME2/lib/python3.9/site-packages/q2cli/builtin/tools.py", line 555, in validate
    result = qiime2.sdk.Result.load(path)
  File "$EBROOTQIIME2/lib/python3.9/site-packages/qiime2/sdk/result.py", line 79, in load
    archiver = archive.Archiver.load(filepath)
  File "$EBROOTQIIME2/lib/python3.9/site-packages/qiime2/core/archive/archiver.py", line 372, in load
    ref = cls(data_path, process_alias, Format(rec), cache)
  File "$EBROOTQIIME2/lib/python3.9/site-packages/qiime2/core/archive/format/v1.py", line 29, in __init__
    super().__init__(archive_record)
  File "$EBROOTQIIME2/lib/python3.9/site-packages/qiime2/core/archive/format/v0.py", line 72, in __init__
    self.type = sdk.parse_type(type)
  File "$EBROOTQIIME2/lib/python3.9/site-packages/qiime2/sdk/util.py", line 68, in parse_type
    type_expr = _parse.ast_to_type(_parse.string_to_ast(string))
  File "$EBROOTQIIME2/lib/python3.9/site-packages/qiime2/core/type/parse.py", line 32, in string_to_ast
    return _expr(expr.value)
  File "$EBROOTQIIME2/lib/python3.9/site-packages/qiime2/core/type/parse.py", line 47, in _expr
    field_expr = expr.slice.value
AttributeError: 'Name' object has no attribute 'value'

There was a problem loading $EBROOTQIIME2/lib/python3.9/site-packages/q2_sample_classifier/tests/data/vaw.qza as a QIIME 2 Result:

  'Name' object has no attribute 'value'

See above for debug info.

(this works with Python 3.8.6)

[migueldiascosta]

superseded by #18407, where the issue with python 3.9 ast was fixed, thanks @smoors !