Add python 3.9 support

[IAlibay]

Is your feature request related to a problem?

Python 3.9 is now officially out 🎉 https://docs.python.org/3.9/whatsnew/3.9.html

We probably should aim to support it soon-ish?

Currently waiting on 3.9 to be released on conda before a test PR can be raised.

Additional context

Related issue: #2688

[richardjgowers]

inversely, do we not support it currently? I.e. was anything we used broken by it? (likely not...)

[IAlibay]

inversely, do we not support it currently? I.e. was anything we used broken by it? (likely not...)

Highly doubt anything will change (at least until we decide to start type hinting everything).

Just needs adding to the test matrix == reward :)

[tylerjereddy]

Will take a while for the ecosystem to build out wheels for 3.9 too--I'm working on that for SciPy, but we're also trying to get out Linux ARM64 wheels at the same time so a bit crazy..

[jbarnoud]

inversely, do we not support it currently? I.e. was anything we used broken by it? (likely not...)

I installed develop on python 3.9 and could run the tests fine. I got an error (#2979) but it was unrelated to the version of python. I got many tests skipped, though (361) so some of them may have failed.

[markf94]

Unfortunately, MDAnalysis does not support Python 3.9 as of right now.

Here's what's happening if I try to add mdanalysis as a dependency to a project using poetry with Python 3.9:

Using version ^1.0.1 for MDAnalysis

Updating dependencies
Resolving dependencies... (0.8s)

  SolverProblemError

  The current project's Python requirement (>=3.9.0,<4.0.0) is not compatible with some of the required packages Python requirement:
    - mdanalysis requires Python >=2.7,!=3.0.*,!=3.1.*,!=3.2.*,!=3.3,!=3.4.*,<3.9, so it will not be satisfied for Python >=3.9.0,<4.0.0
  
  Because no versions of mdanalysis match >1.0.1,<2.0.0
   and mdanalysis (==1.0.1) requires Python >=2.7,!=3.0.*,!=3.1.*,!=3.2.*,!=3.3,!=3.4.*,<3.9, mdanalysis is forbidden.
  So, because dummy-repo depends on MDAnalysis (>=1.0.1,<2.0.0), version solving failed.

  at /usr/local/lib/python3.9/site-packages/poetry/puzzle/solver.py:241 in _solve
      237│             packages = result.packages
      238│         except OverrideNeeded as e:
      239│             return self.solve_in_compatibility_mode(e.overrides, use_latest=use_latest)
      240│         except SolveFailure as e:
    → 241│             raise SolverProblemError(e)
      242│ 
      243│         results = dict(
      244│             depth_first_search(
      245│                 PackageNode(self._package, packages), aggregate_package_nodes

  • Check your dependencies Python requirement: The Python requirement can be specified via the `python` or `markers` properties
    
    For mdanalysis, a possible solution would be to set the `python` property to "<empty>"

    https://python-poetry.org/docs/dependency-specification/#python-restricted-dependencies,
    https://python-poetry.org/docs/dependency-specification/#using-environment-markers

How to reproduce this

1. Make sure you have Python 3.9 and poetry installed.
2. Create a new directory and add this pyproject.toml:

[tool.poetry]
name = "dummy_repo"
version = "0.0.0"
description = "Dummy."
authors = ["Someone"]
license = "MIT"


[tool.poetry.dependencies]
python = "^3.9.0"


[build-system]
requires = ['poetry-core~=1.0']
build-backend = 'poetry.core.masonry.api'

3. Run poetry install to setup the poetry virtual environment (which will be using Python 3.9)
4. Now let's try to add mdanalysis with:

$ poetry add mdanalysis

This should fail with the above error message.

[Seanny123]

To expand on what @markf94 has submitted, it's not that MDAnalysis code doesn't support Python 3.9, it's the package specification that needs to be updated, so it can work with stricter package manages such as Poetry.

Looking at the MDAnalysis Roadmap, I understand if this doesn't happen until 2.0 is released.

[IAlibay]

Indeed, we purposely aren't making 1.0.x py3.9 compatible. 1.0.x is now a legacy bugfix only release which will support py2.7, and py3.5-3.8 (although support for py2.7 and py3.5 is getting increasingly hard to guarantee).

Work is currently in progress to bring py3.9 along to 2.0.0, however not all our optional dependencies have a py3.9 release (at least last I checked). edit: as far as I'm aware there are no code issues on our ends, it's just waiting for everything upstream to align correctly.

To expand on what @markf94 has submitted, it's not that MDAnalysis code doesn't support Python 3.9, it's the package specification that needs to be updated, so it can work with stricter package manages such as Poetry.

@Seanny123 I'm not sure I understand what you mean here. I've not used poetry before so maybe I'm completely wrong, but from the error message that @markf94 shows it seems to be correctly capturing from setup.py that the python version doesn't fit the specs? Any input on how we can improve our setup to better fit more package managers is appreciated.

[markf94]

Yeah poetry is very strict when it comes to package versions. @Seanny123 basically said that there is nothing preventing you guys from making MDAnalysis officially Python 3.9 compatible by updating your setup.py since your code would still work. However, I understand your reasoning behind not doing that yet because of your dependencies.