Update psutil to 5.6.7 and PyElfTools to 0.27

[maxgerhardt]

Summary of changes

Update psutil version in requirements.txt from 5.6.6 to 5.6.7.

If I install the requirements on Windows 10 with Python 3.10.0 (python -m pip install -r mbed-os\requirements.txt) I am greeted with

Installing collected packages: psutil, mbed-ls, mbed-greentea, jsonschema, jinja2, hidapi, cmsis-pack-manager, Click, cbor
    Running setup.py install for psutil ... error
    ERROR: Command errored out with exit status 1:
[...]
    C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.29.30133\bin\HostX86\x64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -DPSUTIL_VERSION=566 -DPSUTIL_WINDOWS=1 -D_WIN32_WINNT=0x01000 -D_AVAIL_WINVER_=0x01000 -D_CRT_SECURE_NO_WARNINGS -DPSAPI_VERSION=1 -IC:\Users\Max Gerhardt\AppData\Local\Programs\Python\Python310\include -IC:\Users\Max Gerhardt\AppData\Local\Programs\Python\Python310\Include -IC:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.29.30133\include -IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\ucrt -IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\shared -IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\um -IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\winrt -IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\cppwinrt /Tcpsutil/_psutil_windows.c /Fobuild\temp.win-amd64-3.10\Release\psutil/_psutil_windows.obj
    _psutil_windows.c
    psutil/_psutil_windows.c(272): error C2094: label 'error' was undefined
    error: command 'C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\BuildTools\\VC\\Tools\\MSVC\\14.29.30133\\bin\\HostX86\\x64\\cl.exe' failed with exit code 2

The compile error is fixed in the next minor version of psutil in accordance to odoo/odoo#62919 (comment), more precisely giampaolo/psutil@e15fbda.

Also updates PyElfTools to 0.27 to get it working on Python 3.10.0. Otherwise

[mbed] ERROR: Unable to update "LoRaTLSv2" to latest revision in the current branch
---
[mbed] Working path "C:\Users\Max Gerhardt\Desktop\work\LoRaTLSv2" (program)
Traceback (most recent call last):
  File "C:\Users\Max Gerhardt\Desktop\work\LoRaTLSv2\mbed-os\tools\project.py", line 34, in <module>
    from tools.export import (
  File "C:\Users\Max Gerhardt\Desktop\work\LoRaTLSv2\mbed-os\tools\export\__init__.py", line 30, in <module>
    from ..config import ALLOWED_FEATURES
  File "C:\Users\Max Gerhardt\Desktop\work\LoRaTLSv2\mbed-os\tools\config\__init__.py", line 33, in <module>
    from ..arm_pack_manager import Cache
  File "C:\Users\Max Gerhardt\Desktop\work\LoRaTLSv2\mbed-os\tools\arm_pack_manager\__init__.py", line 29, in <module>
    from tools.flash_algo import PackFlashAlgo
  File "C:\Users\Max Gerhardt\Desktop\work\LoRaTLSv2\mbed-os\tools\flash_algo\__init__.py", line 34, in <module>
    from elftools.elf.elffile import ELFFile
  File "C:\Users\Max Gerhardt\AppData\Local\Programs\Python\Python310\lib\site-packages\elftools\elf\elffile.py", line 23, in <module>
    from ..common.utils import struct_parse, elf_assert
  File "C:\Users\Max Gerhardt\AppData\Local\Programs\Python\Python310\lib\site-packages\elftools\common\utils.py", line 12, in <module>
    from ..construct import ConstructError
  File "C:\Users\Max Gerhardt\AppData\Local\Programs\Python\Python310\lib\site-packages\elftools\construct\__init__.py", line 33, in <module>
    from .core import *
  File "C:\Users\Max Gerhardt\AppData\Local\Programs\Python\Python310\lib\site-packages\elftools\construct\core.py", line 3, in <module>
    from .lib.py3compat import BytesIO, advance_iterator, bchr
  File "C:\Users\Max Gerhardt\AppData\Local\Programs\Python\Python310\lib\site-packages\elftools\construct\lib\__init__.py", line 4, in <module>
    from .container import (Container, FlagsContainer, ListContainer,
  File "C:\Users\Max Gerhardt\AppData\Local\Programs\Python\Python310\lib\site-packages\elftools\construct\lib\container.py", line 5, in <module>
    from collections import MutableMapping
ImportError: cannot import name 'MutableMapping' from 'collections' (C:\Users\Max Gerhardt\AppData\Local\Programs\Python\Python310\lib\collections\__init__.py)
[mbed] ERROR: "C:\Users\Max Gerhardt\AppData\Local\Programs\Python\Python310\python.exe" returned error.

Is encountered. MutableMapping from collections was moved to collections.abc in newer Python versions (refer to e.g. pypa/setuptools#1401).

Impact of changes

Fixes installation of dependencies on Windows.

Migration actions required

Documentation

None.

---

Pull request type

[X] Patch update (Bug fix / Target update / Docs update / Test update / Refactor)
[] Feature update (New feature / Functionality change / New API)
[] Major update (Breaking change E.g. Return code change / API behaviour change)

---

Test results

[X] No Tests required for this change (E.g docs only update)
[] Covered by existing mbed-os tests (Greentea or Unittest)
[] Tests / results supplied as part of this PR

---

Reviewers

---

[ciarmcom]

@maxgerhardt, thank you for your changes.
@ARMmbed/mbed-os-maintainers please review.

[0xc0170]

Is this backward compatible? We currently test with older version of python (3.6 or so, I can find out if it would make a difference but I believe anything 3.x should work).

[maxgerhardt]

Per Github diff for psutil between 5.6.6 and 5.6.7 only the code compile error was fixed (giampaolo/psutil@release-5.6.6...release-5.6.7) and this should thus not affect any other Python version.

For PyElfTools they fixed the mentioned error in a way to be compatible with all newer Python3 versions while still having back-compatibility with the old one, as one can see in

https://github.com/eliben/pyelftools/blob/4e888804c57cdfb0f04b9e5171f8ba5b36165983/elftools/construct/lib/container.py#L5-L6

the original offending import is now refactored into py3compat.py which does

try:
    from collections.abc import MutableMapping  # python >= 3.3
except ImportError:
    from collections import MutableMapping  # python < 3.3

and is thus compatible with all Python versions.

In total from pyelftools 0.25 to 0.27 a lot has been updated (see eliben/pyelftools@v0.25...v0.27) but given that they CI test with python-version: [2.7, 3.6, 3.7, 3.8] there shouldn't be any problems. Per the README.rst update linked they however state that the minimum Python 3 version has increased from "3.x (x >= 2)" to "3.x (x >= 5)" -- so Python 3.4 from March 16, 2014 might not be supported anymore. I can also test that.

[0xc0170]

We will run internal tests and if all good we can run CI here tomorrow

[0xc0170]

Ci started

[mbed-ci]

Jenkins CI Test : ✔️ SUCCESS

Build Number: 1 | 🔒 Jenkins CI Job | 🌐 Logs & Artifacts

CLICK for Detailed Summary

jobs	Status
jenkins-ci/mbed-os-ci_unittests	✔️
jenkins-ci/mbed-os-ci_build-cloud-example-ARM	✔️
jenkins-ci/mbed-os-ci_build-cloud-example-GCC_ARM	✔️
jenkins-ci/mbed-os-ci_build-greentea-ARM	✔️
jenkins-ci/mbed-os-ci_build-greentea-GCC_ARM	✔️
jenkins-ci/mbed-os-ci_build-example-GCC_ARM	✔️
jenkins-ci/mbed-os-ci_build-example-ARM	✔️
jenkins-ci/mbed-os-ci_greentea-test	✔️