Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

No CI for Python 3.10 yet #2274

Closed
dgw opened this issue May 2, 2022 · 9 comments · Fixed by #2342
Closed

No CI for Python 3.10 yet #2274

dgw opened this issue May 2, 2022 · 9 comments · Fixed by #2342
Labels
Build
Milestone
8.0.0

Comments

@dgw
Copy link
Member

dgw commented May 2, 2022

Tin. We've still not enabled a CI job for Python 3.10 because of various errors encountered along the way while attempting to do so.

Ultimately, the cause is probably related to pytest-dev/pytest#9174, an issue that was closed but it seems not confirmed fixed. Uncertainty remains as to whether fixing this problem is up to the pytest team or the core Python developers. It looks like a regression (breaking something that worked in Python 3.9 and prior versions) but it appears to be taking longer to sort out than the argparse stuff we dealt with a while back.

While we believe Sopel should run just fine on Python 3.10 in most cases, it's not currently possible to test Sopel on that version.

Log from last attempt (CPython 3.10.4, pytest 6.2.5 
coverage run -m py.test -v . --vcr-record=none
Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.[10](https://github.com/sopel-irc/sopel/runs/6252478452?check_suite_focus=true#step:7:10).4/x64/lib/python3.10/site-packages/py/test.py", line 4, in <module>
    sys.exit(pytest.main())
  File "/opt/hostedtoolcache/Python/3.10.4/x64/lib/python3.10/site-packages/_pytest/config/__init__.py", line 143, in main
    config = _prepareconfig(args, plugins)
  File "/opt/hostedtoolcache/Python/3.10.4/x64/lib/python3.10/site-packages/_pytest/config/__init__.py", line 318, in _prepareconfig
    config = pluginmanager.hook.pytest_cmdline_parse(
  File "/opt/hostedtoolcache/Python/3.10.4/x64/lib/python3.10/site-packages/pluggy/_hooks.py", line 265, in __call__
    return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
  File "/opt/hostedtoolcache/Python/3.10.4/x64/lib/python3.10/site-packages/pluggy/_manager.py", line 80, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/opt/hostedtoolcache/Python/3.10.4/x64/lib/python3.10/site-packages/pluggy/_callers.py", line 55, in _multicall
    gen.send(outcome)
  File "/opt/hostedtoolcache/Python/3.10.4/x64/lib/python3.10/site-packages/_pytest/helpconfig.py", line 100, in pytest_cmdline_parse
    config: Config = outcome.get_result()
  File "/opt/hostedtoolcache/Python/3.10.4/x64/lib/python3.10/site-packages/pluggy/_result.py", line 60, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/opt/hostedtoolcache/Python/3.10.4/x64/lib/python3.10/site-packages/pluggy/_callers.py", line 39, in _multicall
    res = hook_impl.function(*args)
  File "/opt/hostedtoolcache/Python/3.10.4/x64/lib/python3.10/site-packages/_pytest/config/__init__.py", line 1003, in pytest_cmdline_parse
    self.parse(args)
  File "/opt/hostedtoolcache/Python/3.10.4/x64/lib/python3.10/site-packages/_pytest/config/__init__.py", line 1283, in parse
    self._preparse(args, addopts=addopts)
  File "/opt/hostedtoolcache/Python/3.10.4/x64/lib/python3.10/site-packages/_pytest/config/__init__.py", line [11](https://github.com/sopel-irc/sopel/runs/6252478452?check_suite_focus=true#step:7:11)72, in _preparse
    self.pluginmanager.load_setuptools_entrypoints("pytest11")
  File "/opt/hostedtoolcache/Python/3.10.4/x64/lib/python3.10/site-packages/pluggy/_manager.py", line 287, in load_setuptools_entrypoints
    plugin = ep.load()
  File "/opt/hostedtoolcache/Python/3.10.4/x64/lib/python3.10/importlib/metadata/__init__.py", line 171, in load
    module = import_module(match.group('module'))
  File "/opt/hostedtoolcache/Python/3.10.4/x64/lib/python3.10/importlib/__init__.py", line [12](https://github.com/sopel-irc/sopel/runs/6252478452?check_suite_focus=true#step:7:12)6, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "/opt/hostedtoolcache/Python/3.10.4/x64/lib/python3.10/site-packages/_pytest/assertion/rewrite.py", line 170, in exec_module
    exec(co, module.__dict__)
  File "/home/runner/work/sopel/sopel/sopel/tests/pytest_plugin.py", line 12, in <module>
    from sopel import bot, loader, plugins, trigger
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "/opt/hostedtoolcache/Python/3.10.4/x64/lib/python3.10/site-packages/_pytest/assertion/rewrite.py", line 170, in exec_module
    exec(co, module.__dict__)
  File "/home/runner/work/sopel/sopel/sopel/bot.py", line 22, in <module>
    from sopel import db, irc, logger, plugin, plugins, tools
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "/opt/hostedtoolcache/Python/3.10.4/x64/lib/python3.10/site-packages/_pytest/assertion/rewrite.py", line 170, in exec_module
    exec(co, module.__dict__)
  File "/home/runner/work/sopel/sopel/sopel/irc/__init__.py", line 37, in <module>
    from .backends import AsynchatBackend, SSLAsynchatBackend
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "/opt/hostedtoolcache/Python/3.10.4/x64/lib/python3.10/site-packages/_pytest/assertion/rewrite.py", line 170, in exec_module
    exec(co, module.__dict__)
  File "/home/runner/work/sopel/sopel/sopel/irc/backends.py", line 20, in <module>
    from .abstract_backends import AbstractIRCBackend
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "/opt/hostedtoolcache/Python/3.10.4/x64/lib/python3.10/site-packages/_pytest/assertion/rewrite.py", line 170, in exec_module
    exec(co, module.__dict__)
  File "/home/runner/work/sopel/sopel/sopel/irc/abstract_backends.py", line 8, in <module>
    from .utils import safe
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "/opt/hostedtoolcache/Python/3.10.4/x64/lib/python3.10/site-packages/_pytest/assertion/rewrite.py", line 170, in exec_module
    exec(co, module.__dict__)
  File "/home/runner/work/sopel/sopel/sopel/irc/utils.py", line 8, in <module>
    from dns import rdtypes, resolver
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "/opt/hostedtoolcache/Python/3.10.4/x64/lib/python3.10/site-packages/_pytest/assertion/rewrite.py", line 170, in exec_module
    exec(co, module.__dict__)
  File "/opt/hostedtoolcache/Python/3.10.4/x64/lib/python3.10/site-packages/dns/resolver.py", line 38, in <module>
    import dns.query
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "/opt/hostedtoolcache/Python/3.10.4/x64/lib/python3.10/site-packages/_pytest/assertion/rewrite.py", line 170, in exec_module
    exec(co, module.__dict__)
  File "/opt/hostedtoolcache/Python/3.10.4/x64/lib/python3.10/site-packages/dns/query.py", line 42, in <module>
    import requests
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "/opt/hostedtoolcache/Python/3.10.4/x64/lib/python3.10/site-packages/_pytest/assertion/rewrite.py", line 170, in exec_module
    exec(co, module.__dict__)
  File "/opt/hostedtoolcache/Python/3.10.4/x64/lib/python3.10/site-packages/requests/__init__.py", line [13](https://github.com/sopel-irc/sopel/runs/6252478452?check_suite_focus=true#step:7:13)3, in <module>
    from . import utils
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "/opt/hostedtoolcache/Python/3.10.4/x64/lib/python3.10/site-packages/_pytest/assertion/rewrite.py", line [17](https://github.com/sopel-irc/sopel/runs/6252478452?check_suite_focus=true#step:7:17)0, in exec_module
    exec(co, module.__dict__)
  File "/opt/hostedtoolcache/Python/3.10.4/x64/lib/python3.10/site-packages/requests/utils.py", line 42, in <module>
    DEFAULT_CA_BUNDLE_PATH = certs.where()
  File "/opt/hostedtoolcache/Python/3.10.4/x64/lib/python3.10/site-packages/certifi/core.py", line 37, in where
    _CACERT_PATH = str(_CACERT_CTX.__enter__())
  File "/opt/hostedtoolcache/Python/3.10.4/x64/lib/python3.10/contextlib.py", line 1[35](https://github.com/sopel-irc/sopel/runs/6252478452?check_suite_focus=true#step:7:35), in __enter__
    return next(self.gen)
  File "/opt/hostedtoolcache/Python/3.10.4/x64/lib/python3.10/importlib/_common.py", line 89, in _tempfile
    os.write(fd, reader())
  File "/opt/hostedtoolcache/Python/3.10.4/x64/lib/python3.10/importlib/abc.py", line [37](https://github.com/sopel-irc/sopel/runs/6252478452?check_suite_focus=true#step:7:37)1, in read_bytes
    with self.open('rb') as strm:
  File "/opt/hostedtoolcache/Python/3.10.4/x64/lib/python3.10/importlib/_adapters.py", line [54](https://github.com/sopel-irc/sopel/runs/6252478452?check_suite_focus=true#step:7:54), in open
    raise ValueError()
ValueError
make: *** [Makefile:17: test_norecord] Error 1
Error: Process completed with exit code 2.
@dgw dgw added the Build label May 2, 2022
@dgw dgw added this to the 8.0.0 milestone May 2, 2022
@half-duplex
Copy link
Member

Is there a reason not to just update to pytest 7? It seems to mostly work on 3.10, save 10 of these:

ERROR sopel/modules/currency.py::test_example_exchange_cmd_0 - AttributeError: module 'sopel.modules.currency' has no attribute 'config'
ERROR sopel/modules/ip.py::test_example_ip_0 - AttributeError: module 'sopel.modules.ip' has no attribute 'config'
ERROR sopel/modules/translate.py::test_example_tr2_0 - AttributeError: module 'sopel.modules.translate' has no attribute 'memory'

@dgw
Copy link
Member Author

dgw commented May 2, 2022

Those are pretty weird ones. WTF?

@half-duplex
Copy link
Member

Yeah, no idea. I tried to dig into it but got confused trying to figure out how sopel tests @example stuff, and the pytest release notes are Long and I didn't know what I was looking for.

@dgw
Copy link
Member Author

dgw commented May 2, 2022

It sounds similar to what people were describing in that issue I linked, where pytest rewrites stuff it shouldn't or otherwise gets confused by namespaces. They seem to think it's importlib's fault.

@half-duplex
Copy link
Member

Same errors on python 3.9.2 / pytest 7.1.2 though 🤔

@dgw
Copy link
Member Author

dgw commented May 2, 2022 •
edited
Loading

🤔 Maybe we should ask @Exirel to take a look? He wrote the pytest plugin that generates/handles the @example tests, and would therefore probably have some idea what to look for in the Long release notes.

@dgw
Copy link
Member Author

dgw commented May 3, 2022

@half-duplex I dug around a bit and to me the errors seem monkeypatch-related. There aren't any notable (again, to me) changes to that part of pytest though, not since 6.2.x, so idk why it's broken. Nonetheless, that breakage doesn't appear to be py3.10-related. Upgrading to pytest==7.1.2 under py3.9 yields the same kind of error.

@Exirel
Copy link
Contributor

Exirel commented Aug 11, 2022

Something something pytest support nose, and it considers any setup function to be a nose-style setup function.

Long story short: use -p no:nose option when running pytest. This should be fixed with Pytest 7.2 (to be released). I hope. I'm sure @dgw can summarize our conversation on IRC.

@dgw
Copy link
Member Author

dgw commented Aug 11, 2022

We (OK, mostly @Exirel) pinned down a way to get the tests working on Python 3.10 and pytest 7:

diff --git a/setup.cfg b/setup.cfg
index b64a8e4b..c3df75a1 100644
--- a/setup.cfg
+++ b/setup.cfg
@@ -69,5 +69,5 @@ show_error_codes = True
 
 [tool:pytest]
 python_files = *.py
-addopts = --tb=short
+addopts = --tb=short -p no:nose
 norecursedirs = contrib

That should let us upgrade pytest, add Python 3.10 to CI, and finally resolve the conflict that has kept this from happening sooner between "new pytest breaks stuff" and "old pytest is broken on 3.10".

The short story is, pytest added new behavior treating any module-level function named setup as a test setup fixture. Disabling pytest's nose plugin is the solution, as shown above. Meanwhile I/we will keep an eye on further discussion and developments in pytest-dev/pytest#9549, pytest-dev/pytest#9886, & pytest-dev/pytest#9907.

Said nose plugin is for backward compatibility with an older Python test library, which has been in maintenance mode for a long time, but I guess people might not feel like completely reworking their test suites for pytest. The plugin lets them run the old tests under pytest. Ideally that plugin wouldn't be loaded by default, and would only take action if the pytest configuration explicitly requests it, but at least it's fairly easy to turn off…

@Exirel Exirel mentioned this issue Aug 13, 2022
Merged
4 tasks
@dgw dgw closed this as completed in #2342 Aug 21, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Build
Projects
None yet
Milestone
8.0.0
Development

Successfully merging a pull request may close this issue.

ci, plugins: Python 3.10 and other improvements Exirel/sopel
3 participants
@dgw @half-duplex @Exirel