Timeout during `test_download_torrent_from_url` #6606

devos50 · 2021-12-01T13:14:06Z

Encountered a timeout during test_download_torrent_from_url (see here).

IIRC, this test setups a local file server and seeder, and then downloads a torrent.

Logs:

failed on teardown with "Failed: Timeout >10.0s"

def finalizer():
        """Yield again, to finalize."""
        async def async_finalizer():
            try:
                await gen_obj.__anext__()
            except StopAsyncIteration:
                pass
            else:
                msg = "Async generator fixture didn't stop."
                msg += "Yield only once."
                raise ValueError(msg)
>       loop.run_until_complete(async_finalizer())

/usr/local/lib/python3.8/site-packages/pytest_asyncio/plugin.py:124: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/usr/local/Cellar/[email protected]/3.8.4/Frameworks/Python.framework/Versions/3.8/lib/python3.8/asyncio/base_events.py:603: in run_until_complete
    self.run_forever()
/usr/local/Cellar/[email protected]/3.8.4/Frameworks/Python.framework/Versions/3.8/lib/python3.8/asyncio/base_events.py:570: in run_forever
    self._run_once()
/usr/local/Cellar/[email protected]/3.8.4/Frameworks/Python.framework/Versions/3.8/lib/python3.8/asyncio/base_events.py:1823: in _run_once
    event_list = self._selector.select(timeout)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <selectors.KqueueSelector object at 0x109307ee0>
timeout = 0.9984638159999975

    def select(self, timeout=None):
        timeout = None if timeout is None else max(timeout, 0)
        max_ev = len(self._fd_to_key)
        ready = []
        try:
>           kev_list = self._selector.control(None, max_ev, timeout)
E           Failed: Timeout >10.0s

/usr/local/Cellar/[email protected]/3.8.4/Frameworks/Python.framework/Versions/3.8/lib/python3.8/selectors.py:558: Failed

The text was updated successfully, but these errors were encountered:

drew2a · 2022-03-23T12:07:46Z

This behavior could be caused by parallel Jobs executions.
I disabled concurrent builds for pytest on macOS to observe will it have an effect or not:

UPD: it doesn't help. Jenkins' settings have been reverted back.

drew2a · 2022-04-11T12:30:45Z

All tested-related issues should be solved in #6851 after merging #6850 and #6855.

If the issue is encountered again, feel free to reopen it.

qstokkink · 2023-11-21T10:49:14Z

Encountered again in https://github.com/Tribler/tribler/actions/runs/6942446753/job/18885431246?pr=7705

Raw logs:
33.txt (this provides very little info though)

One noteworthy thing is that the test before this also slows down a lot, taking 60.342s:

 src/tribler/core/components/libtorrent/tests/test_download_api.py::test_download_torrent_from_file_with_escaped_characters FAILED [ 83%] in 60.342s (4 minutes in total)
10:32:37.231 Starting "test_download_torrent_from_url"...
Timeout (0:01:10)!

In comparison, in a normal run it takes 0.561s:

src/tribler/core/components/libtorrent/tests/test_download_api.py::test_download_torrent_from_file_with_escaped_characters PASSED [ 83%] in 0.561s
src/tribler/core/components/libtorrent/tests/test_download_api.py::test_download_torrent_from_url PASSED [ 83%] in 0.567s

drew2a · 2023-11-21T10:59:47Z

The good news is that faulthandler is working for this type of freezes

Timeout (0:01:10)!
Thread 0x00007f72103ff640 (most recent call first):
  File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/concurrent/futures/thread.py", line 78 in _worker
  File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/threading.py", line 870 in run
  File "/home/runner/.virtualenvs/.venv/lib/python3.8/site-packages/sentry_sdk/integrations/threading.py", line 70 in run
  File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/threading.py", line 932 in _bootstrap_inner
  File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/threading.py", line 890 in _bootstrap

Thread 0x00007f71fe57d640 (most recent call first):
  File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/threading.py", line 302 in wait
  File "/home/runner/.virtualenvs/.venv/lib/python3.8/site-packages/sentry_sdk/_queue.py", line 240 in get
  File "/home/runner/.virtualenvs/.venv/lib/python3.8/site-packages/sentry_sdk/worker.py", line 127 in _target
  File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/threading.py", line 870 in run
  File "/home/runner/.virtualenvs/.venv/lib/python3.8/site-packages/sentry_sdk/integrations/threading.py", line 70 in run
  File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/threading.py", line 932 in _bootstrap_inner
  File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/threading.py", line 890 in _bootstrap

Thread 0x00007f720057f640 (most recent call first):
  File "/home/runner/.virtualenvs/.venv/lib/python3.8/site-packages/sentry_sdk/monitor.py", line 50 in _thread
  File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/threading.py", line 870 in run
  File "/home/runner/.virtualenvs/.venv/lib/python3.8/site-packages/sentry_sdk/integrations/threading.py", line 70 in run
  File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/threading.py", line 932 in _bootstrap_inner
  File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/threading.py", line 890 in _bootstrap

Thread 0x00007f72195abb80 (most recent call first):
  File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/asyncio/base_events.py", line 1823 in _run_once
  File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/asyncio/base_events.py", line 570 in run_forever
  File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/asyncio/base_events.py", line 603 in run_until_complete
  File "/home/runner/.virtualenvs/.venv/lib/python3.8/site-packages/pytest_asyncio/plugin.py", line 532 in inner
  File "/home/runner/.virtualenvs/.venv/lib/python3.8/site-packages/_pytest/python.py", line 194 in pytest_pyfunc_call
  File "/home/runner/.virtualenvs/.venv/lib/python3.8/site-packages/pluggy/_callers.py", line 77 in _multicall
  File "/home/runner/.virtualenvs/.venv/lib/python3.8/site-packages/pluggy/_manager.py", line 115 in _hookexec
  File "/home/runner/.virtualenvs/.venv/lib/python3.8/site-packages/pluggy/_hooks.py", line 493 in __call__
  File "/home/runner/.virtualenvs/.venv/lib/python3.8/site-packages/_pytest/python.py", line 1792 in runtest
  File "/home/runner/.virtualenvs/.venv/lib/python3.8/site-packages/_pytest/runner.py", line 169 in pytest_runtest_call
  File "/home/runner/.virtualenvs/.venv/lib/python3.8/site-packages/pluggy/_callers.py", line 77 in _multicall
  File "/home/runner/.virtualenvs/.venv/lib/python3.8/site-packages/pluggy/_manager.py", line 115 in _hookexec
  File "/home/runner/.virtualenvs/.venv/lib/python3.8/site-packages/pluggy/_hooks.py", line 493 in __call__
  File "/home/runner/.virtualenvs/.venv/lib/python3.8/site-packages/_pytest/runner.py", line 262 in <lambda>
  File "/home/runner/.virtualenvs/.venv/lib/python3.8/site-packages/_pytest/runner.py", line 341 in from_call
  File "/home/runner/.virtualenvs/.venv/lib/python3.8/site-packages/_pytest/runner.py", line 261 in call_runtest_hook
  File "/home/runner/.virtualenvs/.venv/lib/python3.8/site-packages/_pytest/runner.py", line 222 in call_and_report
  File "/home/runner/.virtualenvs/.venv/lib/python3.8/site-packages/_pytest/runner.py", line 133 in runtestprotocol
  File "/home/runner/.virtualenvs/.venv/lib/python3.8/site-packages/_pytest/runner.py", line 114 in pytest_runtest_protocol
  File "/home/runner/.virtualenvs/.venv/lib/python3.8/site-packages/pluggy/_callers.py", line 77 in _multicall
  File "/home/runner/.virtualenvs/.venv/lib/python3.8/site-packages/pluggy/_manager.py", line 115 in _hookexec
  File "/home/runner/.virtualenvs/.venv/lib/python3.8/site-packages/pluggy/_hooks.py", line 493 in __call__
  File "/home/runner/.virtualenvs/.venv/lib/python3.8/site-packages/_pytest/main.py", line 350 in pytest_runtestloop
  File "/home/runner/.virtualenvs/.venv/lib/python3.8/site-packages/pluggy/_callers.py", line 77 in _multicall
  File "/home/runner/.virtualenvs/.venv/lib/python3.8/site-packages/pluggy/_manager.py", line 115 in _hookexec
  File "/home/runner/.virtualenvs/.venv/lib/python3.8/site-packages/pluggy/_hooks.py", line 493 in __call__
  File "/home/runner/.virtualenvs/.venv/lib/python3.8/site-packages/_pytest/main.py", line 325 in _main
  File "/home/runner/.virtualenvs/.venv/lib/python3.8/site-packages/_pytest/main.py", line 271 in wrap_session
  File "/home/runner/.virtualenvs/.venv/lib/python3.8/site-packages/_pytest/main.py", line 318 in pytest_cmdline_main
  File "/home/runner/.virtualenvs/.venv/lib/python3.8/site-packages/pluggy/_callers.py", line 77 in _multicall
  File "/home/runner/.virtualenvs/.venv/lib/python3.8/site-packages/pluggy/_manager.py", line 115 in _hookexec
  File "/home/runner/.virtualenvs/.venv/lib/python3.8/site-packages/pluggy/_hooks.py", line 493 in __call__
  File "/home/runner/.virtualenvs/.venv/lib/python3.8/site-packages/_pytest/config/__init__.py", line 169 in main
  File "/home/runner/.virtualenvs/.venv/lib/python3.8/site-packages/_pytest/config/__init__.py", line 192 in console_main
  File "/home/runner/.virtualenvs/.venv/lib/python3.8/site-packages/pytest/__main__.py", line 5 in <module>
  File "/home/runner/.virtualenvs/.venv/lib/python3.8/site-packages/coverage/execfile.py", line 211 in run
  File "/home/runner/.virtualenvs/.venv/lib/python3.8/site-packages/coverage/cmdline.py", line 861 in do_run
  File "/home/runner/.virtualenvs/.venv/lib/python3.8/site-packages/coverage/cmdline.py", line 684 in command_line
  File "/home/runner/.virtualenvs/.venv/lib/python3.8/site-packages/coverage/cmdline.py", line 973 in main
  File "/home/runner/.virtualenvs/.venv/bin/coverage", line 8 in <module>

drew2a · 2023-11-21T11:49:25Z

This could be a potential deadlock:

Thread 0x00007f71fe57d640 (most recent call first):
  File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/threading.py", line 302 in wait
  File "/home/runner/.virtualenvs/.venv/lib/python3.8/site-packages/sentry_sdk/_queue.py", line 240 in get
  File "/home/runner/.virtualenvs/.venv/lib/python3.8/site-packages/sentry_sdk/worker.py", line 127 in _target
  File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/threading.py", line 870 in run
  File "/home/runner/.virtualenvs/.venv/lib/python3.8/site-packages/sentry_sdk/integrations/threading.py", line 70 in run
  File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/threading.py", line 932 in _bootstrap_inner
  File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/threading.py", line 890 in _bootstrap

As we don't initialize Sentry during tests, the only likely candidate for Sentry initialization is pytest-sentry:

tribler/requirements-test.txt

Line 9 in e44679b

pytest-sentry==0.1.16

Perhaps we could try removing this integration to see if it makes a difference.

devos50 added type: bug estimation: unknown component: tests labels Dec 1, 2021

devos50 added this to the 7.12.0 December milestone Dec 1, 2021

drew2a closed this as completed Apr 11, 2022

qstokkink reopened this Nov 21, 2023

drew2a mentioned this issue Nov 21, 2023

Frozen Tests Registry #7706

Closed

6 tasks

drew2a mentioned this issue Nov 21, 2023

Remove pytest-sentry #7708

Merged

drew2a closed this as completed Dec 4, 2023

drew2a mentioned this issue Dec 4, 2023

Intermittent test suite freezing in GitHub Actions potentially linked to pytest-sentry conflict getsentry/pytest-sentry#25

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Timeout during `test_download_torrent_from_url` #6606

Timeout during `test_download_torrent_from_url` #6606

devos50 commented Dec 1, 2021

drew2a commented Mar 23, 2022 •

edited

Loading

drew2a commented Apr 11, 2022

qstokkink commented Nov 21, 2023 •

edited

Loading

drew2a commented Nov 21, 2023

drew2a commented Nov 21, 2023 •

edited

Loading

Timeout during test_download_torrent_from_url #6606

Timeout during test_download_torrent_from_url #6606

Comments

devos50 commented Dec 1, 2021

drew2a commented Mar 23, 2022 • edited Loading

drew2a commented Apr 11, 2022

qstokkink commented Nov 21, 2023 • edited Loading

drew2a commented Nov 21, 2023

drew2a commented Nov 21, 2023 • edited Loading

Timeout during `test_download_torrent_from_url` #6606

Timeout during `test_download_torrent_from_url` #6606

drew2a commented Mar 23, 2022 •

edited

Loading

qstokkink commented Nov 21, 2023 •

edited

Loading

drew2a commented Nov 21, 2023 •

edited

Loading