test_remove_observer RuntimeError: There is no current event loop in thread 'Dummy-1'

[qstokkink]

From the frontpage builder.

Probably related to calling get_event_loop() directly outside of a unit test.

____________________ ERROR at setup of test_remove_observer ____________________
[gw6] linux -- Python 3.8.5 /usr/bin/python3

    @pytest.fixture(name="notifier")
    def fixture_notifier():
>       return Notifier()

src/tribler-core/tribler_core/tests/test_notifier.py:13: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
src/tribler-core/tribler_core/notifier.py:19: in __init__
    self._event_loop = get_event_loop()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <asyncio.unix_events._UnixDefaultEventLoopPolicy object at 0x7fd11f2ccd30>

    def get_event_loop(self):
        """Get the event loop for the current context.
    
        Returns an instance of EventLoop or raises an exception.
        """
        if (self._local._loop is None and
                not self._local._set_called and
                isinstance(threading.current_thread(), threading._MainThread)):
            self.set_event_loop(self.new_event_loop())
    
        if self._local._loop is None:
>           raise RuntimeError('There is no current event loop in thread %r.'
                               % threading.current_thread().name)
E           RuntimeError: There is no current event loop in thread 'Dummy-1'.

/usr/lib/python3.8/asyncio/events.py:639: RuntimeError

[drew2a]

It seems to be working now.

[devos50]

This does not seem to be fixed - I got it again when testing #6499. Also see here.

[qstokkink]

You could consider using IPv8's TestBase to stabilize these notifier tests. TestBase guarantees a stable event loop in the setUp.

[drew2a]

@kozlovsky probably found a cause of this error.

[kozlovsky]

Hopefully, I fixed the problem. Below is the description what was the reason for the bug:

Tribler test suite runs tests using the following command:

pytest src/tribler-core src/tribler-common -v -n 12 -p no:randomly
--cov=tribler_core --cov=tribler_common  --cov-report=xml:../output/coverage.xml --junitxml=test_report.xml

-n 12 means that pytest should run tests in parallel, using pytest-xdist that in turn uses execnet library as a backend.

Some time ago, the execnet library was broken after adding thread models for gevent/libevent. When pytest-xdist runs child processes to execute tests, each child process should receive incoming messages in the main thread, but due to a race condition, sometimes it gets a message and run a test in a new separate thread.

Tribler tests are not designed to be run in parallel in multiple threads. It can be the reason for numerous mysterious bugs, for example, when different parallel tests cause race conditions by modifying the same global variables. Also, a new thread does not have an event loop instance, so when an unfortunate test tries to get an event loop, it results in the following error:

RuntimeError: There is no current event loop in thread 'Dummy-1'.

After this breaking change, execnet lost its developer and was switched into a "maintenance-only mode". It is not easy to fix this obscure bug, and it is not possible to downgrade to the old version of execnet<1.2 that does not have this bug, as it is too old and does not work well on modern Python.

To fix the problem, I switched from execnet to eventlet. To do this, I changed the -n 12 pytest option in our Jenkins scripts to the following option: --tx '12*popen//execmodel=eventlet', so the full command now looks this way:

pytest src/tribler-core src/tribler-common -v --tx '12*popen//execmodel=eventlet' -p no:randomly
--cov=tribler_core --cov=tribler_common  --cov-report=xml:../output/coverage.xml --junitxml=test_report.xml

It seems that this change fixed the issue and improved test stability. After the change, our tests should be able to still work in parallel.

[drew2a]

@kozlovsky 👏 👏 👏

[synctext]

yeah! ! ! Solid work 👍👍👍