[Main] test_unhandled_error_observer_only_message fails randomly

[drew2a]

I saw test_unhandled_error_observer_only_message fail on Windows during the #6540:

Console full

async def test_unhandled_error_observer_only_message():
        # test that unhandled exception, represented by message, reported to the GUI
        context = {'message': 'Any'}
        CoreExceptionHandler.report_callback = MagicMock()
        CoreExceptionHandler.unhandled_error_observer(None, context)
        CoreExceptionHandler.report_callback.assert_called()
    
        # get the argument that has been passed to the report_callback
        reported_error = CoreExceptionHandler.report_callback.call_args_list[-1][0][0]
        assert reported_error.type == 'Exception'
        assert reported_error.text == 'Received error without exception: Any'
        assert reported_error.long_text == 'Exception: Received error without exception: Any\n'
        assert not reported_error.event
        assert reported_error.context == '{}'
        assert reported_error.should_stop
>       assert reported_error.requires_user_consent
E       AssertionError: assert False
E        +  where False = ReportedError(type='Exception', text='Received error without exception: Any', event={}, long_text='Exception: Received error without exception: Any\n', context='{}', should_stop=True, requires_user_consent=False).requires_user_consent

src\tribler-core\tribler_core\components\reporter\tests\test_exception_handler.py:93: AssertionError

as well as test_unhandled_error_observer_exception:

@patch(f'{sentry_reporter.__name__}.{SentryReporter.__name__}.{SentryReporter.event_from_exception.__name__}',
           new=MagicMock(return_value={'sentry': 'event'}))
    async def test_unhandled_error_observer_exception():
        # test that unhandled exception, represented by Exception, reported to the GUI
        context = {'exception': raise_error(AttributeError('Any')), 'Any key': 'Any value'}
        CoreExceptionHandler.report_callback = MagicMock()
        CoreExceptionHandler.unhandled_error_observer(None, context)
        CoreExceptionHandler.report_callback.assert_called()
    
        # get the argument that has been passed to the report_callback
        reported_error = CoreExceptionHandler.report_callback.call_args_list[-1][0][0]
        assert reported_error.type == 'AttributeError'
        assert reported_error.text == 'Any'
        assert 'raise_error' in reported_error.long_text
        assert reported_error.event == {'sentry': 'event'}
        assert reported_error.context == "{'Any key': 'Any value'}"
        assert reported_error.should_stop
>       assert reported_error.requires_user_consent
E       assert False
E        +  where False = ReportedError(type='AttributeError', text='Any', event={'sentry': 'event'}, long_text='Traceback (most recent call las...raise error\nAttributeError: Any\n', context="{'Any key': 'Any value'}", should_stop=True, requires_user_consent=False).requires_user_consent

[drew2a]

My guess is the root cause of the error is the following: CoreExceptionHandler (that designed as a singletone) is tested by async tests.

And the fragile part of the failed is:

async def test_unhandled_error_observer_only_message():
     ...
    # get the argument that has been passed to the report_callback
    reported_error = CoreExceptionHandler.report_callback.call_args_list[-1][0][0]
    ...    

[kozlovsky]

As we mock global objects, we should not run tests in parallel in multiple threads. I think it should not significantly affect performance, as Python's use of GIL prevents simultaneous execution anyway.

[devos50]

This test seemed to fail again when testing #6600 with the following traceback:

exception_handler = <tribler_core.components.reporter.exception_handler.CoreExceptionHandler object at 0x00000000299319D0>

    async def test_unhandled_error_observer_only_message(exception_handler):
        # test that unhandled exception, represented by message, reported to the GUI
        context = {'message': 'Any'}
        exception_handler.report_callback = MagicMock()
        exception_handler.unhandled_error_observer(None, context)
        exception_handler.report_callback.assert_called()
    
        # get the argument that has been passed to the report_callback
        reported_error = exception_handler.report_callback.call_args_list[-1][0][0]
        assert reported_error.type == 'Exception'
        assert reported_error.text == 'Received error without exception: Any'
        assert reported_error.long_text == 'Exception: Received error without exception: Any\n'
>       assert not reported_error.event
E       AssertionError: assert not {'a': 'b'}
E        +  where {'a': 'b'} = ReportedError(type='Exception', text='Received error without exception: Any', event={'a': 'b'}, long_text='Exception: Received error without exception: Any\n', context='{}', should_stop=True).event

src\tribler-core\tribler_core\components\reporter\tests\test_exception_handler.py:94: AssertionError

Should we create a new issue or use this one?

[drew2a]

Let's use this one.

[drew2a]

All tested-related issues should be solved in #6851 after merging #6850 and #6855.

If the issue is encountered again, feel free to reopen it.