Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Inconsistently failing test_fetch_or_enqueue_task CI #3916

Open
bjester opened this issue Jan 24, 2023 · 2 comments
Open

Inconsistently failing test_fetch_or_enqueue_task CI #3916

bjester opened this issue Jan 24, 2023 · 2 comments
Assignees
Labels
DEV: backend DEV: dev-ops P0 - critical Priority: Release blocker or regression

Comments

@bjester
Copy link
Member

bjester commented Jan 24, 2023

Observed behavior

We've seen the following test failure in our Github CI builds inconsistently

=================================== FAILURES ===================================
_________________ AsyncTaskTestCase.test_fetch_or_enqueue_task _________________

self = <contentcuration.tests.test_asynctask.AsyncTaskTestCase testMethod=test_fetch_or_enqueue_task>

    def test_fetch_or_enqueue_task(self):
        expected_task = test_task.enqueue(self.user, is_test=True)
        async_result = test_task.fetch_or_enqueue(self.user, is_test=True)
>       self.assertEqual(expected_task.task_id, async_result.task_id)
E       AssertionError: 'dc51c290f1bc44f9b7ae19ad123c2c9f' != '49eae93f804f49a19fa363772149be43'
E       - dc51c290f1bc44f9b7ae19ad123c2c9f
E       + 49eae93f804f49a19fa363772149be43

contentcuration/contentcuration/tests/test_asynctask.py:210: AssertionError
@bjester
Copy link
Member Author

bjester commented Jan 24, 2023

Maybe related to this error in the test's task worker:

[2023-01-02 16:16:30,214: ERROR/ForkPoolWorker-2] Task test_task[b8dfd1125418491caee9362b4d08d070] raised unexpected: AssertionError("assert 0 == 1\n +  where 0 = <bound method QuerySet.count of <QuerySet []>>()\n +    where <bound method QuerySet.count of <QuerySet []>> = <QuerySet []>.count\n +      where <QuerySet []> = <bound method BaseManager._get_queryset_methods.<locals>.create_method.<locals>.manager_method of <django_celery_results.managers.TaskResultManager object at 0x7f5c9fb94a00>>(task_id='b8dfd1125418491caee9362b4d08d070')\n +        where <bound method BaseManager._get_queryset_methods.<locals>.create_method.<locals>.manager_method of <django_celery_results.managers.TaskResultManager object at 0x7f5c9fb94a00>> = <django_celery_results.managers.TaskResultManager object at 0x7f5c9fb94a00>.filter\n +          where <django_celery_results.managers.TaskResultManager object at 0x7f5c9fb94a00> = TaskResult.objects\n +        and   'b8dfd1125418491caee9362b4d08d070' = <Context: {'lang': 'py', 'task': 'test_task', 'id': 'b8dfd1125418491caee9362b4d08d070', 'shadow': None, 'eta': None, '...ks': None, 'errbacks': None, 'chain': None, 'chord': None, 'called_directly': False, '_children': [], '_protected': 1}>.id\n +          where <Context: {'lang': 'py', 'task': 'test_task', 'id': 'b8dfd1125418491caee9362b4d08d070', 'shadow': None, 'eta': None, '...ks': None, 'errbacks': None, 'chain': None, 'chord': None, 'called_directly': False, '_children': [], '_protected': 1}> = <@task: test_task of contentcuration at 0x7f5ca9d821f0>.request")
Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.9.16/x64/lib/python3.9/site-packages/celery/app/trace.py", line 451, in trace_task
    R = retval = fun(*args, **kwargs)
  File "/opt/hostedtoolcache/Python/3.9.16/x64/lib/python3.9/site-packages/celery/app/trace.py", line 734, in __protected_call__
    return self.run(*args, **kwargs)
  File "/home/runner/work/studio/studio/contentcuration/contentcuration/tests/test_asynctask.py", line 34, in test_task
    assert TaskResult.objects.filter(task_id=self.request.id).count() == 1
AssertionError: assert 0 == 1
 +  where 0 = <bound method QuerySet.count of <QuerySet []>>()
 +    where <bound method QuerySet.count of <QuerySet []>> = <QuerySet []>.count
 +      where <QuerySet []> = <bound method BaseManager._get_queryset_methods.<locals>.create_method.<locals>.manager_method of <django_celery_results.managers.TaskResultManager object at 0x7f5c9fb94a00>>(task_id='b8dfd1[125](https://github.com/learningequality/studio/actions/runs/3823743205/jobs/6505236178#step:8:126)418491caee9362b4d08d070')
 +        where <bound method BaseManager._get_queryset_methods.<locals>.create_method.<locals>.manager_method of <django_celery_results.managers.TaskResultManager object at 0x7f5c9fb94a00>> = <django_celery_results.managers.TaskResultManager object at 0x7f5c9fb94a00>.filter
 +          where <django_celery_results.managers.TaskResultManager object at 0x7f5c9fb94a00> = TaskResult.objects
 +        and   'b8dfd1125418491caee9362b4d08d070' = <Context: {'lang': 'py', 'task': 'test_task', 'id': 'b8dfd1125418491caee9362b4d08d070', 'shadow': None, 'eta': None, '...ks': None, 'errbacks': None, 'chain': None, 'chord': None, 'called_directly': False, '_children': [], '_protected': 1}>.id
 +          where <Context: {'lang': 'py', 'task': 'test_task', 'id': 'b8dfd1125418491caee9362b4d08d070', 'shadow': None, 'eta': None, '...ks': None, 'errbacks': None, 'chain': None, 'chord': None, 'called_directly': False, '_children': [], '_protected': 1}> = <@task: test_task of contentcuration at 0x7f5ca9d821f0>.request

@bjester bjester changed the title Inconsistently failing async task test in CI Inconsistently failing test_fetch_or_enqueue_task CI Jan 27, 2023
@bjester bjester added the P0 - critical Priority: Release blocker or regression label Apr 17, 2023
@bjester
Copy link
Member Author

bjester commented Apr 17, 2023

This has been happening more and more, primarily affecting PR checks, so bumping the priority

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
DEV: backend DEV: dev-ops P0 - critical Priority: Release blocker or regression
Projects
None yet
Development

No branches or pull requests

1 participant