Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixing flaky test test_cli_can_build_and_search_index by retrying #816

Merged
merged 1 commit into from
Jan 17, 2025

Conversation

jamesbraza
Copy link
Collaborator

We can see from this CI run that test_cli_can_build_and_search_index flaked in build_index with a zlib.error:

  |   File "/home/runner/work/paper-qa/paper-qa/tests/test_cli.py", line 79, in test_cli_can_build_and_search_index
  |     build_index(index_name, stub_data_dir, settings)
  |   File "/home/runner/work/paper-qa/paper-qa/paperqa/agents/__init__.py", line 141, in build_index
  |     return get_loop().run_until_complete(get_directory_index(settings=settings))
  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  |   File "/home/runner/.local/share/uv/python/cpython-3.12.8-linux-x86_64-gnu/lib/python3.12/asyncio/base_events.py", line 686, in run_until_complete
  |     return future.result()
  |            ^^^^^^^^^^^^^^^
  |   File "/home/runner/work/paper-qa/paper-qa/paperqa/agents/search.py", line 674, in get_directory_index
  |     async with anyio.create_task_group() as tg:
  |                ^^^^^^^^^^^^^^^^^^^^^^^^^
  |   File "/home/runner/work/paper-qa/paper-qa/.venv/lib/python3.12/site-packages/anyio/_backends/_asyncio.py", line 767, in __aexit__
  |     raise BaseExceptionGroup(
  | ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception)
  +-+---------------- 1 ----------------
    | Traceback (most recent call last):
    |   File "/home/runner/work/paper-qa/paper-qa/paperqa/agents/search.py", line 476, in process_file
    |     if not await search_index.filecheck(filename=file_location):
    |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/home/runner/work/paper-qa/paper-qa/paperqa/agents/search.py", line 263, in filecheck
    |     index_files = await self.index_files
    |                   ^^^^^^^^^^^^^^^^^^^^^^
    |   File "/home/runner/work/paper-qa/paper-qa/paperqa/agents/search.py", line 252, in index_files
    |     zlib.decompress(content)
    | zlib.error: Error -5 while decompressing data: incomplete or truncated stream
    +------------------------------------

I am not entirely sure why this zlib.error happened, but this change is a first step in figuring it out. If the zlib.error happens across 3 retries, then we know there's deeper more going on.

@jamesbraza jamesbraza added the bug Something isn't working label Jan 17, 2025
@jamesbraza jamesbraza requested a review from a team January 17, 2025 21:31
@jamesbraza jamesbraza self-assigned this Jan 17, 2025
@dosubot dosubot bot added the size:S This PR changes 10-29 lines, ignoring generated files. label Jan 17, 2025
@jamesbraza jamesbraza merged commit 8bc993c into main Jan 17, 2025
5 checks passed
@jamesbraza jamesbraza deleted the fixing-flaky-test-build branch January 17, 2025 23:57
@dosubot dosubot bot added the lgtm This PR has been approved by a maintainer label Jan 18, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working lgtm This PR has been approved by a maintainer size:S This PR changes 10-29 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants