Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use rohmu transfer pool to fix io error on closed fds on restore #182

Conversation

giacomo-alzetta-aiven
Copy link
Contributor

About this change: What it does, why it matters

Currently myhoard can fail with: ValueError: I/O operation on closed file during restore (. The problem is likely due to non thread-safety of the rohmu transfers.

Using rohmu's transfer pool we should eliminate any possible case in which multiple threads operated on the same transfer object, or cases in which a "stale" transfer object was used by a single thread.

@giacomo-alzetta-aiven
Copy link
Contributor Author

Seem like CI is broken? It failed to setup the linting...

@giacomo-alzetta-aiven giacomo-alzetta-aiven force-pushed the giacomo-alzetta-aiven-use-transfer-pool-to-fix-io-error-on-closed-fds branch from 9d7b88d to 61d74eb Compare March 18, 2024 13:18
@codecov-commenter
Copy link

codecov-commenter commented Mar 18, 2024

Codecov Report

Attention: Patch coverage is 83.72093% with 7 lines in your changes are missing coverage. Please review.

Project coverage is 78.54%. Comparing base (c53a64f) to head (0ed5c6a).

Files Patch % Lines
myhoard/restore_coordinator.py 83.72% 4 Missing and 3 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #182      +/-   ##
==========================================
- Coverage   79.22%   78.54%   -0.69%     
==========================================
  Files          16       16              
  Lines        4506     4503       -3     
  Branches     1018     1021       +3     
==========================================
- Hits         3570     3537      -33     
- Misses        699      729      +30     
  Partials      237      237              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@giacomo-alzetta-aiven giacomo-alzetta-aiven force-pushed the giacomo-alzetta-aiven-use-transfer-pool-to-fix-io-error-on-closed-fds branch from 61d74eb to 0ed5c6a Compare March 18, 2024 13:34
@alexole alexole merged commit 9e23cc6 into master Mar 27, 2024
6 checks passed
@alexole alexole deleted the giacomo-alzetta-aiven-use-transfer-pool-to-fix-io-error-on-closed-fds branch March 27, 2024 14:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants