object/put: fix concurrent PUT data corruption #3027
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
carpawell
requested review from
roman-khimov and
cthulhu-rider
as code owners
carpawell
force-pushed
the
fix/data-corruption-on-hight-load
branch
from
943eaaf to
a63db8a
Compare
|
@roman-khimov, this is almost the same as in the |
|
Also we can try node one more time ( |
|
Also, be more aggressive with "close XXX", we have a number of issues here. |
carpawell
force-pushed
the
fix/data-corruption-on-hight-load
branch
from
a63db8a to
004fcf0
Compare
|
@roman-khimov, added issues for my taste. |
carpawell
force-pushed
the
fix/data-corruption-on-hight-load
branch
from
004fcf0 to
ea3e03b
Compare
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #3027 +/- ##
==========================================
- Coverage 22.85% 22.63% -0.23%
==========================================
Files 791 791
Lines 58734 58493 -241
==========================================
- Hits 13425 13238 -187
+ Misses 44412 44358 -54
Partials 897 897 ☔ View full report in Codecov by Sentry. |
carpawell
force-pushed
the
fix/data-corruption-on-hight-load
branch
from
e82555a to
06efc8e
Compare
roman-khimov
approved these changes
Nov 25, 2024
|
Test needs to be fixed. |
If ants pool is busy and cannot take task, early `return` without `wg.Wait()` leads to `iterateNodesForObject`'s `return` and all the buffers for binary replication from now may be reused while are still in use by the other routines inside the pool. Wait for WG before any `return` is called. Closes #2978, closes #2988, closes #2975, closes #2971. Signed-off-by: Pavel Karpy <[email protected]>
If an object cannot be PUT due to local overload (i-th routine for (i-1)-length worker pool), log the error and continue over other nodes, and even other placement vectors. `errNotEnoughNodes` will be also returned as the natural replication number handling in the outer `for`. Signed-off-by: Pavel Karpy <[email protected]>
Signed-off-by: Pavel Karpy <[email protected]>
carpawell
force-pushed
the
fix/data-corruption-on-hight-load
branch
from
06efc8e to
872baaf
Compare
|
Also, added one more test. |
roman-khimov
approved these changes
Nov 25, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
If ants pool is busy and cannot take task, early
returnwithoutwg.Wait()leads to
iterateNodesForObject'sreturnand all the buffers for binaryreplication from now may be reused while are still in use by the other routines
inside the pool. Wait for WG before any
returnis called. Closes #2978, #2988,#2975, #2971.