You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently we have ObjSelector structure that can be used to iterate over objects in the registry database (selecting only objects in specific state).
Current behavior of ObjSelector is to start loop from the first object and then iterating further until the end of objects bucket. When it reaches the end, it immediately jumps to the beginning to start scanning again. This design allows us to pick an object for verification, attempt to verify it, and (in case if verification didn't succeed, e.g. because of timeout) to pick object again on the next pass of object selector.
The issue here is that when we are verifying the last few objects, ObjSelector iterates over them faster than VUs perform verification calls. As result, the same object can be picked up by several VUs concurrently. It does not impose any issues apart from redundant processing.
At the moment we addressed the issue in the way that we iterate ObjSelector only as many times as necessary - i.e. we never force it to swing back to the beginning. However, we could consider a better way to perform iteration over objects so that it would work both for verification process and for reading VUs.
Possible solutions are:
Keep the current iteration pattern in ObjSelector, but add some short-lived lock that prevents object from being picked up if it was recently fetched by some client of ObjSelector.
Change iteration pattern so that ObjSelector iterates only forward unless it is explicitly commanded to swing back to the beginning. It is simpler to implement than p.1 and covers both needs of the verification process (which doesn't need to swing back) and the needs of reading VUs (which can explicitly ask selector to swing back when it reaches the end of current pool of objects in registry).
The text was updated successfully, but these errors were encountered:
Currently we have
ObjSelector
structure that can be used to iterate over objects in the registry database (selecting only objects in specific state).Current behavior of
ObjSelector
is to start loop from the first object and then iterating further until the end of objects bucket. When it reaches the end, it immediately jumps to the beginning to start scanning again. This design allows us to pick an object for verification, attempt to verify it, and (in case if verification didn't succeed, e.g. because of timeout) to pick object again on the next pass of object selector.The issue here is that when we are verifying the last few objects,
ObjSelector
iterates over them faster than VUs perform verification calls. As result, the same object can be picked up by several VUs concurrently. It does not impose any issues apart from redundant processing.At the moment we addressed the issue in the way that we iterate
ObjSelector
only as many times as necessary - i.e. we never force it to swing back to the beginning. However, we could consider a better way to perform iteration over objects so that it would work both for verification process and for reading VUs.Possible solutions are:
ObjSelector
, but add some short-lived lock that prevents object from being picked up if it was recently fetched by some client ofObjSelector
.ObjSelector
iterates only forward unless it is explicitly commanded to swing back to the beginning. It is simpler to implement than p.1 and covers both needs of the verification process (which doesn't need to swing back) and the needs of reading VUs (which can explicitly ask selector to swing back when it reaches the end of current pool of objects in registry).The text was updated successfully, but these errors were encountered: