Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement more flexible object selector #21

Closed
vdomnich-yadro opened this issue Sep 22, 2022 · 0 comments · Fixed by #22
Closed

Implement more flexible object selector #21

vdomnich-yadro opened this issue Sep 22, 2022 · 0 comments · Fixed by #22
Assignees
Labels
U3 Regular

Comments

@vdomnich-yadro
Copy link
Contributor

Currently we have ObjSelector structure that can be used to iterate over objects in the registry database (selecting only objects in specific state).

Current behavior of ObjSelector is to start loop from the first object and then iterating further until the end of objects bucket. When it reaches the end, it immediately jumps to the beginning to start scanning again. This design allows us to pick an object for verification, attempt to verify it, and (in case if verification didn't succeed, e.g. because of timeout) to pick object again on the next pass of object selector.

The issue here is that when we are verifying the last few objects, ObjSelector iterates over them faster than VUs perform verification calls. As result, the same object can be picked up by several VUs concurrently. It does not impose any issues apart from redundant processing.

At the moment we addressed the issue in the way that we iterate ObjSelector only as many times as necessary - i.e. we never force it to swing back to the beginning. However, we could consider a better way to perform iteration over objects so that it would work both for verification process and for reading VUs.

Possible solutions are:

  1. Keep the current iteration pattern in ObjSelector, but add some short-lived lock that prevents object from being picked up if it was recently fetched by some client of ObjSelector.
  2. Change iteration pattern so that ObjSelector iterates only forward unless it is explicitly commanded to swing back to the beginning. It is simpler to implement than p.1 and covers both needs of the verification process (which doesn't need to swing back) and the needs of reading VUs (which can explicitly ask selector to swing back when it reaches the end of current pool of objects in registry).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
U3 Regular
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants