Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reported error in evaluator assignments #9

Closed
jrwashburn opened this issue Sep 26, 2024 · 1 comment
Closed

Reported error in evaluator assignments #9

jrwashburn opened this issue Sep 26, 2024 · 1 comment
Assignees

Comments

@jrwashburn
Copy link
Contributor

Sponsor reports one evaluator did not receive any submissions to review in the last cycle.

Review evaluator assignments and code to confirm whether there is an issue.

@jrwashburn jrwashburn self-assigned this Sep 26, 2024
@jrwashburn
Copy link
Contributor Author

jrwashburn commented Sep 26, 2024

I reviewed the selections.
There were 70 submissions but 83 evaluators.
Confirmed that this is possible based on the way the assignments work currently, and is not a bug.

We create a list of potential evaluators, which is just a triplicate of the registry. Before assigning evaluators, we filter out any ambassadors already selected for this submission, or that have been selected 3 times in total.

Then we simply get a random index into the remaining list and pick that ambassador.

So it will happen from time to time that an ambassador is not assigned any to evaluate.

It may be a good practice to add an email letting them know that they were not selected for any evaluations this period so they are not wondering, and will have that as evidence if there is a problem later. I'll suggest this be added with #8

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant