add 'select oldest unaudited' selection strategy

[perama-v]

Issue addressed

• Closes Add additional audit strategies #62

Proposed changes

Adds the final unimplemented strategy: SelectionStrategy::SelectOldestUnaudited

Description

The strategy finds the oldest content that has not yet been audited and submits it for auditing. As audits take time to complete, the strategy remembers the timestamp of the last loop of submissions to prevent double-submitting the content for another audit (race condition).

Additional info

• unit test for strategy

[pipermerriam]

Curious why this line exists. Am I correct that this is how we get sea-orm to select for rows that don't have an audit?

[perama-v]

Yes this is my understanding. In the test case for this strategy I have added an audited for item 1 of 45, and the test correctly does not select that item for audit.

[pipermerriam]

If I'm reading this correctly, we select the N oldest pieces of content that have no audit.

When I first read "oldest missing" my brain inferred it to mean, the oldest content that was found to be missing from the network (failed audit).

Assuming I'm correct in my interpretation of what this strategy is doing, I think it should potentially be renamed to something like "select oldest unaudited".

My next thought here is that the current implementation introduces what I think can be categorized as a race condition.

• The strategy selects things that need to be audited and puts them in the queue
• The queue is consumed, albeit slowly.
• The strategy selects the same things again because they are still in the queue waiting to be audited.
• The queue gets duplicate items added.

I think one way around this might be to keep note of the FirstAvailableAt timestamp of the last item selected, and then perform subsequent queries with a filter like this filter(content::Column::FirstAvailableAt.gt(previous_latest_first_available_at)).

[perama-v]

... I think it should potentially be renamed to something like "select oldest unaudited".

Updated

I think one way around this might be to keep note of the FirstAvailableAt timestamp of the last item selected, and then perform subsequent queries with a filter like this filter(content::Column::FirstAvailableAt.gt(previous_latest_first_available_at)).

Good idea - implemented