Harvest: Set export can temporarily be adversely affected by full (clean) reindex.

[kcondon]

Set scope is defined by the results of a solr query in the set definition. When the scope changes, such as additions made or removals, the set is updated and subsequent harvests by clients may get additions or delete notifications.

This can accidentally happen if a clean index is performed and the nightly set exporter runs while the index is relatively blank, not yet finished. The net effect would be to mark most of the sets as deleted and any client harvesting before the next nightly set export or a manual reexport, will update their local harvest content by deleting those we have not finished indexing at the time of export. This would be temporary in principle if the client is schedule to run again at a later date -it would get the updated set once our index has finished and export was rerun.

[mreekie]

Review with Leonid

• Believe this is still a real problem.
• Get this estimated and prioritized

[cmbz]

2023/12/19: Prioritized during meeting on 2023/12/18. Added to Needs Sizing.

[landreev]

Sounds like we need some kind of a "full reindex in progress" lock? - Should help with things other than OAI sets too, I would think - would prevent starting multiple reindex runs, for example?