Specify the state of an indexing agent and keep logs

[lecoqlibre]

Storing the state of an indexing agent has several benefits:

• we can track the indexing progress (globally and of a particular job)
• when the agent crashes, it can restart from the point it was before the crash

How to store the state of the Indexing Agent:

• Where do we want to store the state?
  • An indexing agent would have its own POD, its state could be stored here.
• What information do we want to store?
  • the status of the job: active, inactive or error
  • the reason of the status
  • the resources already indexed as well as the resources to be indexed

Also, keeping the logs could be useful to know which resources have already been indexed, when and by who.

[FabienGandon]

In IndeGx we keep the traces of the entire indexing process in Prov-O with some extensions https://github.com/Wimmics/dekalog