Unique constraint violates "exp_history_ledgers_id_idx" when reingesting state

[bartekn]

What version are you using?

Horizon 0.24.1 RC.

What did you do?

Horizon 0.24.1 RC (fb5f489) fixes a bug in expingest so expingest.CurrentVersion was incremented. This made Horizon to reingest state.

What did you expect to see?

No errors.

What did you see instead?

Processor DatabaseProcessor (All) errored: Could not insert ledger: exec failed: pq: duplicate key value violates unique constraint \"exp_history_ledgers_id_idx\"

What happened?

Horizon always reingests state using the latest checkpoint ledger and in most cases the sequence of this ledger will be smaller than the latest ingested ledger. Because of this Horizon will attempt to insert old ledgers into exp_history_ledger again. It seems that the quick fix that should be included in 0.24.1 is to remove all ledgers with the sequence higher than the checkpoint ledger. The longterm fix is #1969.

[tamirms]

fixed in #2055