Sync retention leases on expiration #37902
Conversation
This commit introduces a sync of retention leases when a retention lease expires. As expiration of retention leases is lazy, their expiration is managed only when getting the current retention leases from the replication tracker. At this point, we callback to our full retention lease sync to sync and flush these on all shard copies. With this change, replicas do not locally manage expiration of retention leases; instead, that is done only on the primary.
jasontedor
added
>enhancement
v7.0.0
:Distributed Indexing/Distributed
|
Pinging @elastic/es-distributed |
|
If a replica executes retention-leases sync requests out of order, it may have a different set of retention-leases than the primary. Suppose the primary has no retention lease, then we add two retention-leases L1 and L2. We will execute two sync requests with {L1} and {L1, L2}. If the second request is executed before the first request on a replica, the replica will have one lease {L1} while the primary has two leases: {L1 and L2}. We may need to add primary-term and seqId to the retention-leases sync requests. We can address this either before or after this PR. |
|
@dnhatn Yes, we will need to add a version to it. Primary term/sequence number is insufficient because there’s no correlation between when a retention lease is taken and whether or not the sequence number has advanced. We will instead need our own version number. I want to do this in a follow up after the background sync is in too since it will touch all the syncing code. |
|
@jasontedor I meant a version number like you said :). I think we need to add the primary-term to reject the sync requests that from the demoted primary. |
I don't think this is needed since these are replication actions, so rejection based on the primary term is already handled for us. |
Ah, I found it. Thanks for the explanation. Let's add the version to the sync request. |
This commit introduces a sync of retention leases when a retention lease expires. As expiration of retention leases is lazy, their expiration is managed only when getting the current retention leases from the replication tracker. At this point, we callback to our full retention lease sync to sync and flush these on all shard copies. With this change, replicas do not locally manage expiration of retention leases; instead, that is done only on the primary.
|
I opened #37951. |
This commit introduces a sync of retention leases when a retention lease expires. As expiration of retention leases is lazy, their expiration is managed only when getting the current retention leases from the replication tracker. At this point, we callback to our full retention lease sync to sync and flush these on all shard copies. With this change, replicas do not locally manage expiration of retention leases; instead, that is done only on the primary.
Relates #37165