Segment Replication: Refactor SegmentReplicationSource interface and SegmentReplicationTarget implementation

[ankitkala]

Current interfaces works well for Segment Replication using node to node data copy. With the remtoe store integration, the interface isn't completely aligning with remote store

• SegmentReplicationSource: getCheckpointMetadata and getSegmentFiles can be merged into one as we're combining both the remote store into a single method. We can technically split the logic for remote store usecase as well but it doesn't bring any benefit for remote store.
• In conjunction with the point above, SegmentReplicationTarget can merge the getSegmentCheckpointInfo, getFiles & finalizeReplication into a single method. It'll also help in removing the dependency on StoreFileMetadata as its not passed on for remote store case.

[nknize]

We can technically split the logic for remote store usecase as well but it doesn't bring any benefit for remote store.

For the remote backed segment replication use case it looks like IndexShard#syncSegmentsFromRemoteSegmentStore is roughly the same sequence of events as SegmentReplicationTarget#startReplication. We should be able to reuse the startReplication method instead of writing a largely duplicate method call. This would reduce the tight integration of Remote Store specific logic in IndexShard which has a HUGE benefit to the open source implementation extension point for cloud native or serverless applications.

/cc @mch2

[ankitkala]

We are using the startReplication flow in the current implementation of SegRep using remote.
I agree, we can explore splitting the syncSegmentsFromRemoteStore as per SegmentReplicationSource implementation. Just a call out though, segrep isn't the only flow invoking this methods(other flows like peer recovery, recover from remote store)

The main problem with SegmentReplicationSource is that it relies on StoreFileMetadata for each segment. This is not the case with Remote store implementation however as it maintains its own metadata.
We are getting around this problem by recreating the StoreFileMetadata objects on replica using remote store metadata(here)

[mch2]

@ankitkala Can we not convert to/from these objects? I don't see this as a limitation. @dreamer-89

[ankitkala]

Yep. Its not blocking us in any way. Just need to work around it.

[dreamer-89]

Looking into it.