Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Segment Replication] Added source-side classes for orchestrating replication events #3470

Merged
merged 16 commits into from
Jun 3, 2022
Merged

[Segment Replication] Added source-side classes for orchestrating replication events #3470

merged 16 commits into from
Jun 3, 2022

Commits on Jun 1, 2022

  1. Added CopyState class and relevant downstream methods 

    Signed-off-by: Kartik Ganesh <[email protected]>
    @kartg
    kartg committed Jun 1, 2022
    Configuration menu
    Copy the full SHA
    912d997 View commit details
    Browse the repository at this point in the history

  2. Add PrimaryShardReplicationSource as an implementation of SegmentRepl… 

    …icationSource

Signed-off-by: Kartik Ganesh <[email protected]>
    @kartg
    kartg committed Jun 1, 2022
    Configuration menu
    Copy the full SHA
    f362d44 View commit details
    Browse the repository at this point in the history

  3. Change SegmentReplicationSourceFactory to create a PrimaryShardReplic… 

    …ationSource instance as the implementation

Signed-off-by: Kartik Ganesh <[email protected]>
    @kartg
    kartg committed Jun 1, 2022
    Configuration menu
    Copy the full SHA
    36b4fb1 View commit details
    Browse the repository at this point in the history

  4. Added the SegmentReplicationSourceService service class 

    CopyState has closeInternal now implemented and includes a new getter that is used by the service class. The Action definitions have been moved from the ReplicationSource class to the service class.

Signed-off-by: Kartik Ganesh <[email protected]>
    @kartg
    kartg committed Jun 1, 2022
    Configuration menu
    Copy the full SHA
    f72aa14 View commit details
    Browse the repository at this point in the history

  5. Added SegmentReplicationTransportRequest and its two concrete subclasses 

    PrimaryShardReplicationSource has been updated to wire in these classes.

Signed-off-by: Kartik Ganesh <[email protected]>
    @kartg
    kartg committed Jun 1, 2022
    Configuration menu
    Copy the full SHA
    2e3f243 View commit details
    Browse the repository at this point in the history

  6. Added unit tests for PrimaryShardReplicationSource 

    Signed-off-by: Kartik Ganesh <[email protected]>
    @kartg
    kartg committed Jun 1, 2022
    Configuration menu
    Copy the full SHA
    b874d01 View commit details
    Browse the repository at this point in the history

  7. Added request handlers to SegmentReplicationSourceService 

    Also added an IndicesService member since this is needed to resolve the IndexShard from incoming shardId.

Signed-off-by: Kartik Ganesh <[email protected]>
    @kartg
    kartg committed Jun 1, 2022
    Configuration menu
    Copy the full SHA
    83a47a5 View commit details
    Browse the repository at this point in the history

  8. Refactoring argument order to avoid confusion between source and targ… 

    …et DiscoveryNodes

Also updated variable naming to try and be consistent.

Signed-off-by: Kartik Ganesh <[email protected]>
    @kartg
    kartg committed Jun 1, 2022
    Configuration menu
    Copy the full SHA
    d5b85fe View commit details
    Browse the repository at this point in the history

  9. Added a subset of unit tests 

    Classes updated:
* InternalEngineTests
* IndexShardTests
* StoreTests

Also added EngineConfigTests to test the read-only engine use-case with seg-rep

Signed-off-by: Kartik Ganesh <[email protected]>
    @kartg
    kartg committed Jun 1, 2022
    Configuration menu
    Copy the full SHA
    005e403 View commit details
    Browse the repository at this point in the history

  10. Added and updated Javadocs 

    Signed-off-by: Kartik Ganesh <[email protected]>
    @kartg
    kartg committed Jun 1, 2022
    Configuration menu
    Copy the full SHA
    2227f45 View commit details
    Browse the repository at this point in the history

  11. Added a unit test for CopyState 

    Signed-off-by: Kartik Ganesh <[email protected]>
    @kartg
    kartg committed Jun 1, 2022
    Configuration menu
    Copy the full SHA
    6ecb560 View commit details
    Browse the repository at this point in the history

  12. Incorporating PR comments 

    Signed-off-by: Kartik Ganesh <[email protected]>
    @kartg
    kartg committed Jun 1, 2022
    Configuration menu
    Copy the full SHA
    1a2c70b View commit details
    Browse the repository at this point in the history

  13. Added tests for SegmentReplicationSourceService 

    This includes refactoring CopyStateTests for code reuse. Also fixed CopyStateTests since these were failing.

Signed-off-by: Kartik Ganesh <[email protected]>
    @kartg
    kartg committed Jun 1, 2022
    Configuration menu
    Copy the full SHA
    caca533 View commit details
    Browse the repository at this point in the history

Commits on Jun 2, 2022

  1. Removing default implementation of getProcessedLocalCheckpoint 

    This now requires implementations in child classes. NRTReplicationEngine already has an implementation, so an @OverRide annotation has been added. For ReadOnlyEngine, where no processing occurs, the processed local checkpoint is expected to be equal to the persisted local checkpoint. Unit test for ReadOnlyEngine have been updated.

Signed-off-by: Kartik Ganesh <[email protected]>
    @kartg
    kartg committed Jun 2, 2022
    Configuration menu
    Copy the full SHA
    91372c6 View commit details
    Browse the repository at this point in the history

  2. Made the javadoc for getSegmentInfosSnapshot() better 

    Signed-off-by: Kartik Ganesh <[email protected]>
    @kartg
    kartg committed Jun 2, 2022
    Configuration menu
    Copy the full SHA
    7075793 View commit details
    Browse the repository at this point in the history

Commits on Jun 3, 2022

  1. Incorporating PR feedback on javadocs 

    Signed-off-by: Kartik Ganesh <[email protected]>
    @kartg
    kartg committed Jun 3, 2022
    Configuration menu
    Copy the full SHA
    71e66de View commit details
    Browse the repository at this point in the history