Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sync between 2 databases. #36

Open
Siegurd01 opened this issue Aug 4, 2021 · 0 comments
Open

Sync between 2 databases. #36

Siegurd01 opened this issue Aug 4, 2021 · 0 comments

Comments

@Siegurd01
Copy link

It would be nice to add a sync feature where data can be synchronized between 2 databases on 2 hosts. The sync algorithm is kinda tricky but I made it on different PL so I can describe it's algorithm:

  1. Set up and connect to 2 databases (master A and slave B).
  2. Read only one metric from the 2 databases (reading one metric is quicker then reading whole database) on specified period to find out time arrays ("time holes" between A and B databases).
  3. Here comes the tricky part. Need to find where the time array in A is found in time array from B and if not - sync only the missing data from A to B using the resulting time array. In Matlab ismember does the trick.
  4. Transform resulting time array in to a short start-end pieces to read the data from A and write it to B using small amount of RAM.
  5. profit :)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant