Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Implement stateful cursor #86

Closed
Tracked by #79
scarmuega opened this issue Jan 24, 2022 · 2 comments · Fixed by #116
Closed
Tracked by #79

feat: Implement stateful cursor #86

scarmuega opened this issue Jan 24, 2022 · 2 comments · Fixed by #116
Assignees
Labels
enhancement New feature or request
Milestone

Comments

@scarmuega
Copy link
Member

Problem Statement

When running in daemon mode, a restart of the process will make Oura start crawling from the initially configured point. Sinks should implement idempotency when processing an event, but depending on the use-case, restarting might generate undesirable lags and waste of resources.

Feature Scope

A "cursor" framework will be implemented such that it is agnostic of any concrete source, filter or sink. This framework will resemble a very simple key/value store. A sink stage would use the cursor framework to persist the point of the chain at certain "checkpoint" events. A source sink would use the cursor framework to check the last persisted checkpoint before starting to pull data from the node.

@scarmuega scarmuega self-assigned this Jan 24, 2022
@scarmuega scarmuega added the enhancement New feature or request label Jan 24, 2022
@scarmuega scarmuega added this to the v1.2 milestone Jan 24, 2022
@scarmuega scarmuega removed their assignment Jan 24, 2022
@gitmachtl
Copy link

is it possible to enable/disable that?

@mark-stopka mark-stopka changed the title Implement stateful cursor feat: Implement stateful cursor Jan 25, 2022
@scarmuega
Copy link
Member Author

@gitmachtl it will be an opt-in feature for the daemon mode. You will have to enable it via config. It is meant for sinks such as Elasticsearch and Rotating Logs. The watch and dump command will keep working as they currently do.

Repository owner moved this from Todo to Done in Oura Development Feb 2, 2022
@scarmuega scarmuega modified the milestones: v1.2, v1.1 Feb 2, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
No open projects
Status: Done
Development

Successfully merging a pull request may close this issue.

2 participants