Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix: state not set in file-provided pipelines #909

Merged
merged 6 commits into from
Feb 24, 2023

Conversation

hariso
Copy link
Contributor

@hariso hariso commented Feb 24, 2023

Description

This PR fixes the following bug.

Bug description
When Conduit is restarted with a pipeline, that was provided from a configuration file, the state is not correctly set after restart and lost.

Steps to reproduce

  1. Start Conduit with a pipeline configuration file
  2. Shut down Conduit gracefully
  3. Start Conduit again

Expected behavior
After the restart, Conduit should start from the previous source position. Only new records are read.

Actual behavior
After the restart, Conduit re-reads all the records from source. Following can be found in logs:

could not apply old state, the connector will start from the beginning

Quick checks:

  • I have followed the Code Guidelines.
  • There is no other pull request for the same update/change.
  • I have written unit tests.
  • I have made sure that the PR is of reasonable size and can be easily reviewed.

@hariso hariso requested a review from a team as a code owner February 24, 2023 09:21
@hariso hariso added this to the 0.6.0 milestone Feb 24, 2023
@hariso hariso added the bug Something isn't working label Feb 24, 2023
Copy link
Member

@lovromazgon lovromazgon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch! 👍

pkg/connector/service_test.go Outdated Show resolved Hide resolved
@hariso hariso enabled auto-merge (squash) February 24, 2023 10:54
@hariso hariso merged commit a3c963a into main Feb 24, 2023
@hariso hariso deleted the haris/pipeline-config-file-states branch February 24, 2023 10:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

2 participants