Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] S3 source CSV codec drops data without metrics #2512

Closed
dlvenable opened this issue Apr 17, 2023 · 0 comments · Fixed by #2513
Closed

[BUG] S3 source CSV codec drops data without metrics #2512

dlvenable opened this issue Apr 17, 2023 · 0 comments · Fixed by #2513
Labels
bug Something isn't working
Milestone
v2.2

Comments

@dlvenable
Copy link
Member

Describe the bug

The CSV codec will skip some rows of data without any metrics, only logging.

Expected behavior

I'd like a more robust error handling situation. In the meantime though, failing fast is better than difficult to detect data drops.
@dlvenable dlvenable added bug Something isn't working untriaged labels Apr 17, 2023
@dlvenable dlvenable added this to the v2.2 milestone Apr 17, 2023
dlvenable added a commit to dlvenable/data-prepper that referenced this issue Apr 17, 2023
@dlvenable
Change the behavior of the CSV codec in the S3 source to fail when it…
aeed6a0 
… is unable to parse CSV rows. Resolves opensearch-project#2512.

Signed-off-by: David Venable <[email protected]>
@dlvenable dlvenable mentioned this issue Apr 17, 2023
Merged
4 tasks
dlvenable added a commit that referenced this issue Apr 18, 2023
@dlvenable
Change the behavior of the CSV codec in the S3 source to fail when it…
bc75494 
… is unable to parse CSV rows. Resolves #2512. (#2513)

Signed-off-by: David Venable <[email protected]>
@github-project-automation github-project-automation bot moved this from Untriaged to Done in Data Prepper Tracking Board Apr 18, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
Data Prepper Tracking Board
Archived in project
Milestone
v2.2
Development

Successfully merging a pull request may close this issue.

Change the behavior of the CSV codec in the S3 source to fail fast dlvenable/data-prepper
1 participant
@dlvenable