Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

KAFKA-16448 [WIP]: Add Kafka Streams exception handler for exceptions occuring during processing #15973

Closed
wants to merge 31 commits into from
Closed

KAFKA-16448 [WIP]: Add Kafka Streams exception handler for exceptions occuring during processing #15973

wants to merge 31 commits into from

Conversation

loicgreffier
Copy link
Contributor

This PR aims to bring a ProcessingExceptionHandler to Kafka Streams in order to deal with exceptions that occur during processing.

Jira: https://issues.apache.org/jira/browse/KAFKA-16448.

Contributors

@Dabz
@sebastienviale
@loicgreffier

Committer Checklist (excluded from commit message)

  • Verify design and implementation
  • Verify test coverage and CI build status
  • Verify documentation (including upgrade notes)

sebastienviale and others added 30 commits May 15, 2024 10:15
@sebastienviale
KAFKA-16448: add new processor for error handling
d0f6cc7
@sebastienviale
KAFKA-16448: enum to get the origin of a SerializationException
bac5ed1
@loicgreffier
KAFKA-16448: Add ProcessingExceptionHandler and update ProductionExce…
c97cf4c 
…ptionHandler
@loicgreffier
KAFKA-16448: Add ProcessingExceptionHandler impls
962e2e2
@loicgreffier
KAFKA-16448: Add error handler context impl
00e2bf3
@loicgreffier
KAFKA-16448: Catch processing exceptions at Stream Task level
d99b48f
@loicgreffier
KAFKA-16448: Move processing exception catch to processor context imp…
5ffca71 
…l to catch errors on child processors
@sebastienviale
KAFKA-16448: add error handling for deserialisation
a5ce52b
@sebastienviale
KAFKA-16448: add error handling for deserialisation
3a71905
@sebastienviale
KAFKA-16448: fix checkstyle
5f621c2
@loicgreffier
KAFKA-16448: Provide rawSourceKey and rawSourceValue in the ErrorHand…
2b7a528 
…lerContext
@sebastienviale
KAFKA-16448: add boolean to catch only one time an exception in the p…
76cc4be 
…rocessor
@loicgreffier
KAFKA-16448: Fix deprecation in unit tests
6906dd7
@loicgreffier
KAFKA-16448: Move raw record out of StampedRecord
259f01a
@loicgreffier
KAFKA-16448: Add tests for ProcessorContextImpl
521a51d
@sebastienviale
KAFKA-16448: split deserializer key and value to get origin
ea13eea
@sebastienviale
Merge branch 'KAFKA-16448' of https://github.com/loicgreffier/kafka i…
5e3eaad 
…nto KAFKA-16448
@sebastienviale
KAFKA-16448: fix TU after classCastException
db45b6b
@sebastienviale
KAFKA-16448: remove deprecated
2171fbe
@loicgreffier
KAFKA-16448: Fix indent
028ba33
@loicgreffier
Merge branch 'trunk' into KAFKA-16448
7462001
@loicgreffier
KAFKA-16448: Update from trunk
655f274
@loicgreffier
KAFKA-16448: Fix bad indent
81b80f4
@loicgreffier
KAFKA-16448: Fix bad indent
99ae674
@loicgreffier
KAFKA-16448: Fix bad indent
a22f983
@loicgreffier
KAFKA-16448: Handle processing exception on punctuate
50fa22d
@sebastienviale
KAFKA-16448
8f95697 
fix rawRecord and add unit test
@loicgreffier
KAFKA-16448: Simplify unit tests and handle metrics
9c84ceb
@loicgreffier
KAFKA-16448: Add metrics in tests
d3e2c58
@loicgreffier
KAFKA-16448: Move topology test driver to integration tests
c388a06
@cadonna cadonna self-requested a review May 27, 2024 11:13
@cadonna
Copy link
Contributor

cadonna commented May 27, 2024

@loicgreffier Thanks for the PR!
A 1700 lines PR is quite hard to review.
Could you please split this PR into multiple smaller ones?
300-500 lines is a good upper bound for the size.

@loicgreffier
Copy link
Contributor Author

@cadonna Understood

@loicgreffier
Copy link
Contributor Author

This PR is currently being divided into multiples PRs:

@loicgreffier loicgreffier mentioned this pull request May 28, 2024
Merged
3 tasks
@loicgreffier
Copy link
Contributor Author

Closing this outdated PR

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@cadonna cadonna Awaiting requested review from cadonna

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@loicgreffier @cadonna @sebastienviale