-
Notifications
You must be signed in to change notification settings - Fork 41
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
common: update the validation of the enriched event with static field lengths #608
Comments
pondzix
added a commit
that referenced
this issue
May 26, 2022
pondzix
added a commit
that referenced
this issue
May 26, 2022
pondzix
added a commit
that referenced
this issue
May 26, 2022
pondzix
added a commit
that referenced
this issue
May 27, 2022
pondzix
added a commit
that referenced
this issue
May 27, 2022
benjben
pushed a commit
that referenced
this issue
May 31, 2022
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
In 93078e2 we've introduced the validation of the enriched event according to atomic schema before emitting.
The implementation is to convert the enriched event to a JSON and to use Iglu client to validate it against the schema. Being based on JSON manipulation, we've observed a significant bump in the use of CPU by the enriching step of the app; ~ x5 in case no enrichments are activated (other steps like reading from/writing to message queue being not impacted).
For this reason, for now we'd like to limit the validation to the validation of each field length (which was the original problem we were trying to solve), like this is done in analytics-sdk.
When this issue will be implemented, we'll see to add the full validation with Iglu client again.
The text was updated successfully, but these errors were encountered: