forked from kroxylicious/kroxylicious
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Fix kroxylicious#1415: Improve record validation docs
- Loading branch information
Showing
3 changed files
with
84 additions
and
36 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,83 @@ | ||
// file included in the following: | ||
// | ||
// index.adoc | ||
|
||
[id='con-record-validation-{context}'] | ||
= Record Validation | ||
|
||
[role="_abstract"] | ||
The Record Validation filter validates records sent by a producer. Only records that pass the validation are sent to | ||
the broker. This filter can be used to prevent poison messages entering the Kafka system, that may otherwise lead to | ||
consumer failure. | ||
|
||
The filter currently supports two modes of operation: | ||
|
||
1. Schema validation validates the content of the record against a schema stored in an https://www.apicur.io/registry/[Apicurio Registry]. | ||
2. Syntactically Correct JSON validation ensures the producer is producing messages that contain syntactically valid JSON. | ||
|
||
Validation rules can validate the content of the Kafka record key or value. | ||
|
||
If the validation fails, the product request is rejected and the producing application receives an error response. The broker | ||
will not receive the rejected records. | ||
|
||
== How to use the filter | ||
|
||
There is just one step to use the filter. | ||
|
||
1. <<Configuring virtual clusters>> | ||
2. Configuring the filter within Kroxylicious. | ||
|
||
=== Configuring the filter within Kroxylicious. | ||
|
||
.Example Record Validation filter configuration | ||
|
||
[source,yaml] | ||
---- | ||
filters: | ||
- type: ProduceValidationFilterFactory | ||
config: | ||
rules: | ||
- topicNames: # <1> | ||
- <topic name> | ||
keyRule: | ||
<rule definition> # <2> | ||
valueRule: | ||
<rule definition> # <3> | ||
isForwardPartialRequests: false # <4> | ||
---- | ||
<1> List of topic names to which the validation rules will be applied. | ||
<2> Validation rules that are applied to the record's key. | ||
<3> Validation rules that are applied to the record's value. | ||
<4> If false, any record that fails the validation will cause | ||
the entire batch to be rejected. If true, only records that fail validation | ||
will be rejected. Defaults to false. | ||
|
||
.Example Schema Validation Rule Definition | ||
|
||
The Schema Validation rule validates that the key or value matches the identified schema. | ||
|
||
[source,yaml] | ||
---- | ||
schemaValidationConfig: | ||
apicurioGlobalId: 1001 # <1> | ||
apicurioRegistryUrl: http://registry.local:8080 # <2> | ||
allowNulls: true # <3> | ||
allowEmpty: true # <4> | ||
---- | ||
<1> Apicurio registry global id identifying the schema that will be enforced | ||
<2> Apicurio Registry endpoint | ||
<3> if true, the validator allows null keys/values, the default is false. | ||
<4> if true, the validator allow empty keys/values, the default is false. | ||
|
||
.Example Syntactically Correct JSON Validation Rule Definition | ||
|
||
[source,yaml] | ||
---- | ||
syntacticallyCorrectJson: | ||
validateObjectKeysUnique: true # <1> | ||
allowNulls: true # <2> | ||
allowEmpty: true # <3> | ||
---- | ||
<1> if true, the validator will enforce that objects keys are unique, the default is false. | ||
<2> if true, the validator allow null keys/values, the default is false. | ||
<3> if true, the validator allow empty keys/values, the default is false. |
This file was deleted.
Oops, something went wrong.