Skip to content

Commit

Permalink
Fix kroxylicious#1415: Improve record validation docs
Browse files Browse the repository at this point in the history
  • Loading branch information
k-wall committed Aug 13, 2024
1 parent 98fee04 commit 3cb36d3
Show file tree
Hide file tree
Showing 3 changed files with 84 additions and 36 deletions.
2 changes: 1 addition & 1 deletion docs/assemblies/assembly-built-in-filters.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -10,5 +10,5 @@ Kroxylicious comes with a suite of built-in filters designed to enhance the func

include::assembly-record-encryption-filter.adoc[leveloffset=+1]
include::../modules/multi-tenancy/con-multi-tenancy.adoc[leveloffset=+1]
include::../modules/schema-validation/con-schema-validation.adoc[leveloffset=+1]
include::../modules/schema-validation/con-record-validation.adoc[leveloffset=+1]
include::../modules/oauthbearer/con-oauthbearer.adoc[leveloffset=+1]
83 changes: 83 additions & 0 deletions docs/modules/schema-validation/con-record-validation.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
// file included in the following:
//
// index.adoc

[id='con-record-validation-{context}']
= Record Validation

[role="_abstract"]
The Record Validation filter validates records sent by a producer. Only records that pass the validation are sent to
the broker. This filter can be used to prevent poison messages entering the Kafka system, that may otherwise lead to
consumer failure.

The filter currently supports two modes of operation:

1. Schema validation validates the content of the record against a schema stored in an https://www.apicur.io/registry/[Apicurio Registry].
2. Syntactically Correct JSON validation ensures the producer is producing messages that contain syntactically valid JSON.

Validation rules can validate the content of the Kafka record key or value.

If the validation fails, the product request is rejected and the producing application receives an error response. The broker
will not receive the rejected records.

== How to use the filter

There is just one step to use the filter.

1. <<Configuring virtual clusters>>
2. Configuring the filter within Kroxylicious.

=== Configuring the filter within Kroxylicious.

.Example Record Validation filter configuration

[source,yaml]
----
filters:
- type: ProduceValidationFilterFactory
config:
rules:
- topicNames: # <1>
- <topic name>
keyRule:
<rule definition> # <2>
valueRule:
<rule definition> # <3>
isForwardPartialRequests: false # <4>
----
<1> List of topic names to which the validation rules will be applied.
<2> Validation rules that are applied to the record's key.
<3> Validation rules that are applied to the record's value.
<4> If false, any record that fails the validation will cause
the entire batch to be rejected. If true, only records that fail validation
will be rejected. Defaults to false.

.Example Schema Validation Rule Definition

The Schema Validation rule validates that the key or value matches the identified schema.

[source,yaml]
----
schemaValidationConfig:
apicurioGlobalId: 1001 # <1>
apicurioRegistryUrl: http://registry.local:8080 # <2>
allowNulls: true # <3>
allowEmpty: true # <4>
----
<1> Apicurio registry global id identifying the schema that will be enforced
<2> Apicurio Registry endpoint
<3> if true, the validator allows null keys/values, the default is false.
<4> if true, the validator allow empty keys/values, the default is false.

.Example Syntactically Correct JSON Validation Rule Definition

[source,yaml]
----
syntacticallyCorrectJson:
validateObjectKeysUnique: true # <1>
allowNulls: true # <2>
allowEmpty: true # <3>
----
<1> if true, the validator will enforce that objects keys are unique, the default is false.
<2> if true, the validator allow null keys/values, the default is false.
<3> if true, the validator allow empty keys/values, the default is false.
35 changes: 0 additions & 35 deletions docs/modules/schema-validation/con-schema-validation.adoc

This file was deleted.

0 comments on commit 3cb36d3

Please sign in to comment.