Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add topicConfiguration property to channel object #171

Merged
merged 3 commits into from
Nov 25, 2022
Merged

feat: add topicConfiguration property to channel object #171

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
e51cb09
feat: add topicConfiguration property to channel object
thake Nov 10, 2022
9d6e17d
referring to kafka documentation for constraints.
thake Nov 11, 2022
874de25
Merge branch 'master' into master
lbroudoux Nov 24, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
57 changes: 45 additions & 12 deletions kafka/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ This document defines how to describe Kafka-specific information on AsyncAPI.

## Version

Current version is `0.3.0`.
Current version is `0.4.0`.


<a name="server"></a>
Expand All @@ -32,7 +32,7 @@ servers:
kafka:
schemaRegistryUrl: 'https://my-schema-registry.com'
schemaRegistryVendor: 'confluent'
bindingVersion: '0.3.0'
bindingVersion: '0.4.0'
```


Expand All @@ -44,12 +44,14 @@ This object contains information about the channel representation in Kafka (eg.

##### Fixed Fields

Field Name | Type | Description | Applicability [default] | Constraints
---|:---:|:---:|:---:|---
<a name="channelBindingObjectTopic"></a>`topic` | string | Kafka topic name if different from channel name. | OPTIONAL | -
<a name="channelBindingObjectPartitions"></a>`partitions` | integer | Number of partitions configured on this topic (useful to know how many parallel consumers you may run). | OPTIONAL | Must be positive
<a name="channelBindingObjectReplicas"></a>`replicas` | integer | Number of replicas configured on this topic. | OPTIONAL | MUST be positive
<a name="channelBindingObjectBindingVersion"></a>`bindingVersion` | string | The version of this binding. If omitted, "latest" MUST be assumed. | OPTIONAL [`latest`] | -
Field Name | Type | Description | Applicability [default] | Constraints
---|:------------------------------------------------:|:-------------------------------------------------------------------------------------------------------:|:-----------------------:|---
<a name="channelBindingObjectTopic"></a>`topic` | string | Kafka topic name if different from channel name. | OPTIONAL | -
<a name="channelBindingObjectPartitions"></a>`partitions` | integer | Number of partitions configured on this topic (useful to know how many parallel consumers you may run). | OPTIONAL | Must be positive
<a name="channelBindingObjectReplicas"></a>`replicas` | integer | Number of replicas configured on this topic. | OPTIONAL | MUST be positive
<a name="channelBindingObjectTopicConfiguration"></a>`topicConfiguration` | [TopicConfiguration Object](#topicConfiguration) | Topic configuration properties that are relevant for the API. | OPTIONAL | -
<a name="channelBindingObjectBindingVersion"></a>`bindingVersion` | string | The version of this binding. If omitted, "latest" MUST be assumed. | OPTIONAL [`latest`] | -


This object MUST contain only the properties defined above.

Expand All @@ -65,7 +67,38 @@ channels:
topic: 'my-specific-topic-name'
partitions: 20
replicas: 3
bindingVersion: '0.3.0'
topicConfiguration:
cleanup.policy: ["delete", "compact"]
retention.ms: 604800000
retention.bytes: 1000000000
delete.retention.ms: 86400000
max.message.bytes: 1048588
bindingVersion: '0.4.0'
```
<a name="topicConfiguration"></a>
## TopicConfiguration Object

This objects contains information about the API relevant topic configuration in Kafka.

Field Name | Type | Description | Applicability [default] | Constraints
---|:-------:|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------:|:-----------------------:|---
<a name="topicConfigurationCleanupPolicy"></a>`cleanup.policy` | array | The [`cleanup.policy`](https://kafka.apache.org/documentation/#topicconfigs_cleanup.policy) configuration option. | OPTIONAL | array may only contain `delete` and/or `compact`
thake marked this conversation as resolved.
Show resolved Hide resolved
<a name="topicConfigurationRetentionMs"></a>`retention.ms` | integer | The [`retention.ms`](https://kafka.apache.org/documentation/#topicconfigs_retention.ms) configuration option. | OPTIONAL | see kafka documentation
<a name="topicConfigurationRetentionBytes"></a>`retention.bytes` | integer | The [`retention.bytes`](https://kafka.apache.org/documentation/#topicconfigs_retention.bytes) configuration option. | OPTIONAL | see kafka documentation
<a name="topicConfigurationDeleteRetentionBytes"></a>`delete.retention.ms` | integer | The [`delete.retention.ms`](https://kafka.apache.org/documentation/#topicconfigs_delete.retention.ms) configuration option. | OPTIONAL | see kafka documentation
<a name="topicConfigurationMaxMessageBytes"></a>`max.message.bytes` | integer | The [`max.message.bytes`](https://kafka.apache.org/documentation/#topicconfigs_max.message.bytes) configuration option. | OPTIONAL | see kafka documentation

This object MUST contain only the properties defined above.

##### Example

```yaml
topicConfiguration:
cleanup.policy: ["delete", "compact"]
retention.ms: 604800000
retention.bytes: 1000000000
delete.retention.ms: 86400000
max.message.bytes: 1048588
```

<a name="operation"></a>
Expand Down Expand Up @@ -98,7 +131,7 @@ channels:
clientId:
type: string
enum: ['myClientId']
bindingVersion: '0.3.0'
bindingVersion: '0.4.0'
```


Expand Down Expand Up @@ -134,7 +167,7 @@ channels:
enum: ['myKey']
schemaIdLocation: 'payload'
schemaIdPayloadEncoding: '4'
bindingVersion: '0.3.0'
bindingVersion: '0.4.0'
```

This is another example that describes the use if Apicurio schema registry. We describe the `apicurio-new` way of serializing without details on how it's implemented. We reference a [specific lookup strategy](https://www.apicur.io/registry/docs/apicurio-registry/2.2.x/getting-started/assembly-using-kafka-client-serdes.html#registry-serdes-concepts-strategy_registry) that may be used to retrieve schema Id from registry during serialization.
Expand All @@ -152,7 +185,7 @@ channels:
schemaIdLocation: 'payload'
schemaIdPayloadEncoding: 'apicurio-new'
schemaLookupStrategy: 'TopicIdStrategy'
bindingVersion: '0.3.0'
bindingVersion: '0.4.0'
```

[schemaObject]: https://www.asyncapi.com/docs/specifications/2.4.0/#schemaObject
37 changes: 36 additions & 1 deletion kafka/json_schemas/channel.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,10 +25,45 @@
"minimum": 1,
"description": "Number of replicas configured on this topic."
},
"topicConfiguration" : {
"description": "Topic configuration properties that are relevant for the API.",
"type": "object",
"additionalProperties": false,
"properties": {
"cleanup.policy": {
"description": "The [`cleanup.policy`](https://kafka.apache.org/documentation/#topicconfigs_cleanup.policy) configuration option.",
"type": "array",
"items":{
"type": "string",
"enum": ["compact", "delete"]
}
},
"retention.ms": {
"description": "The [`retention.ms`](https://kafka.apache.org/documentation/#topicconfigs_retention.ms) configuration option.",
"type": "integer",
"minimum": -1
},
"retention.bytes": {
"description": "The [`retention.bytes`](https://kafka.apache.org/documentation/#topicconfigs_retention.bytes) configuration option.",
"type": "integer",
"minimum": -1
},
"delete.retention.ms": {
"description": "The [`delete.retention.ms`](https://kafka.apache.org/documentation/#topicconfigs_delete.retention.ms) configuration option.",
"type": "integer",
"minimum": 0
},
"max.message.bytes": {
"description": "The [`max.message.bytes`](https://kafka.apache.org/documentation/#topicconfigs_max.message.bytes) configuration option.",
"type": "integer",
"minimum": 0
}
}
},
"bindingVersion": {
"type": "string",
"enum": [
"0.3.0"
"0.4.0"
],
"description": "The version of this binding. If omitted, 'latest' MUST be assumed."
}
Expand Down
2 changes: 1 addition & 1 deletion kafka/json_schemas/message.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@
"bindingVersion": {
"type": "string",
"enum": [
"0.3.0"
"0.4.0"
],
"description": "The version of this binding. If omitted, 'latest' MUST be assumed."
}
Expand Down
2 changes: 1 addition & 1 deletion kafka/json_schemas/operation.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@
"bindingVersion": {
"type": "string",
"enum": [
"0.3.0"
"0.4.0"
],
"description": "The version of this binding. If omitted, 'latest' MUST be assumed."
}
Expand Down
2 changes: 1 addition & 1 deletion kafka/json_schemas/server.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@
"bindingVersion": {
"type": "string",
"enum": [
"0.3.0"
"0.4.0"
],
"description": "The version of this binding."
}
Expand Down