Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: tutorial for bindings with Kafka #2318

Merged
Merged
Changes from 27 commits
Commits
Show all changes
49 commits
Select commit Hold shift + click to select a range
232bb55
Initial draft for bindings with Kafka tutorials
CynthiaPeter Nov 14, 2023
b5fdf9f
Added feedback from Ludovic and moved the questions to the comments s…
CynthiaPeter Nov 15, 2023
94e4f50
Added the first version of a diagram
CynthiaPeter Nov 15, 2023
a8020e0
Rewrote the document
CynthiaPeter Nov 22, 2023
9506b89
Merge branch 'master' into feature/tutorial-bindings-with-kafka
quetzalliwrites Nov 28, 2023
a4217aa
implemented chnages and feedback
CynthiaPeter Nov 29, 2023
fb5d1e8
implemented feedback
CynthiaPeter Nov 29, 2023
60510bf
Updated docs
CynthiaPeter Dec 6, 2023
efcd397
Made changes based on feedback received
CynthiaPeter Dec 6, 2023
5cd1ac6
Added a new version for diagram
CynthiaPeter Dec 6, 2023
f3faba0
Updated the title to make it brief
CynthiaPeter Dec 6, 2023
bee1145
updated based on feedback
CynthiaPeter Dec 6, 2023
7edaf18
Apply suggestions from code review
quetzalliwrites Dec 6, 2023
cc4bcb1
Updated
CynthiaPeter Dec 6, 2023
2ca88a2
updated based on feedback received
CynthiaPeter Dec 12, 2023
c6746db
Updated background context
CynthiaPeter Dec 12, 2023
60172ce
Updated changes
CynthiaPeter Jan 11, 2024
8a8ea80
Removed index and section md
CynthiaPeter Jan 25, 2024
bc630a8
added code snippets from previous tutorial step
CynthiaPeter Feb 7, 2024
29383e9
editorial rewrite
quetzalliwrites Feb 10, 2024
3e201f2
Update pages/docs/tutorials/kafka/bindings-with-kafka.md
quetzalliwrites Feb 10, 2024
74dc1f3
Merge branch 'master' into feature/tutorial-bindings-with-kafka
quetzalliwrites Feb 10, 2024
abba6f8
Added metadata for page weight
CynthiaPeter Feb 12, 2024
ebab8c0
Merge branch 'asyncapi:master' into feature/tutorial-bindings-with-kafka
CynthiaPeter Feb 12, 2024
98d0cdb
Update pages/docs/tutorials/kafka/bindings-with-kafka.md
CynthiaPeter Feb 19, 2024
a33769a
Updated document based on feedback on PR
CynthiaPeter Feb 24, 2024
4ef8542
updated based on feedback from Dale. Used just channel.address
CynthiaPeter Mar 13, 2024
f50d8a8
update docs based on feedback
CynthiaPeter Mar 13, 2024
b3f6d3d
removed production
CynthiaPeter Mar 13, 2024
822a08a
edited bindingVersion
CynthiaPeter Mar 14, 2024
39f25e1
Update pages/docs/tutorials/kafka/bindings-with-kafka.md
CynthiaPeter Mar 14, 2024
37bba1d
Update pages/docs/tutorials/kafka/bindings-with-kafka.md
CynthiaPeter Mar 14, 2024
3b2c352
Update pages/docs/tutorials/kafka/bindings-with-kafka.md
CynthiaPeter Mar 14, 2024
f97c06c
Update pages/docs/tutorials/kafka/bindings-with-kafka.md
CynthiaPeter Mar 14, 2024
bc47bf1
Update pages/docs/tutorials/kafka/bindings-with-kafka.md
CynthiaPeter Mar 14, 2024
d2b19a7
update docs
CynthiaPeter Mar 14, 2024
1ef1c5e
fixed updates
CynthiaPeter Mar 14, 2024
fb2869a
Update pages/docs/tutorials/kafka/bindings-with-kafka.md
CynthiaPeter Mar 14, 2024
ac7da56
Update pages/docs/tutorials/kafka/bindings-with-kafka.md
CynthiaPeter Mar 14, 2024
4490849
update docs
CynthiaPeter Mar 14, 2024
6c57f54
Merge branch 'feature/tutorial-bindings-with-kafka' of https://github…
CynthiaPeter Mar 14, 2024
9626df0
removed the note on kafka.topic
CynthiaPeter Mar 14, 2024
b3b1bb7
fixed link error
CynthiaPeter Mar 14, 2024
194e6d5
Merge branch 'master' into feature/tutorial-bindings-with-kafka
derberg Mar 14, 2024
7141d68
Merge branch 'master' into feature/tutorial-bindings-with-kafka
quetzalliwrites Mar 15, 2024
3af3bab
grammar fixes
quetzalliwrites Mar 15, 2024
56610de
grammar fixes
quetzalliwrites Mar 15, 2024
87d457c
fix broken links
quetzalliwrites Mar 16, 2024
024e64a
Merge branch 'master' into feature/tutorial-bindings-with-kafka
quetzalliwrites Mar 16, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
183 changes: 183 additions & 0 deletions pages/docs/tutorials/kafka/bindings-with-kafka.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,183 @@
---
title: Kafka bindings
description: Learn how to configure Kafka bindings in your AsyncAPI document.
weight: 300
---

CynthiaPeter marked this conversation as resolved.
Show resolved Hide resolved
## Introduction

In a previous tutorial, you learned how to manage schemas with a schema registry. This tutorial teaches you how Kafka bindings function by defining Kafka messages and expanding your AsyncAPI document with protocol-specific details.
CynthiaPeter marked this conversation as resolved.
Show resolved Hide resolved

## Background context

CynthiaPeter marked this conversation as resolved.
Show resolved Hide resolved
Bindings are essential for event-driven applications because they provide protocol-specific details, abstracting the complexities of message handling from your application's core logic. They enhance the API's clarity and usability by offering setup options and context for different protocols. Bindings include details like the topics your application reads from or writes to, message formatting, and rules for interacting with multiple data or messages.
CynthiaPeter marked this conversation as resolved.
Show resolved Hide resolved

In an AsyncAPI document, bindings can be added to various sections like servers, channels, or messages. They contain protocol-specific details unique to each protocol. Binding definitions let you specify functionalities specific to the protocol, which are not covered by AsyncAPI's core features.

You can configure several objects using Kafka bindings. However, for the scope of this tutorial, you will focus on four levels of bindings: [server bindings](https://github.com/asyncapi/bindings/tree/master/kafka#server-binding-object), [operations binding](https://github.com/asyncapi/bindings/tree/master/kafka#operation-binding-object), [channel bindings](https://github.com/asyncapi/bindings/tree/master/kafka#channel-binding-object) and [message bindings](https://github.com/asyncapi/bindings/tree/master/kafka#message-binding-object).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please consider modification of this paragraph

here you should just have an intro to kafka binding and point to https://github.com/asyncapi/bindings/tree/master/kafka

then the links that you have, to specific objects, like message or others, they are useful, but super useful they would be if instead of here, they are in respective sections. So for example, as reader I would prefer to have a link to server kafka binding in server-related section, like a sentence below each code snippet


Using the code snippets from the previous tutorial, where you learned [how to manage Avro schemas using a centralized schema registry that enables you to share schemas across multiple applications](pages/docs/tutorials/kafka/managing-schemas-using-schema-registry), you will add configurations for server, operations, channel, and message bindings.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Using the code snippets from the previous tutorial, where you learned [how to manage Avro schemas using a centralized schema registry that enables you to share schemas across multiple applications](pages/docs/tutorials/kafka/managing-schemas-using-schema-registry), you will add configurations for server, operations, channel, and message bindings.
Using the code snippets from the previous tutorial, where you learned [how to manage Avro schemas using a centralized schema registry that enables you to share schemas across multiple applications](/docs/tutorials/kafka/managing-schemas-using-schema-registry), you will add configurations for server, operations, channel, and message bindings.


CynthiaPeter marked this conversation as resolved.
Show resolved Hide resolved
## Add server bindings

Server bindings provide protocol-specific configuration details for connecting and interacting with a server. For server bindings, we will add three fields:
CynthiaPeter marked this conversation as resolved.
Show resolved Hide resolved

```yaml
servers:
kafkaServer:
host: test.mykafkacluster.org:8092
description: Kafka Server
protocol: kafka
production:
bindings:
kafka:
schemaRegistryUrl: 'http://localhost:8080/apis/registry/'
schemaRegistryVendor: 'apicurio'
bindingVersion: '0.5.0'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please always validate with studio if your snippets are correct

Suggested change
servers:
kafkaServer:
host: test.mykafkacluster.org:8092
description: Kafka Server
protocol: kafka
production:
bindings:
kafka:
schemaRegistryUrl: 'http://localhost:8080/apis/registry/'
schemaRegistryVendor: 'apicurio'
bindingVersion: '0.5.0'
servers:
kafkaServer:
host: test.mykafkacluster.org:8092
description: Kafka Server
protocol: kafka
bindings:
kafka:
schemaRegistryUrl: 'http://localhost:8080/apis/registry/'
schemaRegistryVendor: 'apicurio'
bindingVersion: '0.4.0'

```

> Important: `Binding-version` is the field version of a binding. If omitted, the "latest" version MUST be assumed. The `bindingversion` field is an optional field that is available for all bindings.


## Add operation bindings

Operation bindings object contains information about the operation representation in Kafka (eg. the way to consume messages).

```yaml
operations:
onUserSignedUp:
action: receive
channel:
$ref: '#/channels/userSignedUp'
bindings:
kafka:
groupId:
type: string
enum: ['myGroupId']
clientId:
type: string
enum: ['myClientId']
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
operations:
onUserSignedUp:
action: receive
channel:
$ref: '#/channels/userSignedUp'
bindings:
kafka:
groupId:
type: string
enum: ['myGroupId']
clientId:
type: string
enum: ['myClientId']
operations:
onUserSignedUp:
action: receive
channel:
$ref: '#/channels/userSignedUp'
bindings:
kafka:
bindingVersion: '0.4.0'
groupId:
type: string
enum: ['myGroupId']
clientId:
type: string
enum: ['myClientId']

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a particular reason why we have bindingVersion specific to every configuration? Also, does this mean that in the final version, we have 4/5 different bindingVersion or we have just one?

```

## Add channel bindings

Channel bindings provide protocol-specific information for a particular channel. In Kafka, you can specify a given topic's number of partitions or replicas.

```yaml
channels:
userSignedUp:
description: This channel contains a message per each user who signs up in our application.
address: user_signedup
messages:
userSignedUp:
$ref: '#/components/messages/userSignedUp'
bindings:
kafka:
topic: 'UserSignedUp'
CynthiaPeter marked this conversation as resolved.
Show resolved Hide resolved
partitions: 10
replicas: 2
topicConfiguration:
cleanup.policy: ["delete", "compact"]
retention.ms: 604800000
retention.bytes: 1000000000
delete.retention.ms: 86400000
max.message.bytes: 1048588
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
kafka:
topic: 'UserSignedUp'
partitions: 10
replicas: 2
topicConfiguration:
cleanup.policy: ["delete", "compact"]
retention.ms: 604800000
retention.bytes: 1000000000
delete.retention.ms: 86400000
max.message.bytes: 1048588
kafka:
bindingVersion: '0.4.0'
topic: 'UserSignedUp'
partitions: 10
replicas: 2
topicConfiguration:
cleanup.policy: ["delete", "compact"]
retention.ms: 604800000
retention.bytes: 1000000000
delete.retention.ms: 86400000
max.message.bytes: 1048588

```

## Add message bindings

Message bindings provide protocol-specific information for a specific message. Like the Kafka topic, it's associated with a message key (if any) and Kafka-specific properties.
CynthiaPeter marked this conversation as resolved.
Show resolved Hide resolved

```yaml
components:
messages:
userSignedUp:
bindings:
kafka:
key:
type: string
enum: ['myKey']
schemaIdLocation: 'payload'
schemaIdPayloadEncoding: 'apicurio-new'
schemaLookupStrategy: 'TopicIdStrategy'
bindingVersion: '0.5.0'
derberg marked this conversation as resolved.
Show resolved Hide resolved
payload:
schemaFormat: 'application/vnd.apache.avro+json;version=1.9.0'
schema:
$ref: http://localhost:8080/apis/registry/v2/groups/my-group/artifacts/UserSignedUp
```

Congratulations, you've completed the tutorial! Putting these blocks together gives you an AsyncAPI document all ready to go.

```yaml
CynthiaPeter marked this conversation as resolved.
Show resolved Hide resolved
asyncapi: 3.0.0
info:
title: User Signup API
version: 1.0.0
description: The API notifies you whenever a new user signs up in the application.
servers:
kafkaServer:
host: test.mykafkacluster.org:8092
description: Kafka Server
protocol: kafka
production:
bindings:
kafka:
schemaRegistryUrl: 'http://localhost:8080/apis/registry/'
schemaRegistryVendor: 'apicurio'
bindingVersion: '0.5.0'
operations:
onUserSignedUp:
action: receive
channel:
$ref: '#/channels/userSignedUp'
bindings:
kafka:
groupId:
type: string
enum: ['myGroupId']
clientId:
type: string
enum: ['myClientId']
channels:
userSignedUp:
description: This channel contains a message per each user who signs up in our application.
address: user_signedup
messages:
userSignedUp:
$ref: '#/components/messages/userSignedUp'
bindings:
kafka:
partitions: 10
replicas: 2
topicConfiguration:
cleanup.policy: ["delete", "compact"]
retention.ms: 604800000
retention.bytes: 1000000000
delete.retention.ms: 86400000
max.message.bytes: 1048588
components:
messages:
userSignedUp:
bindings:
kafka:
key:
type: string
enum: ['myKey']
schemaIdLocation: 'payload'
schemaIdPayloadEncoding: 'apicurio-new'
schemaLookupStrategy: 'TopicIdStrategy'
payload:
schemaFormat: 'application/vnd.apache.avro+json;version=1.9.0'
schema:
$ref: http://localhost:8080/apis/registry/v2/groups/my-group/artifacts/UserSignedUp
```

## Summary

In this tutorial, you learned how to configure server and channel bindings. You also learned that bindings are essential when integrating Kafka with different systems, platforms, or protocols — especially in API specifications like AsyncAPI.


## Next steps

Now that you have completed this tutorial, you can [learn more about other Kakfa bindings](https://github.com/asyncapi/bindings/tree/master/kafka) or [protocol-specific bindings](https://github.com/asyncapi/bindings).
Loading