input: kafka: Document data_format option (#1118)

* input: kafka: Document `data_format` option Signed-off-by: Thiago Padilha <[email protected]> * Rename data_format to format Signed-off-by: Thiago Padilha <[email protected]> --------- Signed-off-by: Thiago Padilha <[email protected]>
fluent · Jan 4, 2024 · 0e548b7 · 0e548b7
1 parent 5c2bc16
commit 0e548b7
Showing 1 changed file with 4 additions and 0 deletions.
diff --git a/pipeline/inputs/kafka.md b/pipeline/inputs/kafka.md
@@ -9,6 +9,7 @@ This plugin uses the official [librdkafka C library](https://github.com/edenhill
 | :--- | :--- | :--- |
 | brokers | Single or multiple list of Kafka Brokers, e.g: 192.168.1.3:9092, 192.168.1.4:9092. |  |
 | topics | Single entry or list of topics separated by comma \(,\) that Fluent Bit will subscribe to. |  |
+| format | Serialization format of the messages. If set to "json", the payload will be parsed as json. | none  |
 | client\_id | Client id passed to librdkafka. | |
 | group\_id | Group id passed to librdkafka. | fluent-bit |
 | poll\_ms | Kafka brokers polling interval in milliseconds. | 500 |
@@ -53,6 +54,7 @@ The fluent-bit source repository contains a full example of using fluent-bit to
     brokers kafka-broker:9092
     topics fb-source
     poll_ms 100
+    format json
 
 [FILTER]
     Name    lua
@@ -68,6 +70,8 @@ The fluent-bit source repository contains a full example of using fluent-bit to
 
 The above will connect to the broker listening on `kafka-broker:9092` and subscribe to the `fb-source` topic, polling for new messages every 100 milliseconds.
 
+Since the payload will be in json format, we ask the plugin to automatically parse the payload with `format json`.
+
 Every message received is then processed with `kafka.lua` and sent back to the `fb-sink` topic of the same broker.
 
 The example can be executed locally with `make start` in the `examples/kafka_filter` directory (docker/compose is used).