Skip to content

Commit

Permalink
Incremental alter configs implementation [KIP-339] (#4110)
Browse files Browse the repository at this point in the history 
requires broker version >= 2.3.0
---------

Co-authored-by: Emanuele Sabellico <[email protected]>
  • Loading branch information
@PrasanthV454 @emasab
PrasanthV454 and emasab authored Jul 10, 2023
1 parent 961946e commit 53a6a50
Show file tree
Hide file tree
Showing 22 changed files with 1,427 additions and 216 deletions.
4 changes: 3 additions & 1 deletion CHANGELOG.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,10 @@ librdkafka v2.2.0 is a feature release:
closes as normal ones (#4294).
* Added `fetch.queue.backoff.ms` to the consumer to control how long
the consumer backs off next fetch attempt. (@bitemyapp, @edenhill, #2879)
* [KIP-235](https://cwiki.apache.org/confluence/display/KAFKA/KIP-235%3A+Add+DNS+alias+support+for+secured+connection):
* [KIP-235](https://cwiki.apache.org/confluence/display/KAFKA/KIP-235%3A+Add+DNS+alias+support+for+secured+connection):
Add DNS alias support for secured connection (#4292).
* [KIP-339](https://cwiki.apache.org/confluence/display/KAFKA/KIP-339%3A+Create+a+new+IncrementalAlterConfigs+API):
IncrementalAlterConfigs API (started by @PrasanthV454, #4110).


## Enhancements
Expand Down
67 changes: 34 additions & 33 deletions INTRODUCTION.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -1909,7 +1909,7 @@ The [Apache Kafka Implementation Proposals (KIPs)](https://cwiki.apache.org/conf
| KIP-302 - Use all addresses for resolved broker hostname | 2.1.0 | Supported |
| KIP-320 - Consumer: handle log truncation | 2.1.0, 2.2.0 | Supported |
| KIP-322 - DeleteTopics disabled error code | 2.1.0 | Supported |
| KIP-339 - AdminAPI: incrementalAlterConfigs | 2.3.0 | Not supported |
| KIP-339 - AdminAPI: incrementalAlterConfigs | 2.3.0 | Supported |
| KIP-341 - Update Sticky partition assignment data | 2.3.0 | Not supported (superceeded by KIP-429) |
| KIP-342 - Custom SASL OAUTHBEARER extensions | 2.1.0 | Supported |
| KIP-345 - Consumer: Static membership | 2.4.0 | Supported |
Expand Down Expand Up @@ -1964,42 +1964,43 @@ The [Apache Kafka Implementation Proposals (KIPs)](https://cwiki.apache.org/conf

### Supported protocol versions

"Kafka max" is the maximum ApiVersion supported in Apache Kafka 3.3.1, while
"Kafka max" is the maximum ApiVersion supported in Apache Kafka 3.4.0, while
"librdkafka max" is the maximum ApiVersion supported in the latest
release of librdkafka.


| ApiKey | Request name | Kafka max | librdkafka max |
| ------- | ------------------- | ----------- | ----------------------- |
| 0 | Produce | 9 | 7 |
| 1 | Fetch | 13 | 11 |
| 2 | ListOffsets | 7 | 2 |
| 3 | Metadata | 12 | 9 |
| 8 | OffsetCommit | 8 | 7 |
| 9 | OffsetFetch | 8 | 7 |
| 10 | FindCoordinator | 4 | 2 |
| 11 | JoinGroup | 9 | 5 |
| 12 | Heartbeat | 4 | 3 |
| 13 | LeaveGroup | 5 | 1 |
| 14 | SyncGroup | 5 | 3 |
| 15 | DescribeGroups | 5 | 4 |
| 16 | ListGroups | 4 | 4 |
| 17 | SaslHandshake | 1 | 1 |
| 18 | ApiVersions | 3 | 3 |
| 19 | CreateTopics | 7 | 4 |
| 20 | DeleteTopics | 6 | 1 |
| 21 | DeleteRecords | 2 | 1 |
| 22 | InitProducerId | 4 | 4 |
| 24 | AddPartitionsToTxn | 3 | 0 |
| 25 | AddOffsetsToTxn | 3 | 0 |
| 26 | EndTxn | 3 | 1 |
| 28 | TxnOffsetCommit | 3 | 3 |
| 32 | DescribeConfigs | 4 | 1 |
| 33 | AlterConfigs | 2 | 1 |
| 36 | SaslAuthenticate | 2 | 1 |
| 37 | CreatePartitions | 3 | 0 |
| 42 | DeleteGroups | 2 | 1 |
| 47 | OffsetDelete | 0 | 0 |
| ApiKey | Request name | Kafka max | librdkafka max |
| ------- | ------------------------| ----------- | ----------------------- |
| 0 | Produce | 9 | 7 |
| 1 | Fetch | 13 | 11 |
| 2 | ListOffsets | 7 | 2 |
| 3 | Metadata | 12 | 9 |
| 8 | OffsetCommit | 8 | 7 |
| 9 | OffsetFetch | 8 | 7 |
| 10 | FindCoordinator | 4 | 2 |
| 11 | JoinGroup | 9 | 5 |
| 12 | Heartbeat | 4 | 3 |
| 13 | LeaveGroup | 5 | 1 |
| 14 | SyncGroup | 5 | 3 |
| 15 | DescribeGroups | 5 | 4 |
| 16 | ListGroups | 4 | 4 |
| 17 | SaslHandshake | 1 | 1 |
| 18 | ApiVersions | 3 | 3 |
| 19 | CreateTopics | 7 | 4 |
| 20 | DeleteTopics | 6 | 1 |
| 21 | DeleteRecords | 2 | 1 |
| 22 | InitProducerId | 4 | 4 |
| 24 | AddPartitionsToTxn | 3 | 0 |
| 25 | AddOffsetsToTxn | 3 | 0 |
| 26 | EndTxn | 3 | 1 |
| 28 | TxnOffsetCommit | 3 | 3 |
| 32 | DescribeConfigs | 4 | 1 |
| 33 | AlterConfigs | 2 | 2 |
| 36 | SaslAuthenticate | 2 | 0 |
| 37 | CreatePartitions | 3 | 0 |
| 42 | DeleteGroups | 2 | 1 |
| 44 | IncrementalAlterConfigs | 1 | 1 |
| 47 | OffsetDelete | 0 | 0 |



Expand Down
2 changes: 1 addition & 1 deletion LICENSES.txt
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ LICENSE
librdkafka - Apache Kafka C driver library

Copyright (c) 2012-2022, Magnus Edenhill
2023 Confluent Inc.
2023, Confluent Inc.
All rights reserved.

Redistribution and use in source and binary forms, with or without
Expand Down
1 change: 1 addition & 0 deletions examples/.gitignore
View file
Original file line number Diff line number Diff line change
Expand Up @@ -17,3 +17,4 @@ list_consumer_groups
describe_consumer_groups
list_consumer_group_offsets
alter_consumer_group_offsets
incremental_alter_configs
23 changes: 23 additions & 0 deletions examples/CMakeLists.txt
View file
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,29 @@ target_link_libraries(openssl_engine_example_cpp PUBLIC rdkafka++)
add_executable(misc misc.c ${win32_sources})
target_link_libraries(misc PUBLIC rdkafka)

add_executable(idempotent_producer idempotent_producer.c ${win32_sources})
target_link_libraries(idempotent_producer PUBLIC rdkafka)

add_executable(transactions transactions.c ${win32_sources})
target_link_libraries(transactions PUBLIC rdkafka)

add_executable(delete_records delete_records.c ${win32_sources})
target_link_libraries(delete_records PUBLIC rdkafka)

add_executable(list_consumer_groups list_consumer_groups.c ${win32_sources})
target_link_libraries(list_consumer_groups PUBLIC rdkafka)

add_executable(describe_consumer_groups describe_consumer_groups.c ${win32_sources})
target_link_libraries(describe_consumer_groups PUBLIC rdkafka)

add_executable(list_consumer_group_offsets list_consumer_group_offsets.c ${win32_sources})
target_link_libraries(list_consumer_group_offsets PUBLIC rdkafka)

add_executable(alter_consumer_group_offsets alter_consumer_group_offsets.c ${win32_sources})
target_link_libraries(alter_consumer_group_offsets PUBLIC rdkafka)

add_executable(incremental_alter_configs incremental_alter_configs.c ${win32_sources})
target_link_libraries(incremental_alter_configs PUBLIC rdkafka)

# The targets below has Unix include dirs and do not compile on Windows.
if(NOT WIN32)
Expand Down
5 changes: 5 additions & 0 deletions examples/Makefile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ EXAMPLES ?= rdkafka_example rdkafka_performance rdkafka_example_cpp \
describe_consumer_groups \
list_consumer_group_offsets \
alter_consumer_group_offsets \
incremental_alter_configs \
misc

all: $(EXAMPLES)
Expand Down Expand Up @@ -80,6 +81,10 @@ alter_consumer_group_offsets: ../src/librdkafka.a alter_consumer_group_offsets.c
$(CC) $(CPPFLAGS) $(CFLAGS) $@.c -o $@ $(LDFLAGS) \
../src/librdkafka.a $(LIBS)

incremental_alter_configs: ../src/librdkafka.a incremental_alter_configs.c
$(CC) $(CPPFLAGS) $(CFLAGS) $@.c -o $@ $(LDFLAGS) \
../src/librdkafka.a $(LIBS)

rdkafka_complex_consumer_example: ../src/librdkafka.a rdkafka_complex_consumer_example.c
$(CC) $(CPPFLAGS) $(CFLAGS) rdkafka_complex_consumer_example.c -o $@ $(LDFLAGS) \
../src/librdkafka.a $(LIBS)
Expand Down
1 change: 1 addition & 0 deletions examples/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -36,3 +36,4 @@ For more complex uses, see:
* [describe_consumer_groups.c](describe_consumer_groups.c) - Describe consumer groups.
* [list_consumer_group_offsets.c](list_consumer_group_offsets.c) - List offsets of a consumer group.
* [alter_consumer_group_offsets.c](alter_consumer_group_offsets.c) - Alter offsets of a consumer group.
* [incremental_alter_configs.c](incremental_alter_configs.c) - Incrementally alter resource configurations.
Loading

0 comments on commit 53a6a50

Please sign in to comment.