Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: Upgrade Kafka dependencies to 2.3.1 #9582

Merged
merged 1 commit into from
Jan 22, 2020

Conversation

adamkotwasinski
Copy link
Contributor

@adamkotwasinski adamkotwasinski commented Jan 7, 2020

Description:
Upgrade Kafka client & server dependencies to 2.3.1.
In detail, we generate C++ code from metadata files in Kafka Java client source:

  • 2.3 introduced default null value for some fields
  • 2.3 introduced sub-structures that have the same name in different requests (e.g. AlterConfigs & IncrementalAlterConfigs have both AlterConfig sub-structures)
  • 2.3 introduced a structure that has 10 fields (so need to generate one more composite deserializer)

This PR is the first one in upgrade series, I will need to follow up with upgrade to 2.4, it demands more work than I expected as it includes binary protocol changes described in https://cwiki.apache.org/confluence/display/KAFKA/KIP-482%3A+The+Kafka+Protocol+should+Support+Optional+Tagged+Fields
(tldr: new data types, changed structures, input json files also changed)

Risk Level: Low
Testing:

  • unit tests
  • integration tests using real Kafka server (2.3.1 as in dependencies) via bazel test //test/extensions/filters/network/kafka/broker/integration_test:kafka_broker_integration_test --runs_per_test 100
  • manual testing with 2.3.1 Java Kafka client (on my machine)

Docs Changes: Kafka version mentions in docs
Release Notes: N/A

@adamkotwasinski
Copy link
Contributor Author

/wait
(client moved to 2.3.1, but 2.4 will need some more python scripting)

@stale
Copy link

stale bot commented Jan 14, 2020

This pull request has been automatically marked as stale because it has not had activity in the last 7 days. It will be closed in 7 days if no further activity occurs. Please feel free to give a status update now, ping for review, or re-open when it's ready. Thank you for your contributions!

@stale stale bot added the stale stalebot believes this issue/PR has not been touched recently label Jan 14, 2020
@stale stale bot removed the stale stalebot believes this issue/PR has not been touched recently label Jan 17, 2020
@adamkotwasinski adamkotwasinski changed the title WIP: Upgrade Kafka dependencies to the most recent versions WIP: Upgrade Kafka dependencies to 2.3.1 Jan 22, 2020
@adamkotwasinski
Copy link
Contributor Author

/assign @mattklein123

Copy link
Member

@mattklein123 mattklein123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@mattklein123 mattklein123 merged commit a8e0b71 into envoyproxy:master Jan 22, 2020
@adamkotwasinski adamkotwasinski deleted the kafka branch January 22, 2020 23:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants