Is there a plan for migrating off kafka-python and aiokafka?

[brokenjacobs]

What can we help you with?

I'm concerned that karapace, while actively maintained, is dependent on two kafka libraries that are not.

https://github.com/dpkp/kafka-python
Last released two years ago.

https://github.com/aio-libs/aiokafka
Last released 1 year ago.

In the case of kakfa-python the project appears dead. Simple PR's aren't being merged for python 3.10 compatibility. For instance: dpkp/kafka-python#2304

Is there a plan for dealing with this in karapace? I have similar concerns for internal software based on kafka-python and am trying to figure out what do as well.

Where would you expect to find this information?

A software roadmap on github or in the documentation.

[tvainika]

Hi,

we have identified also that kafka-python is not maintained. Our migration has been towards aiokafka, which has some development activity even though there has not been releases lately.

This issue highlights whole Python ecosystem with Kafka. Probably the best maintained library is confluent-kafka-python, which is rdkafka based implementation. However that API is not async in Python.

So to conclude, there is a goal to get rid of kafka-python for now.

[jjaakola-aiven]

The change to rdkafka is mostly done. For schema registry the primary selection still relies on kafka-python and the BaseCoordinator + KafkaClient.

[brokenjacobs]

looks like someone is maintaining kafka-python fyi:
https://github.com/wbarnha/kafka-python-ng