diff --git a/_posts/2024-08-05-Debezium-ui-update.adoc b/_posts/2024-08-05-Debezium-ui-update.adoc new file mode 100644 index 00000000000..466d436d998 --- /dev/null +++ b/_posts/2024-08-05-Debezium-ui-update.adoc @@ -0,0 +1,101 @@ +--- +layout: post +title: Status of Debezium UI +date: 2024-08-05 +tags: [ debezium, ui, features, kubernetes, operator, debezium-server ] +author: jcechace +--- +# Status of Debezium UI + +Hello everyone, Jakub here. You may have noticed that there wasn't much happening around Debezium UI lately. +This, however, would be only partially true. We own you an explanation in this regard, so please bear with me. + +Let's start with the status of the **current** UI project. It became increasing clear that while UI +for Debezium is an important part of our vision, developing a UI strictly tied to Kafka Connect is +not the right direction. It was thus decided to freeze the development of the **current** web UI +project. There were two main reasons for this decision: + +1. The team doesn't have the bandwidth (nor the desire) to develop a general purpose Kafka connector UI that would +be on par with some of the existing offerings (e.g. Lenses). So our UI would only ever be able to work with Debezium connectors specifically. +2. Debezium Server is gaining popularity and we would like to offer a user friendly way of managing this flavour of Debezium as well. +Unfortunately, the deployment model of Debezium Server is very different from deploying Debezium as a connector. + +## What's Next? +As the previous paragraph mentioned the **current** UI, it's probably not a suprise that we did not give up on the idea. Part of the decision +was to explore a possibility of creating a new UI which, while still focused exclusively on Debezium, would gradually support multiple deployment +models. The idea is to start with Debezium Server deployed on Kubernetes, followed by bare metal deployment of Debezium Server. +At some point in the future we would like to cycle back to Kafka Connect and support also the deployment and operation of Debezium as Kafka +Connect connectors. + +++++ +
+ Architecture +
+++++ + +## Designing the New UI +One of the goals we defined for ourselves when designing the new platform was to simplify the understanding of what Debezium does. +For this reason, we are going to abstract away from the underlying deployment model. Instead of focusing on servers and connectors, +we want to allow our users to think about Debezium in terms of data. Thus the new generation of the UI should operate with data pipelines +-- you define the source, the destination, and any data transformations. + +These can then be composed into a deployable pipeline. Depending on the target environment (Kafka Connect, Kubernetes, bare metal), +an appropriate deployment will be used. + +++++ +
+ Initial Wireframe Design +
+++++ + +When pipeline is deployed to a Kubernetes or a bare metal machine + +- The Source is the configuration of the Debezium Source connector +- Transformations are the usual SMTs +- The Destination coresponds to the configuration of Debezium Server's Sink. + +When pipeline is deployed to Kafka Connect + +- The Source is the configuration of the Debezium Source connector +- Transformations are the usual SMTs +- The destination is always a Kafka topic + +In the future, we could also allow our JDBC and MongoDB sink connectors to be used as destinations, allowing for truly end-to-end defined pipelines +when Kafka Connect is targeted. + +## Where Do We Stand Right Now +We are at the begining of this journey. Nevetheless, there is a rough PoC prototype ready for you to https://github.com/debezium/debezium-platform-conductor/tree/main?tab=readme-ov-file#running-the-compose-kind-kafka-example[examine]. +As already briefly mentioned, initially the plan is to target only Kubernetes as supported deployment environment, more specifically +Debezium Server on Kubernetes. This choice allows us to take an advantage of our Debezium Operator. + +++++ +
+ Screenshot: Destination types list +
+++++ + +We intend to introduce configuration and feature support gradually. Thus the configuration is deliberately opinionated and +simplistic -- currently all pipelines can only use ephemeral storage. In this early version, you must also set all configuration properties as simple key-value pairs. We intend to change this in the future, and as a next step, you can look forward to auto-completion of property names, +and ultimately, fully specialized forms will be rendered. + +++++ +
+ Screenshot: Create new source +
+++++ + +You can review the current status of the codebase on GitHub: + +- https://github.com/debezium/debezium-platform-conductor/[Debezium Platform Conductor (Backend)] +- https://github.com/debezium/debezium-platform-stage/[Debezium Platform Stage (Frontend)] + +## We need your help! +At the end of this short report, we would like to ask you to try this early PoC and let us know your opinion. +We would especially appreciate if you could let as know your answers to the following questions: + +- What features are a must for you? +- What features would you consider nice to have? +- What is your visual impression? + +Thank you and expect to hear from us soon with more updates to come! + diff --git a/assets/images/2024-08-05-debezium-ui-update/architecture.png b/assets/images/2024-08-05-debezium-ui-update/architecture.png new file mode 100644 index 00000000000..06d73e5671b Binary files /dev/null and b/assets/images/2024-08-05-debezium-ui-update/architecture.png differ diff --git a/assets/images/2024-08-05-debezium-ui-update/screenshot_create_source.png b/assets/images/2024-08-05-debezium-ui-update/screenshot_create_source.png new file mode 100644 index 00000000000..d32983b6c2b Binary files /dev/null and b/assets/images/2024-08-05-debezium-ui-update/screenshot_create_source.png differ diff --git a/assets/images/2024-08-05-debezium-ui-update/screenshot_list_destionations.png b/assets/images/2024-08-05-debezium-ui-update/screenshot_list_destionations.png new file mode 100644 index 00000000000..9b41f3703b3 Binary files /dev/null and b/assets/images/2024-08-05-debezium-ui-update/screenshot_list_destionations.png differ diff --git a/assets/images/2024-08-05-debezium-ui-update/wireframe.png b/assets/images/2024-08-05-debezium-ui-update/wireframe.png new file mode 100644 index 00000000000..581ef6b3790 Binary files /dev/null and b/assets/images/2024-08-05-debezium-ui-update/wireframe.png differ