Promote `UnidirectionalTopicOperator` feature gate to GA #9840

fvaleri · 2024-03-15T12:49:34Z

This change promotes the UnidirectionalTopicOperator feature gate to GA. It removes the Bidirectional Topic Operator code and the UnidirectionalTopicOperator feature gate. It also updates all related tests and documentation.

Additional notes:

Azure pipeline changes needs to be verified, as I don't have access to that platform.
The KRaftKafkaUpgradeDowngradeST fails because STRIMZI_USE_NODE_POOLS_IN_TESTS is set to false by default (fixed separately).
There is a backlog of STs marked with UTONotSupported that should be re-enabled (fixed separately).

im-konge

Left few comments. Thanks a lot for the changes :)

api/src/main/java/io/strimzi/api/kafka/model/kafka/entityoperator/EntityOperatorSpec.java

documentation/modules/appendix_crds.adoc

documentation/modules/operators/con-topic-replication.adoc

documentation/modules/operators/ref-operator-cluster-feature-gates.adoc

systemtest/src/test/java/io/strimzi/systemtest/cruisecontrol/CruiseControlConfigurationST.java

systemtest/src/test/java/io/strimzi/systemtest/operators/topic/TopicST.java

scholzj

Thanks for the Pr, this looks good. I left some comments ... also some other things I think are missing:

In docker-images, I think we do not need to install the stunnel anymore and the scripts for running / configuring it.
In the docs, I left some comments. But in general, the docs are for Strimzi 0.41 and all mentiones of bidirectional TO can be removed apart form the feautre gates section that lists the (now unused) UTO feature gate and what it was doing. There is no need to explain that before 0.41 there was some other topic operator.

cluster-operator/src/test/java/io/strimzi/operator/cluster/model/EntityOperatorTest.java

documentation/modules/deploying/proc-deploy-migrate-kraft.adoc

documentation/modules/operators/con-topic-replication.adoc

documentation/modules/operators/proc-changing-topic-operator-mode.adoc

documentation/modules/operators/ref-operator-cluster-feature-gates.adoc

systemtest/src/test/java/io/strimzi/systemtest/mirrormaker/MirrorMaker2ST.java

CHANGELOG.md

fvaleri · 2024-03-15T17:13:39Z

@scholzj I guess these warnings should not appear when the deprecated properties are not used right?

status:
  clusterId: vPnoXMxwTyay53EMBJ3chQ
  conditions:
  - lastTransitionTime: "2024-03-15T17:11:41.886032728Z"
    status: "True"
    type: Ready
  - lastTransitionTime: "2024-03-15T17:10:09.681532310Z"
    message: In resource Kafka(test/my-cluster) in API version kafka.strimzi.io/v1beta2
      the zookeeperSessionTimeoutSeconds property at path spec.entityOperator.topicOperator.zookeeperSessionTimeoutSeconds
      has been deprecated. This property has been deprecated because ZooKeeper is
      not used anymore by the Topic Operator.
    reason: DeprecatedFields
    status: "True"
    type: Warning
  - lastTransitionTime: "2024-03-15T17:10:09.681532310Z"
    message: In resource Kafka(test/my-cluster) in API version kafka.strimzi.io/v1beta2
      the topicMetadataMaxAttempts property at path spec.entityOperator.topicOperator.topicMetadataMaxAttempts
      has been deprecated. This property has been deprecated because ZooKeeper is
      not used anymore by the Topic Operator.
    reason: DeprecatedFields
    status: "True"
    type: Warning
  kafkaMetadataState: ZooKeeper
  kafkaVersion: 3.7.0
...

scholzj · 2024-03-15T17:15:20Z

@fvaleri Are those int fields? MAybe you need to change them to Integer so that they default to null?

fvaleri · 2024-03-16T12:31:57Z

@fvaleri Are those int fields? MAybe you need to change them to Integer so that they default to null?

Not just that, we were also setting default values for these fields. Now it looks good. Please, run regression suites when you have time. Thanks.

see-quick · 2024-03-16T13:21:29Z

/azp run regression

azure-pipelines · 2024-03-16T13:21:40Z

Azure Pipelines successfully started running 1 pipeline(s).

scholzj · 2024-03-16T17:18:43Z

/azp run feature-gates-regression

azure-pipelines · 2024-03-16T17:18:54Z

Azure Pipelines successfully started running 1 pipeline(s).

scholzj

Some more nits. You should also:

Update the Helm Chart in packaging to temove the TLS sidecar image
Render the installation YAML from it

api/src/main/java/io/strimzi/api/kafka/model/kafka/entityoperator/EntityOperatorSpec.java

api/src/main/java/io/strimzi/api/kafka/model/kafka/entityoperator/EntityTopicOperatorSpec.java

cluster-operator/src/test/java/io/strimzi/operator/cluster/model/EntityTopicOperatorTest.java

scholzj · 2024-03-18T00:20:07Z

/azp run kraft-regression

azure-pipelines · 2024-03-18T00:20:19Z

Azure Pipelines successfully started running 1 pipeline(s).

scholzj · 2024-03-18T00:21:00Z

/azp run upgrade

azure-pipelines · 2024-03-18T00:21:08Z

Azure Pipelines successfully started running 1 pipeline(s).

ppatierno

Overall lgtm. I left a few minor comments.

CHANGELOG.md

cluster-operator/src/main/java/io/strimzi/operator/cluster/model/KRaftUtils.java

CHANGELOG.md

api/src/main/java/io/strimzi/api/kafka/model/kafka/entityoperator/EntityOperatorSpec.java

api/src/main/java/io/strimzi/api/kafka/model/kafka/entityoperator/EntityTopicOperatorSpec.java

Signed-off-by: Federico Valeri <[email protected]>

scholzj · 2024-03-19T20:01:29Z

/azp run kraft-regression

azure-pipelines · 2024-03-19T20:01:41Z

Azure Pipelines successfully started running 1 pipeline(s).

scholzj · 2024-03-19T20:02:28Z

/azp run upgrade

azure-pipelines · 2024-03-19T20:02:38Z

Azure Pipelines successfully started running 1 pipeline(s).

scholzj · 2024-03-20T06:44:50Z

/azp run regression

azure-pipelines · 2024-03-20T06:45:01Z

Azure Pipelines successfully started running 1 pipeline(s).

scholzj · 2024-03-20T06:45:11Z

/azp run feature-gates-regression

azure-pipelines · 2024-03-20T06:45:20Z

Azure Pipelines successfully started running 1 pipeline(s).

scholzj · 2024-03-20T06:54:56Z

The KafkaST#testRemoveComponentsFromEntityOperator in kraft-regression pipeline is working fine on my end, so I guess it is a bit flaky.

@fvaleri I restarted it. But KafkaST.testRemoveComponentsFromEntityOperator keeps failing and seems to be failing only on your PR.

see-quick

LGTM, assuming tests pass. 👍

Signed-off-by: Federico Valeri <[email protected]>

fvaleri · 2024-03-20T07:53:53Z

@fvaleri I restarted it. But KafkaST.testRemoveComponentsFromEntityOperator keeps failing and seems to be failing only on your PR.

Ok, found the issue. With the last commit KafkaST.testRemoveComponentsFromEntityOperator should work with both STRIMZI_USE_KRAFT_IN_TESTS equal true (test currently failing) and false.

scholzj · 2024-03-20T08:47:25Z

/azp run kraft-regression

azure-pipelines · 2024-03-20T08:47:34Z

Azure Pipelines successfully started running 1 pipeline(s).

scholzj · 2024-03-20T19:00:02Z

/azp run regression

azure-pipelines · 2024-03-20T19:00:16Z

Azure Pipelines successfully started running 1 pipeline(s).

scholzj · 2024-03-20T19:00:22Z

/azp run feature-gates-regression

azure-pipelines · 2024-03-20T19:00:32Z

Azure Pipelines successfully started running 1 pipeline(s).

Signed-off-by: Federico Valeri <[email protected]> Signed-off-by: Lukas Kral <[email protected]> Co-authored-by: Lukas Kral <[email protected]> Signed-off-by: Steffen Karlsson <[email protected]>

Signed-off-by: Federico Valeri <[email protected]> Signed-off-by: Lukas Kral <[email protected]> Co-authored-by: Lukas Kral <[email protected]>

fvaleri requested review from PaulRMellor, scholzj and tombentley and removed request for PaulRMellor and scholzj March 15, 2024 12:49

fvaleri added this to the 0.41.0 milestone Mar 15, 2024

fvaleri requested review from ppatierno and im-konge March 15, 2024 12:50

im-konge reviewed Mar 15, 2024

View reviewed changes

scholzj reviewed Mar 15, 2024

View reviewed changes

fvaleri force-pushed the promote-uto-ga branch from 4b13652 to 56e317e Compare March 15, 2024 17:14

fvaleri force-pushed the promote-uto-ga branch from 56e317e to 46f264c Compare March 16, 2024 12:31

scholzj reviewed Mar 16, 2024

View reviewed changes

ppatierno reviewed Mar 18, 2024

View reviewed changes

CHANGELOG.md Outdated Show resolved Hide resolved

CHANGELOG.md Outdated Show resolved Hide resolved

cluster-operator/src/main/java/io/strimzi/operator/cluster/model/KRaftUtils.java Show resolved Hide resolved

ppatierno mentioned this pull request Mar 18, 2024

Remove any stunnel reference and usage jboss-container-images/amqstreams-1-openshift-image#455

Open

scholzj reviewed Mar 18, 2024

View reviewed changes

CHANGELOG.md Outdated Show resolved Hide resolved

scholzj reviewed Mar 18, 2024

View reviewed changes

fvaleri force-pushed the promote-uto-ga branch from 0312a72 to b4c4a96 Compare March 19, 2024 15:53

Fix docs error

b7106be

Signed-off-by: Federico Valeri <[email protected]>

see-quick approved these changes Mar 20, 2024

View reviewed changes

im-konge approved these changes Mar 20, 2024

View reviewed changes

Fix KafkaST

3bb7201

Signed-off-by: Federico Valeri <[email protected]>

fvaleri requested a review from scholzj March 21, 2024 11:51

scholzj approved these changes Mar 21, 2024

View reviewed changes

scholzj merged commit 05d0c94 into strimzi:main Mar 21, 2024
37 checks passed

fvaleri deleted the promote-uto-ga branch March 21, 2024 13:32

PaulRMellor mentioned this pull request Mar 27, 2024

docs(uto): removes some references to using the bidirectional topic operator #9885

Merged

8 tasks

scholzj mentioned this pull request May 7, 2024

Remove the unused Entity Operator TLS sidecar configuration from the installation files, Helm Chart and docs #10077

Merged

2 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Promote `UnidirectionalTopicOperator` feature gate to GA #9840

Promote `UnidirectionalTopicOperator` feature gate to GA #9840

fvaleri commented Mar 15, 2024

im-konge left a comment

scholzj left a comment

fvaleri commented Mar 15, 2024

scholzj commented Mar 15, 2024

fvaleri commented Mar 16, 2024 •

edited

Loading

see-quick commented Mar 16, 2024

azure-pipelines bot commented Mar 16, 2024

scholzj commented Mar 16, 2024

azure-pipelines bot commented Mar 16, 2024

scholzj left a comment

scholzj commented Mar 18, 2024

azure-pipelines bot commented Mar 18, 2024

scholzj commented Mar 18, 2024

azure-pipelines bot commented Mar 18, 2024

ppatierno left a comment

scholzj commented Mar 19, 2024

azure-pipelines bot commented Mar 19, 2024

scholzj commented Mar 19, 2024

azure-pipelines bot commented Mar 19, 2024

scholzj commented Mar 20, 2024

azure-pipelines bot commented Mar 20, 2024

scholzj commented Mar 20, 2024

azure-pipelines bot commented Mar 20, 2024

scholzj commented Mar 20, 2024

see-quick left a comment

fvaleri commented Mar 20, 2024 •

edited

Loading

scholzj commented Mar 20, 2024

azure-pipelines bot commented Mar 20, 2024

scholzj commented Mar 20, 2024

azure-pipelines bot commented Mar 20, 2024

scholzj commented Mar 20, 2024

azure-pipelines bot commented Mar 20, 2024

Promote UnidirectionalTopicOperator feature gate to GA #9840

Promote UnidirectionalTopicOperator feature gate to GA #9840

Conversation

fvaleri commented Mar 15, 2024

im-konge left a comment

Choose a reason for hiding this comment

scholzj left a comment

Choose a reason for hiding this comment

fvaleri commented Mar 15, 2024

scholzj commented Mar 15, 2024

fvaleri commented Mar 16, 2024 • edited Loading

see-quick commented Mar 16, 2024

azure-pipelines bot commented Mar 16, 2024

scholzj commented Mar 16, 2024

azure-pipelines bot commented Mar 16, 2024

scholzj left a comment

Choose a reason for hiding this comment

scholzj commented Mar 18, 2024

azure-pipelines bot commented Mar 18, 2024

scholzj commented Mar 18, 2024

azure-pipelines bot commented Mar 18, 2024

ppatierno left a comment

Choose a reason for hiding this comment

scholzj commented Mar 19, 2024

azure-pipelines bot commented Mar 19, 2024

scholzj commented Mar 19, 2024

azure-pipelines bot commented Mar 19, 2024

scholzj commented Mar 20, 2024

azure-pipelines bot commented Mar 20, 2024

scholzj commented Mar 20, 2024

azure-pipelines bot commented Mar 20, 2024

scholzj commented Mar 20, 2024

see-quick left a comment

Choose a reason for hiding this comment

fvaleri commented Mar 20, 2024 • edited Loading

scholzj commented Mar 20, 2024

azure-pipelines bot commented Mar 20, 2024

scholzj commented Mar 20, 2024

azure-pipelines bot commented Mar 20, 2024

scholzj commented Mar 20, 2024

azure-pipelines bot commented Mar 20, 2024

Promote `UnidirectionalTopicOperator` feature gate to GA #9840

Promote `UnidirectionalTopicOperator` feature gate to GA #9840

fvaleri commented Mar 16, 2024 •

edited

Loading

fvaleri commented Mar 20, 2024 •

edited

Loading