Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Missing attribute "confluent_topic_type" of Kafka topic config in Terraform provider #427

Open
RishuSinghS opened this issue Sep 2, 2024 · 4 comments
Labels
bug Something isn't working good first issue Good for newcomers

Comments

@RishuSinghS
Copy link

RishuSinghS commented Sep 2, 2024

Hello,

I have noticed that a new attribute, confluent_topic_type, is automatically added to Kafka Topic configurations when created manually through the Confluent portal.

image

However, I cannot find this attribute currently documented or supported in the Terraform provider for Kafka topics: Terraform Kafka Topic Resource.

Our workflow fails when attempting to import a manually created topic through Terraform, as it tries to remove the confluent.topic.type attribute:

### 0 to add, 1 to change, 0 to destroy, 0 to replace.
- change
    - module.kafka_topic.confluent_kafka_topic.kafka_topic
<details><summary>Change details</summary>

# module.kafka_topic.confluent_kafka_topic.kafka_topic will be updated in-place
@@ -3,7 +3,6 @@
     "cleanup.policy": "delete",
     "confluent.key.schema.validation": "false",
     "confluent.key.subject.name.strategy": "io.confluent.kafka.serializers.subject.TopicNameStrategy",
-    "confluent.topic.type": "standard",
     "confluent.value.schema.validation": "true",
     "confluent.value.subject.name.strategy": "io.confluent.kafka.serializers.subject.TopicNameStrategy",
     "delete.retention.ms": "86400000",

Could you please assist with a fix for this issue?

@linouk23 linouk23 added bug Something isn't working good first issue Good for newcomers labels Sep 3, 2024
@channingdong
Copy link
Contributor

Hi Customer,

Which TF provider version were you using to replicate this issue?

And to replicate the issue, is the step-by-step workflow as below?

  • Create a Kafka topic in UI
  • Make sure confluent.topic.type is present.
  • Import this Kafka topic through TF import command
  • Run terraform plan
  • Noticed the TF drift described above

@channingdong
Copy link
Contributor

Also, could you please share how do you import the manually created Kafka topic through TF? You don't need to share the whole code block, the essential code snippet is okay.

@dedovicnermin
Copy link

dedovicnermin commented Oct 10, 2024

Important to create the topic through the UI (DYNAMIC_TOPIC_CONFIG)

TF import block
Instead of running terraform import command, include an import block and run terraform plan

import {
  to = confluent_kafka_topic.example
  id = "lkc-xyz/orders"
}

resource "confluent_kafka_topic" "example" {
  kafka_cluster {
    id = lkc-xyz
  }
  topic_name         = "orders"
  partitions_count   = 4
  rest_endpoint      = confluent_kafka_cluster.basic-cluster.rest_endpoint
  config = {
    "cleanup.policy"                      = "delete"
    "delete.retention.ms"                 = "86400000"
    "max.compaction.lag.ms"               = "9223372036854775807"
    "max.message.bytes"                   = "2097164"
    "message.timestamp.after.max.ms"      = "9223372036854775807"
    "message.timestamp.before.max.ms"     = "9223372036854775807"      
    "message.timestamp.difference.max.ms" = "9223372036854775807"
    "message.timestamp.type"              = "CreateTime"
    "min.compaction.lag.ms"               = "0"
    "min.insync.replicas"                 = "2"
    "retention.bytes"                     = "-1"
    "retention.ms"                        = "604800000"
    "segment.bytes"                       = "104857600"
    "segment.ms"                          = "604800000"
  }
  credentials {
    key    = confluent_api_key.app-manager-kafka-api-key.id
    secret = confluent_api_key.app-manager-kafka-api-key.secret
  }
}

@davekpatrick
Copy link

any update on the resolution of this issue ? Pull request
#437 has been opened to update the editableTopicSettings code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working good first issue Good for newcomers
Projects
None yet
Development

No branches or pull requests

5 participants