BQ Data Transfer Configs stuck on "Updating"

[AlexandruAG6]

Checklist

• I did not find a related open issue.
• I did not find a solution in the troubleshooting guide: (https://cloud.google.com/config-connector/docs/troubleshooting)
• If this issue is time-sensitive, I have submitted a corresponding issue with GCP support.

Bug Description

Scheduled Queries created via BQ Data Transfer Configs keep the KCC object stuck in "Updating", while a new Scheduled Query is created in GCP each update cycle.

Additional Diagnostic Information

I'm deploying a BQ Data Transfer Config via KCC, by applying a k8s manifest via Terraform. The Data Transfer Config is used to set up a Scheduled Query. The object is created in Kubernetes, and the actual Scheduled Query is created in GCP, but the k8s object is stuck continuously in the "Updating" state, and with each update cycle, a new Scheduled Query is created in GCP. The k8s object does not seem to be aware that the Scheduled Query is actually created, and will keep trying to create new ones.

Kubernetes Cluster Version

Server Version: v1.27.8-gke.1067004

Config Connector Version

1.113.0

Config Connector Mode

namespaced mode (default)

Log Output

No response

Steps to reproduce the issue

1. deploy a BigQuery Scheduled Query via KCC
2. check the status/count of the Scheduled Queries in GCP, and the status of the KCC object

YAML snippets

apiVersion: bigquerydatatransfer.cnrm.cloud.google.com/v1alpha1
kind: BigQueryDataTransferConfig
metadata:
  annotations:
    cnrm.cloud.google.com/management-conflict-prevention-policy: none
    cnrm.cloud.google.com/state-into-spec: merge
  creationTimestamp: "2024-03-26T09:54:58Z"
  finalizers:
  - cnrm.cloud.google.com/finalizer
  - cnrm.cloud.google.com/deletion-defender
  generation: 1
  name: tstquery9
  namespace: placeholder
  resourceVersion: "223432102"
  uid: aa945d85-6014-465b-bcd6-a99a17f98ac9
spec:
  dataSourceId: scheduled_query
  destinationDatasetId: test
  displayName: tstquery9
  location: EU
  params:
    destination_table_name_template: tsttable3
    query: SELECT qtr,nonrep FROM `project-placeholder.test.tsttable`
    write_disposition: WRITE_APPEND
  projectRef:
    name: project-placeholder
  schedule: every monday 08:00

[jingyih]

/assign

[jingyih]

This is an alpha resource. The request is to stabilize them and support them (i.e. move to beta).

[jingyih]

Scheduled Queries created via BQ Data Transfer Configs keep the KCC object stuck in "Updating", while a new Scheduled Query is created in GCP each update cycle.

We implemented direct cloud reconciler for this resource in Config Connector 1.123. This issue should be fixed.