[CC-26984] metric-export: add resource for prometheus integration

[aa-joshi]

This change introduces resource MetricExportPrometheusConfig which manages prometheus metric export integration in gcp and aws cloud providers.

Commit checklist

• Changelog
• Doc gen (make generate)
• Integration test(s)
• Acceptance test(s)
• Example(s)

[fantapop]

@aa-joshi @marksoper considering that there is no actual configuration for enabling prometheus what you do think of having it as an additional field on the cluster resource? It could be something like:

resource "cockroach_cluster" "dedicated" {
  name           = "cockroach-dedicated"
  cloud_provider = "GCP"
  prometheus_metric_export_enabled = true
  dedicated = {
  ....
  }

If we were planning to add configuration options to this at some point in the near future, it seems like keeping it as its own resource would be desirable but even in that case we could just have that resource just represent the configuration and keep the enabled / disabled flag on cluster.

For example,

resource "cockroach_cluster" "dedicated" {
  name           = "cockroach-dedicated"
  cloud_provider = "GCP"
  prometheus_metric_export_enabled = true
  dedicated = {
  ....
  }
resource "cockroach_cluster_prometheus_export_config" "dedicated" {
  cluster_id = cockroach_cluster.dedicated.id
  some_config_option = 1234
}

[fantapop]

I guess another consideration is whether we'd expose some of the outputs as part of the resource. I see that the api response has the following data:

PrometheusMetricExportInfo
{
    cluster_id*: string
    status: enum  // Allowed: NOT_DEPLOYED┃DISABLING┃ENABLING┃ENABLED┃ERROR
    targets: {   // targets is a map of ports exposing metrics to regions.
        [any-key]: string
    }
    user_message: string
}

[aa-joshi]

@aa-joshi @marksoper considering that there is no actual configuration for enabling prometheus what you do think of having it as an additional field on the cluster resource? It could be something like:

resource "cockroach_cluster" "dedicated" {
  name           = "cockroach-dedicated"
  cloud_provider = "GCP"
  prometheus_metric_export_enabled = true
  dedicated = {
  ....
  }

If we were planning to add configuration options to this at some point in the near future, it seems like keeping it as its own resource would be desirable but even in that case we could just have that resource just represent the configuration and keep the enabled / disabled flag on cluster.

For example,

resource "cockroach_cluster" "dedicated" {
  name           = "cockroach-dedicated"
  cloud_provider = "GCP"
  prometheus_metric_export_enabled = true
  dedicated = {
  ....
  }
resource "cockroach_cluster_prometheus_export_config" "dedicated" {
  cluster_id = cockroach_cluster.dedicated.id
  some_config_option = 1234
}

We have existing metric export integration which has their own resources:

• cockroach_metric_export_datadog_config
• cockroach_metric_export_cloudwatch_config

Hence we would like to keep it consistent with other metric export integrations. I will create a seperate backlog item to track integrations against the cluster resource.

[aa-joshi]

I guess another consideration is whether we'd expose some of the outputs as part of the resource. I see that the api response has the following data:

PrometheusMetricExportInfo
{
    cluster_id*: string
    status: enum  // Allowed: NOT_DEPLOYED┃DISABLING┃ENABLING┃ENABLED┃ERROR
    targets: {   // targets is a map of ports exposing metrics to regions.
        [any-key]: string
    }
    user_message: string
}

Thanks for suggestion! I have made required changes to keep track of targets.

[fantapop]

Hence we would like to keep it consistent with other metric export integrations. I will create a seperate backlog item to track integrations against the cluster resource.

For future consideration, I think should always re-evaluate the best way to expose the api for any new resources. Existing resource patterns is of course valuable input into this decision but not the only consideration. In this case, since we want to expose the additional outputs, I agree that it should be a new resource.

[fantapop]

Hey, thanks for taking this on. I left some feedback for you.

[arjunmahishi]

Went over the changes in a pair programming session (learnt a lot about terraform lifecycle today!). This LGTM.
Will wait for @fantapop's ✅

[fantapop]

This is a looking a lot cleaner. Thanks for your work here. I added some followup comments and some that I missed earlier.

[arjunmahishi]

@fantapop Can you please give it one final look?
Thanks!

[arjunmahishi]

@fantapop we have made the requested changes

[fantapop]

I've added some additional small nits to the review. Please take a look at these before merging. Also, please rebase off the latest main branch. Thanks for everyone's work on this!