Add resources to validate a domain

[toadjaune]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment. If the issue is assigned to the "modular-magician" user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If the issue is assigned to a user, that user is claiming responsibility for the issue. If the issue is assigned to "hashibot", a community member has claimed the issue already.

Description

Several operations on Google Cloud require you to validate domain ownership.
For example, creating a GCS bucket for static website hosting.
This is the usecase I met and the only one I know of, I suppose however there should be other affected resources.

There are currently no terraform resources for automating the validation process, nor to delegate domain ownership to a service account, which has several implications :

• You need to manually validate the domain before being able to automatically create some resources. This can be an important source of friction for the operator, especially if the validation relies on resources created by terraform (say, registering a domain …). You would then either need to first run your resource creation, have it fail in the middle, manually validate, then restart your resource creation. Or structuring it differently. Either way, there's a manual step and extra complexity.
• As far as I can tell, there is no way with standard tooling to validate domain ownership directly with a service account, meaning that if you want to run terraform from a service account you need to manually validate from an actual user account and then delegate ownership to this service account (using the old webmaster interface, the new one does not offer this option). And if your personal user account is ever disabled, this delegation apparently goes away too.

TL;DR : a lot of friction and manual work no matter what you do.

There is an API for automating site verification.
I don't know if it can be used when authenticated as a service account, but even if not, being able to automate this would be a great step forward.

New or Affected Resource(s)

AWS have several resources requiring similar validation processes, my API proposal is based on the one used by aws_ses_domain_identity_verification

• google_domain_owner_challenge
• google_domain_owner_validation
• google_domain_owner

Potential Terraform Configuration

# Request domain ownership.
# This resource returns a TXT record that you will need to publish in order for google
# to recognize you as an owner
resource "google_domain_owner_challenge" "example" {
  domain = "example.com"
}

# Publish the DNS record for the challenge
# This does not need to be google, any managed DNS would do the job
resource "google_dns_record_set" "challenge" {
  name = google_domain_owner_challenge.example.challenge.name
  type = "TXT"
  ttl  = 300

  # Of course, you would need to define this zone
  managed_zone = google_dns_managed_zone.example.name

  rrdatas = [google_domain_owner_challenge.example.challenge.value]
}

# This resource does not actually create anything on the google side.
# It's there to wait for the domain validation to be accepted by google
# (DNS propagation, etc…) before doing anything actually requiring
# the corresponding permissions
resource "google_domain_owner_validation" "example" {
  name = "example.com"
  depends_on = [google_dns_record_set.challenge]
}

# Create a bucket with the previously registered domain
# Please note the implicit dependency on the validation object
resource "google_storage_bucket" "static_front_test" {
  name = google_domain_owner_validation.example.name
  website {
    main_page_suffix = "index.html"
    not_found_page   = "index.html"
  }
}

# Delegate ownership to another user
# That could be a human administrator that needs to manage SEO in the search console…
# Please note the implicit dependency on the validation object
resource "google_domain_owner" "example_webmaster" {
  domain = google_domain_owner_validation.example.name
  user = "[email protected]"
}

References

Relevant documentation was referenced above

[hectorj]

Similar issue was closed before: #1724

But I think it would be a good addition to this provider!

In the meantime, you can use this: https://github.com/hectorj/terraform-provider-googlesiteverification

Edit: I'm interested in trying to contribute it to this repo. I'll take a shot at it soon-ish.
Edit2: I do not have much free time for this at the moment. If someone else feels like taking a shot at it, do not hesitate.

[mvanholsteijn]

@danawillow @hectorj and I are perfectly willing to assist in providing the required resources. If you feel it can be auto-generated, then that would be awesome. I do think it needs some work as the API is not orthogonal. ie. the create and update have a different signature.

[danawillow]

If you already have handwritten code for the resource, then it's not mandatory to do it generated, especially if it's unusual. The one thing I would be very careful about is authentication- our guideline right now is that you should be able to use the same authentication method for all your resources, and based on #1724 (comment) I'm not 100% sure you can.

[mvanholsteijn]

@hectorj actually programmed it to work properly; if you look at the code at https://github.com/hectorj/terraform-provider-googlesiteverification, you can specify JSON service account key to authenticate with and that works. the site verification API did not work with the default Oauth 2.0 credentials, I actually created a separate service account to verify the domain with. See:

• https://github.com/binxio/blog-how-to-automate-google-site-verification-with-terraform/blob/master/service-accounts.tf
• https://github.com/binxio/blog-how-to-automate-google-site-verification-with-terraform/blob/master/provider.tf

So this issue can be documented or even resolved in the resource definition.

[voycey]

This is absolutely required from an automation stand point and I am actually incredulous that this isn't standard in the provider or that the APIs aren't set up in such a way at GCP.

The above module doesn't work (at least not with latest versions of Terraform) and there is no documentation - please get this supported in the provider asap

[TrieBr]

At 2 years old and 44 upvotes, is there any progress on this or plans to add it? This totally breaks automation when creating GCS buckets. I managed to create a workaround which manually invokes the site verification APIs through http data requests. The I use the following module to verify a domain. The downside is that the access token appears in the logs, but using a short lived one (2s) should help.

variable "project_id" { type = string }
variable "dns_name" { type = string }
variable "managed_dns_zone" { type = string }
variable "service_account_email" { type = string }
// Create an access token.
data "google_service_account_access_token" "verification" {
  provider               = google
  target_service_account = "${var.service_account_email}"
  scopes                 = ["https://www.googleapis.com/auth/siteverification", "https://www.googleapis.com/auth/siteverification.verify_only"]
  lifetime               = "2s"
}

// Invokes the Site Verification API to get a token.
// https://developers.google.com/site-verification/v1/invoking#verify
data "http" "api_verification" {
  url = "https://www.googleapis.com/siteVerification/v1/token?access_token=${data.google_service_account_access_token.verification.access_token}"
  method = "POST"
  request_body = <<EOT
                {
                "verificationMethod": "DNS_TXT",
                "site": {
                  "identifier": "${var.dns_name}",
                  "type": "INET_DOMAIN"
                }
              }
              EOT

  # Optional request headers
  request_headers = {
    Content-Type = "application/json"
  }
  lifecycle {
    postcondition {
      condition     = contains([201, 204, 200], self.status_code)
      error_message = self.body
    }
  }
  depends_on = [
    data.google_service_account_access_token.verification
  ]
}

// Domain verification for static storage url.
resource "google_dns_record_set" "storage_static_verification" {
  project     = var.project_id
  name = var.dns_name
  type = "TXT"
  ttl  = 300
  managed_zone = var.managed_dns_zone
  rrdatas = [jsondecode(data.http.api_verification.response_body)["token"]]
  depends_on = [data.http.api_verification]
}


data "google_service_account_access_token" "verification_confirm" {
  provider               = google
  target_service_account = "${var.service_account_email}"
  scopes                 = ["https://www.googleapis.com/auth/siteverification", "https://www.googleapis.com/auth/siteverification.verify_only"]
  lifetime               = "2s"
}
// From https://developers.google.com/site-verification/v1/invoking#exampleInsert
data "http" "api_verification_insert" {
  url = "https://www.googleapis.com/siteVerification/v1/webResource?verificationMethod=DNS_TXT&access_token=${data.google_service_account_access_token.verification_confirm.access_token}"
  method = "POST"
  request_body = <<EOT
                {
                "site": {
                  "identifier": "${var.dns_name}",
                  "type": "INET_DOMAIN"
                }
              }
              EOT

  request_headers = {
    Content-Type = "application/json"
  }
  lifecycle {
    postcondition {
      condition     = contains([201, 204, 200], self.status_code)
      error_message = self.body
    }
  }
  depends_on = [
    data.google_service_account_access_token.verification_confirm
  ]
}

[sreid99]

Any chance this could be added ? Ideally with OpenID Connect as the authentication method.

[melinath]

I was able to confirm that this API can use the same OAuth 2.0 credentials that the Terraform provider operates on, so this seems like it would make sense in the google and google-beta providers.

[SirGitsalot]

👋

[andromaqui]

We are also facing the same issue currently - a fix would be much apppreciated

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

[SirGitsalot]

Note for future readers of this issue: this is now implemented.