Support non-primitive roles on BigQuery datasets

[rolandkool]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment. If the issue is assigned to the "modular-magician" user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If the issue is assigned to a user, that user is claiming responsibility for the issue. If the issue is assigned to "hashibot", a community member has claimed the issue already.

Description

BigQuery datasets have an access property to configure permissions on datasets. Currently, terraform has three primitive roles hardcoded. However, recently it became possible to apply non-primitive roles, like custom roles, on datasets. Terraform should support this.

New or Affected Resource(s)

• google_bigquery_dataset

Potential Terraform Configuration

resource "google_bigquery_dataset" "dataset" {
  dataset_id  = "dataset"

  access {
    role = "roles/bigquery.viewer"
    user_by_email = "[email protected]"
  }
}

References

Maybe related to #2051

[danawillow]

Hi @rolandkool, I'm taking a look at the docs at https://cloud.google.com/bigquery/docs/access-control and it seems to me like the only ones that are applied at the dataset level are the primitive roles, and that others would be project or organization level IAM. This also aligns with the description in the API at https://cloud.google.com/bigquery/docs/reference/rest/v2/datasets:

The following legacy mappings will be applied: OWNER <=> roles/bigquery.dataOwner WRITER <=> roles/bigquery.dataEditor READER <=> roles/bigquery.dataViewer This field will accept any of the above formats, but will return only the legacy format. For example, if you set this field to "roles/bigquery.dataOwner", it will be returned back as "OWNER".

Are you seeing other docs that imply it can be done differently?

[rolandkool]

Hi @danawillow .
I haven't actually checked the documentation if it is in there somewhere. A colleague of mine discovered this and I'm not sure how he found out. Maybe just by trial and error.
I did test it myself using this patch and it works. The bq tool show the permissions as they are set by terraform.
Also, I don't think terraform should actually do this check. Isn't that the job of the API to validate this kind of input? Do resources for managing other iam resources (for buckets/pubsub/etc) validate the supported roles?
Thanks for taking a look at my PR.

[danawillow]

The difference between Terraform checking and the API checking is that Terraform can check at plan-time, which means if someone made a typo they can find out before they actually run Terraform, which is really helpful for large configurations with chained dependencies.

Using your patch, if you run Terraform with a non-primitive role set, and then run terraform plan again, does it say nothing to add or does it try to redo it because the API returned the primitive role back?

[hanfi]

actually using this json as source to bq update works fine :

  "access": [
    {
      "role": "WRITER", 
      "specialGroup": "projectWriters"
    }, 
    {
      "role": "OWNER", 
      "specialGroup": "projectOwners"
    }, 
    {
      "role": "OWNER", 
      "userByEmail": "[email protected]"
    }, 
    {
      "role": "READER", 
      "specialGroup": "projectReaders"
    }, 
    {
      "role": "projects/myproject/roles/CustomBigQueryDataViewer", 
      "userByEmail": "[email protected]"
    }
  ]
 ...

bq update --source this.json MY_DATASET

i get this :

I think that removing the string validation on roles will work perfectly : https://github.com/terraform-providers/terraform-provider-google/blob/master/google/resource_bigquery_dataset.go#L143

[hanfi]

pretty sure that removing the validation will work
here is a snippet of my terraform.state after

terraform import google_bigquery_dataset.default

"access.694453432.domain": "",
"access.694453432.group_by_email": "",
"access.694453432.role": "organizations/xxxxxxxxxx/roles/CustomBigQueryDataViewer",
"access.694453432.special_group": "",
"access.694453432.user_by_email": "[email protected]",
"access.694453432.view.#": "0",

[hanfi]

i was going to do the PR when i saw that @danawillow did it before me 😢
#3451

👍 good job

[danawillow]

Great, thanks for the info on custom roles! I'll comment again on the PR.

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. If you feel I made an error 🤖 🙉 , please reach out to my human friends 👉 [email protected]. Thanks!