Skip to content

Commit

Permalink
added prepare_backup_projects_for_terraform.sh and update README
Browse files Browse the repository at this point in the history
  • Loading branch information
kwadie committed Feb 27, 2023
1 parent 52a26ba commit bd25a86
Show file tree
Hide file tree
Showing 2 changed files with 47 additions and 2 deletions.
17 changes: 15 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -415,7 +415,9 @@ BigQuery Types to Avro Logical Types mapping:
| `TIME` | `timestamp-micro` (annotates Avro `LONG`) |
| `DATETIME` | `STRING` (custom named logical type `datetime`) |

##### Configure Additional Backup Projects
##### Configure Backup Projects

###### Additional Backup Projects

Terraform needs to deploy resources to the backup projects where the backup operations will run. For example, log
sinks that send notifications to the Tagger once a backup operation has completed.
Expand All @@ -430,6 +432,18 @@ additional_backup_projects = ["project1", "project2", ..]

If you're only using the fallback backup policy and without table-level external policies, you can set this variable to an empty list `[]`

###### Configure Terraform SA permissions on Backup Projects

In order for Terraform to deploy resources on the backup projects (configured in the previous step), the service account
used by Terraform must have the required permissions on these projects. To do so, run the following command:

```shell
./scripts/prepare_backup_projects_for_terraform.sh <project1> <project2> <etc>
```

The list of projects must include all projects you're planning to store backups in. This includes all projects listed under the `backup_project`
field in the fallback policy, plus the ones included in the `additional_backup_projects` Terraform variable.

#### Terraform Deployment

```shell
Expand All @@ -446,7 +460,6 @@ terraform apply -var-file=$VARS -auto-approve

```


#### Setup Access to Sources and Destinations

##### Set Environment Variables
Expand Down
32 changes: 32 additions & 0 deletions scripts/prepare_backup_projects_for_terraform.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
#!/bin/bash

#
# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#

# run this script for all projects that are used to store backups
set -e

for project in "$@"
do

echo "Preparing backup project ${project} for Terraform .."

# Terraform needs to create log sinks to capture GCS export operation completion
gcloud projects add-iam-policy-binding "${project}" \
--member="serviceAccount:${TF_SA}@${PROJECT_ID}.iam.gserviceaccount.com" \
--role="roles/logging.configWriter"

done

0 comments on commit bd25a86

Please sign in to comment.