add required permissions for uploading image to a given bucket

[msaligane]

When uploading the built runner image for GCP (command shown below)

# Upload the resulting tar archive
./upload_gcp_image.sh $PROJECT $BUCKET

The user encounters an error that is relevant to missing bucket permissions.
This PR translates manual settings in the GCloud console to portable commands in Linux
The following demonstrates the missing steps to generate the appropriate permissions for the service account that is uploading the image

1. Save the bucket's IAM policy to a temporary (arbitrary) JSON file

gsutil iam get gs://$BUCKET > /arbitrary/path/file.json

2. Get the project name and default service account email address. Adjust filter accordingly if a different service account is used

export PROJECT=$(gcloud config get-value project)
export SA=$(gcloud iam service-accounts list --filter=default | grep -E -o '[a-z0-9._%+-]+@[a-z0-9.-]+(\.[a-z0-9._%+-]+)?[a-z]{2,4}')

3. Get the absolute path of the Bucket config file

export BUCKET_FILE=/arbitrary/path/file.json

4. Using the sed utility to insert required permissions associated with the bucket

sed -i 's/"bindings": \[/"bindings": \[\
    {\
      "members": \[\
        "projectEditor:'"$PROJECT"'",\
        "projectOwner:'"$PROJECT"'",\
        "serviceAccount:'"$SA"'"\
      \],\
      "role": "roles\/storage.legacyBucketOwner"\
    \},/' $BUCKET_FILE

5. Upload the modified bucket file back to GCloud

gsutil iam set $BUCKET_FILE gs://$BUCKET

FYI @erictaur

[msaligane]

@kgugala and @mithro Could you please take a look at this PR?

[AdamOlech]

Hey @msaligane, thanks for the PR.

I tried reproducing your issue with permissions but I was unable to.

After creating a GCP bucket using gsutil mb gs://$BUCKET I am able to upload files using gsutil cp example.txt $BUCKET.

This is because after creating the bucket, it automatically gets the Storage Legacy Bucket Owner role assigned to the editors and owners of the project (which is the underlying assumption of README). Therefore, no changes need to be made to the permissions of the bucket

Again, we're making the assertion that you have either editor or owner permissions in the project (which you will end up having if you're the one creating the project and that is the flow that is described in README).

Regarding the export SA=$(...) command -- I'm not sure if assigning the roles/storage.legacyBucketOwner role to the default compute engine account is necessary (I would actually argue that this is potentially insecure -- by doing that, you'd essentially give the implicit permission to manage the bucket to all compute instances within the project).

It is important to note that any inability to perform operations on the storage bucket would result from your personal account having insufficient permissions (and not the default Compute Engine account), because that's what the gcloud utility is authenticating with.

[msaligane]

Hi @AdamOlech

Thanks for the review. My understanding on the usage of service accounts (SA), we should be using the SA associated to the VM instance. In other words, Google does not recommended using the personal account when working in VM instances. Does this make sense?
In our case, the SA (which is the default SA) associated to the VM used to set up the runner did not have the legacyBucketOwner role.
As an alternative, we can create a SA that has limited permissions and assign that particular SA to instances that are relevant to the runner.

Thanks!

[AdamOlech]

My understanding on the usage of service accounts (SA), we should be using the SA associated to the VM instance. In other words, Google does not recommended using the personal account when working in VM instances. Does this make sense?

If you're trying to setup the runner from a Compute Engine instance then yes, it makes sense. The way we usually do this over at Antmicro (and such assumption is reflected in README), is we interact with the Compute Engine using personal developer machines. Because of that, we use our personal accounts to authenticate with gcloud.

As an alternative, we can create a SA that has limited permissions and assign that particular SA to instances that are relevant to the runner.

Yes, I think this sounds better than assigning permissions to the default instance account. To reiterate, doing so would result in all instances using the default service account (and any new instance that is created uses it unless specified otherwise) being able to perform read/write bucket operations within the project.