[PVC] the prebuild workspace is unable to be up and running

[jenting]

Bug description

The prebuild workspace is unable to be up and running.

It's the limitation of the GCP regional disk. We can't create a new zonal persistent disk from the same snapshot in the same zone, more than once in 10 minutes. https://cloud.google.com/compute/docs/troubleshooting/troubleshooting-snapshots#creating_new_zonal_persistent_disks_from_snapshots

Steps to reproduce

Enable the projects prebuild PVC

https://gitpod.slack.com/archives/C02F19UUW6S/p1666105041014279

Workspace affected

No response

Expected behavior

The prebuild workspace should be up and running.

Example repository

No response

Anything else?

#7901

[kylos101]

@jenting can you pause on this issue and its PRs? Let's wait to see if Google can remove the snapshot/volume limit. I ask because we may proceed and do PVC just for regular workspaces.

cc: @sagor999

[kylos101]

@jenting adding blocked label for the meantime, I recommend looking at alternative groundwork for the meantime. @sagor999 if you need any help from @jenting let him know? 🙏

[jenting]

Internal discussion https://www.notion.so/gitpod/Next-steps-for-PVC-given-snapshot-volume-limits-593d5d804e734f459a8935208b57903b

[jenting]

For prebuild workspaces, we'll use GCP image to backup prebuild workspace PVC. Ref to https://www.notion.so/gitpod/Next-steps-for-PVC-given-snapshot-volume-limits-593d5d804e734f459a8935208b57903b.
For regular workspace, we'll use GCP snapshot to backup regular workspace PVC.

[jenting]

PR both merged, waiting for the deployment, and closing it.