Async PV for workspace: enabling feature (toggle)

[benoitf]

Enable feature

subtask of #15384

By default workspaces are always using a PVC (or emptyDir for ephemeral workspaces)

In the same approach used for ephemeral we could add a new attribute of the workspace definition

attributes:
  asyncPersist: true

attribute name can be changed.

By using that toggle, it means that a new container will be added to the workspace/pod which will do the sync. And of course, it will use emptyDir as well

It will allow to have this feature enabled per workspace to try how it works and be able to compare, etc.

[ibuziuk]

Proposed UD design - an experimental checkbox that could be enabled only when temporary storage toggle is on:

[ibuziuk]

cc @beaumorley any comments ^

[benoitf]

if you check async but you disable temporary storage IMHO it doesn't work as well so this is why I was thinking of a dropdown

[ibuziuk]

@benoitf my idea was that checkbox can only be enabled (maybe even visible) only if the temporary toggle is on

[ibuziuk]

After the discussion during prio call it looks like it would be better to opt for dropdown:

[vparfonov]

Demo: https://youtu.be/EU_4M1ZHELw

[ibuziuk]

@vparfonov we have a separate issue for UD customizations:

• Finalize Storage Type switcher #17230
• Align Get Started and Create Custom Workspace pages in propagating persistVolumes attribute #17170
  I believe we can close this one once Prepare environment for async storage of project sources #16972 is merged