If project-clone container is killed while cloning, repos are left in inconsistent state

[amisevsk]

Description

The project-clone container checks if directories exist for projects in the devfile, and does nothing if they do. This is done to avoid modifying repos on subsequent startups in case local changes have been made, but also means that if the project clone container fails during cloning/unzipping, the repos are left in an inconsistent state.

To avoid this, we should try to make the project-clone container act as atomically as possible, so that it either does nothing, or it sets up projects successfully.

Additional context

Originally reported as "problem 7" here: eclipse-che/che#20785. In this example, the project clone container is restarted because a slow OpenShift cluster adds an imagePullSecret to the workspace serviceaccount after creation, triggering an update to the workspace deployment (and thus the creation of a new pod)