Parallel safe local directory build

[gaborbernat]

pip 20.1 now builds local directories in-place via #7882. One side effect (and regression) of this is that now pip install/pip wheel is no longer parallel safe. Two pips running in parallel installing/building the same folder can now step on each other's toes by modifying in parallel the build backends cache folders. E.g. one starts, but another starts later and cleans the build folder while the first is generating the wheel folder.

As a stop-gap solution pip could lock the local folder while building, and don't start a new build while that lock is in place.

Ideally, pip should set a different build folder for each of these, that's throw away at the end of the build. This goes back to my proposal from last year about the frontend being able to specify the backend the cache folder, see https://discuss.python.org/t/proposal-adding-a-persistent-cache-directory-to-pep-517-hooks/2303/9 . Guess I should actually write up that PEP now.

PS. This broke the virtualenv CI, is how I bumped into this issue.

[uranusjr]

See also #8165. It would be best if PEP 517 can add a way to mandate the backend to perform an out-of-tree build, and your proposal would be a good way to do that IMO.