pack build could run faster

[dgageot]

Hi, I have a prototype of pack build that always runs 8 seconds faster on my mac, using docker desktop. 8s is not a huge difference for a full build. But for an incremental build that pulls most of the things from cache, it really changes my perception.

Basically, instead of running 6 times docker run and ephemeral volumes, I use one docker run and one docker exec.

Would you be interested in this change?

[jromero]

@ekcasey / @sclevine ☝️

I know we've talked about performance a lot lately. This seems to be right up that alley.

[jromero]

I'm curious as to how this plays out with the overall architecture and the impact on aspects of the building process. One of the reasons for running 6 separate containers would be for isolation during phases.

See: https://github.com/buildpack/spec/blob/master/platform.md#security-considerations

The spec mentions SHOULD instead of MUST meaning that we could probably make this improvement but not sure we'd want to take on the implications and it's probably not the best practice, therefor not the best example of a reference implementation.

@ekcasey and @sclevine might have more input or differing opinions.

[ekcasey]

@dgageot One of the reasons for running the lifecycle phases in different containers is for credential isolation. We only provide registry credentials to analyze and exporter contains so that arbitrary buildpacks do not have access these credentials. In addition, when running w/o the --publish flag analyze and export containers run as root so they can access the mounted docker daemon socket.

I have actually been considering whether restore/analyzer/export/cache should run in a containers based off of an entirely different image for further isolation, b/c unlike detect and build there is no need to access buildpacks and no need to run on the proper stack when executing these phases. This would prevent unvetted builder images from stealing credentials.

[squee1945]

cc @lukasberger (see above comment, just FYI)

[natalieparellano]

This proposal is currently being fleshed out in buildpacks/rfcs#46 and we expect subsequent work to resolve this issue.

Blocked by buildpacks/rfcs#46 & buildpacks/lifecycle#243

[dfreilich]

👍 to running faster!

I think that the work on using trusted-builders, and thereby reducing the number of containers, helped with this particular point, but we should definitely be optimizing speed, as #632 and #652 point out. Given the trusted-builder point, though, I'll close this for now.