Temporary files are not deleted after build

[matejvasek]

Summary

Temporary files are not deleted (nor reused) after build when using with podman.

The podman doesn't support "sparse" tarballs, so pack has to download builder layers to temp.
The issue is that the files are not deleted after build finished.

Even more so when pack build is run repeatedly multiple identical temporary directories occupy temp filesystem.
This is particularly problematic when /tmp is in RAM.

I think pack should one of the following:

• delete the temp directory right after build
• do not delete the directory after build and reuse it for subsequent builds

[matejvasek]

Seems reusing might be against https://refspecs.linuxfoundation.org/FHS_2.3/fhs-2.3.html#TMPTEMPORARYFILES
Programs must not assume that any files or directories in /tmp are preserved between invocations of the program.

[dwillist]

Comments from Platform Sync.
Needs some investigation around what layers are not getting cleaned up here.

• Are these from the ephemeral builder?
• Are these a lifecycle concern?
  Some additional investigation is needed to figure out which layers are being left around.

[matejvasek]

@dwillist I think this is related to ephemeral builder. The thing is that pack uses clever hack to speed up build. It pushes only newly create layers (they are small) for ephemeral builder and it leaves out some rather large layers that are already present in daemon.

While such an archive missing layer is not 100% valid docker happily accepts it provided the missing layers are already in it. This is not the case for podman which is less permissive (podman is alternative docker implementation).

Some changes had to be done to fix it #925, buildpacks/imgutil#81.

podman works now, however the fix leads to accumulation of temp files.

[jjbustamante]

@matejvasek

Could you test this with the latest pack version? I tried to reproduce it today and after building a sample java app, I checked my tmp folder and I didn't see anything there. I am running pack on a arm64 machine using podman