proposal: support for COPY that uses git ls-files as source

[myitcv]

Apologies if this is the wrong place to be raising this issue, but I figured given the experiments discussed in https://github.com/moby/buildkit/blob/master/frontend/dockerfile/docs/experimental.md it might be the best place.

In my limited experience of creating Dockerfiles, I find myself writing .dockerignore files that are basically the same as .gitignore, but with the addition of /.git.

Is there mileage therefore in supporting a mode by which the source of COPY is taken from git ls-files, or more generally the output of some command that is then passed as context?

Thanks

[quangkieu]

I think you could use context in build command from github commit hash snapshot

[myitcv]

I think you could use context in build command from github commit hash snapshot

I'm not sure I fully understand your point/suggestion. Can you explain more with an example?

This issue could potentially be retitled as a more generic "allow the user to specify arbitrary context" (not sure context is the right word here) but I presented a specific example to start the discussion.

There are other cases where having environment variables as the only means of passing context falls short, e.g. #1512 (comment). Being able to "inject" the result of arbitrary commands would allow the user more flexibility/control, with the risk being entirely passed onto them in terms of adverse side effects etc.

[tonistiigi]

You can build from a git repository, passing the repo address instead of the local dir.

Possibility for doing the same for local repos without pushing (but still probably committing) was discussed as a follow-up for #1048 once it makes some progress.

[myitcv]

Thanks @tonistiigi. Yes, I'm looking to do this from a local git repo as opposed to a remote one (although I can see others might well require the use case of remote repos).

I think I roughly see the direction in which #1048 is heading but I believe it's slightly different to what I'm proposing/need here (talking specifically about the security concerns of using git). I'm very likely lacking a lot of context (apologies, really overloading that term now!), but at least for my simple use case I don't really require any new primitives. COPY does what I need, I'm simply looking to parameterise my use of it with the results of a command invocation, in this case git ls-files. By definition, I would expect that command (git ls-files) to be run outside of a container, using the git command that I use on my system. It effectively represents shorthand for my supplying the list of src files/directories manually to the COPY command by hand.

My understanding of #1048 is that it looks to solve a slightly different problem more aligned with reproducibility of builds, hence the focus on running git (or whatever) within a container, thereby taking the user's system versions, setup etc largely out of the equation.

Would appreciate any guidance if I'm simply holding things wrong!