Improve remote buildkit and proxy environment variables

[tavlima]

I am investigating the use of remote buildkit instances to offload the image building process from our CI/CD instances and got really confusing by what seems to be the expected behavior, regarding proxy environment variables.

It seems that the daemon environment variables are inherited only by directives like FROM, but as @tonistiigi pointed out, the internal containers created by RUN steps need those environment variables to be explicitly provided through --build-arg.

IMHO, it seems really odd that we need to have the same configuration in both places (daemon env and --build-arg) in order to build single image (assuming proxies are required for whatever image/command is used in both FROM and RUN steps).

My team is expected to maintain a cluster with tens of buildkit instances, and I'd rather not have our whole org worrying about what is the right proxy configuration to fetch image XYZ, especially because the environment running our buildkit cluster requires dozens of CIDRs and wildcard domains to be included in the NO_PROXY envvar.

Alternatives I can think of:

1. completely abstracting that away from the clients

This would be ideal, IMHO, like allowing RUN steps to also inherit the proxy envvars from the daemon env. Such behavior should probably be gated by a new flag in the buildkitd.toml, disabled by default.

2. do this setup just once, like when "registering" the remote builder in each client

This would definitely be an improvement, perhaps using some new --driver-opt to set default --build-arg, somewhat similar to docker/buildx#170.

3. preserve the current state of affairs

Having to include --build-arg for all proxy envvars, every time someone wants to use our remote builder cluster? That is already proving itself to be a hard sell, for my team.

Original discussion starts here.