feature request: more control over the docker build context

[majelbstoat]

I have a monorepo with shared code. Because docker doesn't follow symlinks and never will, I have to copy that shared code into the right place before building the image for each service. There's currently no facility for doing that with skaffold. I'm using VSCode's Cloud Code extension, so running a wrapper script isn't straightforward, but in any case, I'd like the build to be fully described in the definition, not be dependent on some external glue.

#1441 describes one approach to solving this, which would be great, but I don't need the full flexibility of a custom script.

What I'd like to see is the ability to control which parts of the context are put into the tarball that is sent to Docker. Being able to specify paths, and mappings would be sufficient for my use-case. For example:

context:
  root: .
  paths:
    - mapRef:
        from: services/mercury/src
        to: .
    - mapRef:
        from: shared/nodejs/src
        to: ./lib

This would create a tarball that had files from mercury at the root, and files from the shared source at ./lib. Ideally, it would do this without actually copying the shared files into the mercury service in the local file system.

The absence of the path key would cause behaviour as currently is, i.e. everything beneath the context.

[corneliusweig]

Would it be an option to create hard links instead of soft links? This should work with docker, however it's probably not feasible if you need to sync whole directories.

[majelbstoat]

Yeah, hard links cannot be created for directories, so unfortunately not.

[corneliusweig]

Yeah, hard links cannot be created for directories, so unfortunately not.

So recursively creating hard links is also not an option?

cp -rl from to

[majelbstoat]

The code is shared to multiple different locations in the monorepo, and the benefit of symlinks is whenever a new shared library is added, it is immediately available to all the services. Running a script each time a new file is added complicates the developer workflow. I already have a hacky script to glue bits and pieces together, replacing it with another isn't an improvement :)

(With the rise of lerna et al, this is not an uncommon pattern, btw. Most organisations with a monorepo and micro services will probably come across some form of this.)

[balopat]

I think it would be interesting to provide support for a flexible Docker context creation.
Skaffold could assemble the tar file based on the context spec and pipe it to docker build.
It is big enough though that I think a good design proposal would be beneficial before someone jumps in to implement it.

[tstromberg]

Thank you for the idea!

I'm closing this issue as it's been open a while, and it is not clear if it's still an open issue. No one has recently stated an interest in addressing this, but if you feel strongly about it, please feel free to add a comment or send us a PR!

[majelbstoat]

I mean, it's still an issue, and it's the primary reason why I didn't end up using skaffold. That might not be enough to reopen it though, which is fine if no-one else wants this 🙂

[CaffeineDaemon]

I am also interested in this feature, and the addtion of
https://www.docker.com/blog/dockerfiles-now-support-multiple-build-contexts/
should make this a lot easier, though it needs buildx to be installed.

As i understand it, you woud just need to pass the names and paths of the buildcontexts as addtional arguments to docker build, which maybe could be done with cliflags, but i dont know how to make skaffold watch those extra directorys for changes, so direct support through the skaffold cli would be very helpful.

To specify the additional buildcontexts in yaml, i would suggest the following:

build:
  artifacts:
    - image: gcr.io/k8s-skaffold/example
      context: .
      docker:
        additionalContexts:
          someExtraContext: ressources/context
          someMoreContext: src/someSharedProject

As this is currently only available through docker buildx, a option is needed that changes the build command skaffold calls, so something similiar to useBuildKit: true.

I would make a PR, but i am no go developer and have no idea where i would have to start making changes :(

[MarlonGamez]

hey @CaffeineDaemon , thanks for giving some more context and insight into your use case here. I'm going to reopen this, however, our team doesn't have the bandwidth to implement this right now. Maybe someone else who is interested will be able to pick this up and work on it if they're willing to write up a quick design proposal for us to look at