Created dockerized workflow for building protobufs. #734
Conversation
Signed-off-by: Cody Littley <[email protected]>
Signed-off-by: Cody Littley <[email protected]>
api/builder/Dockerfile
Outdated
| @@ -0,0 +1,38 @@ | |||
| FROM debian | |||
There was a problem hiding this comment.
Which version of debian does this actually use? For consistency reasons we should pin to a specific release. Also, as noted in lightnode docker PR, I recommend using distroless which will make the docker build faster and significantly smaller.
There was a problem hiding this comment.
Being more explicit about the exact starting image definitely makes sense. I've selected FROM golang:1.21.12-bookworm.
Working on a multi-layer docker image to reduce overall size. Will report back.
There was a problem hiding this comment.
I'm encountering difficulty trying to get this working on top of distroless. The core of the issue is that the tools required to build the protobuffs have a bunch of various dependencies, and it's notrivial to copy those all over to a clean image. Unlike the example in smallest-golang-docker-image, this is not a container where we are just trying to deploy a simple executable. We need our entire build toolchain, and that isn't packaged in a way that makes it simple to copy over.
The size of the dockerfile built on top of golang:1.21.12-bookworm is ~1gb. The size of a distroless image that just contains the go and protoc dependencies is ~0.33gb. Decently smaller, but not orders of magnitude smaller. Since this is something in the build pipeline and not something that we are going to need to ship to customers (unless they want to compile from scratch), IMO reducing the size of the container is not urgent.
Based on this information, would you be ok proceeding with this image as I currently have it?
There was a problem hiding this comment.
I think that given we are currently using this as a utility, with the committed protobufs remaining the standard of truth, I'm happy to not spend more time trying to fine-tune our choice of an image. @pschork, please let us know if you have additional thoughts.
There was a problem hiding this comment.
Can we also update the root Makefile:protoc target to use this new build approach?
I plan to do this, but I am intentionally not doing it as a part of this PR. Changing the build process for everybody has the potential of being disruptive to CI pipelines and manual workflows, and I'd rather split high risk changes like that into their own PRs.
Signed-off-by: Cody Littley <[email protected]>
Signed-off-by: Cody Littley <[email protected]>
Why are these changes needed?
Creates a dockerized workflow for building protobufs.
Checks