Proposal: ONBUILD image

[ahmetb]

For those who are not familiar, Dockerfiles can have an instruction called ONBUILD which invokes the preceding command when the image is used as a base image for building another image (typically app image).

Right now simplest ASP.NET application image would look like this:

FROM microsoft/aspnet
COPY . /app
WORKDIR /app
RUN kpm restore
CMD ["k", "kestrel"]

With the help of ONBUILD directives, we can simplify this Dockerfile as short as this:

FROM microsoft/aspnet:onbuild

and that would be the only Dockerfile to build an ASP.NET app image and ship it to the Docker host. Relevant work is suggested by @friism (#11) and @vlesierse (#7) in the pull requests. Therefore I am starting a discussion about possible risks and implementation details.

Risks of breaking users

The dockerfile written above is almost the same for all web applications.

1. It copies app to /app (or /usr/src/app open to debase, but ok)
2. Goes to /app and runs kpm restore (always needed?)
3. Runs k executable with kestrel parameter.

Since onbuild image will bring quite some easiness, if we happen to change the path (i.e. /app) and people take dependency on that, we will be breaking the apps built with the -onbuild image.

Also, if kpm restore takes any parameter (e.g. feed), unfortunately we won't be able to pass that to the ONBUILD RUN kpm restore. So it will stay as bare kpm restore. Does this handle the 90% use case?

It's ok to break users now since this image is still in preview, however in the long term this will not be feasible, therefore we should be coming up with a solution that can stabilize and get adopted by the community over time.

Maintenance

Along with any version (e.g. 1.0.0-beta1) we need to maintain an -onbuild Dockerfile equivalent as well (e.g. 1.0.0-beta1-onbuild). That's not a big cost, I have no concerns in this area. (example)

Entrypoint

Right now it appears like k kestrel is the de facto entrypoint for the web server containers. However I do not want to leave the k run (command line app) use case out of the onbuild images. The both entrypoint common in all use cases is k itself and the rest can be changed.

So in the onbuild image, if we specify the entrypoint as:

ENTRYPOINT ["k"]
CMD ["kestrel"]

• In the simplest Dockerfile (just contains FROM .... above the entrypoint will be k kestrel.

• If users need to override the k parameter in their app image, they can use:
  FROM microsoft/aspnet
  CMD ["run"]
  and the entrypoint will be overriden as k run.

• If users need to override the entrypoint entirely, they can use ENTRYPOINT directive in their app's Dockerfile.

• Passing parameters: by overriding the CMD I can pass extra parameters to the k command (e.g.k run --param foo) in Docker cli from an onbuild image like:

  docker run -i -t myapp run --param foo

This will result in a cmd like ["k", "run", "--param", "foo"] which is valid.

Tagging onbuild images

An onbuild image for a normal image will just have -onbuild suffix in the Docker image tag. (e.g. microsoft/aspnet:1.0.0 will have corresponding microsoft/aspnet:1.0.0-onbuild image).

An ONBUILD dockerfile for a version should be stored at version/onbuild/Dockerfile (e.g. onbuild Dockerfile for 1.0.0/Dockerfile should be at 1.0.0/onbuild/Dockerfile). This will keep things organized at the root level view on the repo.

This also appears to be the convention.

The ONBUILD image for latest image will just be tagged as onbuild and will be referenced as microsoft/aspnet:onbuild.

Release steps

Once the PR is merged, we'll push the onbuild image for latest(=1.0.0-beta1) immediately and for the next version releases, ONBUILD image has to be included as a part of the version update PR. The README.md should also be updated to reflect the changes in the tags.

Once changes are merged administrators must update the automated build configuration to point latest/onbuild tags to the correct version.

Going forward

@vlesierse's PR #7 uses ENTRYPOINT and CMD separately, which makes sense in the description above.

I really appreciate input from you guys on going forward. Do you see any problems with the proposal above or do you see additional issues/concerns we should be aware of? At this point iterating on @vlesierse's PR #7 looks like a better choice for me.

/cc: @davidfowl @johngossman

[vlesierse]

This proposal looks great.

I see the onbuild addition to a tag at other images as well. Also some images has put this solution in the naming like microsoft/aspnet-runtime which I personally don't like.

With this proposal you currently end up with the following onbuild images:

• microsoft/aspnet:onbuild
• microsoft/aspnet:1.0.0-beta1-onbuild

The downside I'm currently facing is when the application is part of a solution. It is possible to run kpm restore in the solution folder but executing the k command requires the WORKDIR been set to the project folder. Maybe I'm taking this too far, but it could be an addition for the global.json to define the startup project.

Having the ENTRYPOINT set to [ "k" ]works great together with the default command (which can be either kestrel or run). This makes the command for running the container more natural.

[friism]

@ahmetalpbalkan This looks good to me. I recommend /usr/src/app as the location since that's what the other language stacks seem to default to. It's also where the base Mono onbuild image places things: https://github.com/mono/docker/blob/master/3.10.0/onbuild/Dockerfile

Great work! Looking forward to retiring my image :-)

[ahmetb]

@vlesierse great point about solutions. I have no idea how kpm restore works for multi-project solutions. So is it enough to run kpm restore where entrypoint project's project.json is or should it be ran at an upper level like global.json? That's really not my expertise. @davidfowl we may need your input if really a feature like "startup project" needs to be added to global.json etc.

I'll leave it up to you folks. Are the multi-project solutions the common case in ASP.NET ecosystem? If so, we must shape the onbuild image accordingly.

[Eilon]

Multi-project solutions are very common; however, generally just one project is deployed because you use kpm pack to package up the app into a deployable/runnable form. So it depends on the scenario. If you're in progress developing things then you run multiple projects off of sources. On a production site you generally pre-compile and pack everything first.

[vlesierse]

I believe Docker is mainly used in a production environment and pre-compiling makes sense in that case. If I'm correct kpm pack on Beta 1 doesn't produce a shell script for Linux yet. I'm also wondering if there is a noticeable difference in terms of performance which make pre-compiling the application using kpm pack necessary.

[Eilon]

@vlesierse startup time is mostly what's affected by precompilation. Not having to load Roslyn, read 100's of *.cs files and *.cshtml files from disk, parsing and compiling them all, etc. can be a huge saving. (In a trivial app or other very specific app patterns it might not be terribly significant.)

BTW regarding kpm pack producing shell scripts, we've fixed some issues in the past month or so. If you're still seeing issues, please log the issues in the https://github.com/aspnet/KRuntime/issues repo.

[vlesierse]

@Eilon ok, having a kpm pack is really something you want when you deploy your application using Docker. I want to try out some scenarios, because it's hard to determine the startup application within a solution where kpm pack has to be executed.

[friism]

Couldn't the convention be for people to specify the command that they want to run in the dockerfile they place with their source code?

Also note that the Docker people are introducing additional ways to specify what commands should be run in containers (see "Docker Compose"), so I wouldn't get too hung up on that...

[ahmetb]

@vlesierse thanks for bringing up the multi-project use case. I did not realize this before. This changes pretty much everything in onbuild image. It appears that since Dockerfile can't run commands on the client side (i.e. kpm pack), perhaps it should accept only the kpm pack output as the input?

I'll try this multi-project setup out next week and see what we can do about it.

@friism can you please elaborate (maybe an example)? Mostly onbuild images just work because the commands executed to download dependencies etc. on every Python app or Golang app are >90% the same. That's why onbuild images handle the common case. Also, I'm not sure if Docker Compose has anything to with commands executed inside a container (maybe I'm missing something). I favor waiting a bit for creating the onbuild image and come up with something right and handles the common case.

[friism]

@ahmetalpbalkan have you considered the topic of project.lock.json? Should onbuild one to present, or not? I don't know if the semantics are the same, but the Rails image requires one: https://registry.hub.docker.com/_/rails/

[ahmetb]

@friism hmm no idea what that is, I couldn't figure out by looking at the rails image. What does that do?

[ahmetb]

hmm found this https://github.com/aspnet/Home/wiki/Lock-file but I'm trying to find an example.

[friism]

@ahmetalpbalkan There's one in musicstore: https://github.com/aspnet/MusicStore/tree/master/src/MusicStore

My current thinking is that onbuild should copy it in with project.json, eg:

FROM aspnet:latest
ENV SRC_DIR /src
RUN mkdir -p $SRC_DIR
WORKDIR $SRC_DIR
ONBUILD ADD NuGet.Config $SRC_DIR/
ONBUILD ADD project*.json $SRC_DIR/
ONBUILD RUN dnu restore
ONBUILD ADD . $SRC_DIR/

[ahmetb]

@friism sounds about right.

[Eilon]

@davidfowl re: lock files

I do think that the lock file should be copied - the runtime uses that file to determine the exact versions of packages to load. I think it'll work without it, but you might get unexpected results. It also serves as an optimization because the entire package graph is already resolved. But Fowler will have ton confirm.

[friism]

@ahmetalpbalkan what about multiple project.json files? I think that one would ideally like to glob in all of them (eg. ADD **/project*.json /src) and then run dnu restore, but I don't think that's possible with Dockerfile. Maybe just limit to one project.json at a time?

[ahmetb]

@friism you're making an assumption like I'm using ASP.NET, on the contrary I don't know much about ASP.NET at this point, so I don't know what multiple projects mean. 😄

If it means multiple applications, then they probably should go into separate images.

If it means an executable entrypoint and bunch of other projects providing DLLs which the entrypoint project depends on, then it should be a single image, however I'm not sure how a single onbuild image can handle building all these...

My plan was to run PartsUnlimited sample application with this onbuild image but last time I tried it was not working on Mono for some reason (I think the feed was messed up somehow and I was getting version-related errors in restore). If you can get it working (I believe it has multiple projects, too) we can iterate on that.

Just a reminder, with this ONBUILD image, we're aiming to handle the 90% of the use cases (i.e. common case) and if it's possible at all, I would love to move further on this proposal.

[muratg]

Putting this in RC2 for now and assigning to @glennc make a decision/talk to @ahmetalpbalkan etc.