Docker : Create official image

[zekth]

@hayd started a while https://github.com/hayd/deno-docker

This can be ported to be integrated as the Official Deno docker image. So 3 steps are needed:

• Integrate generation of docker image in the CI
• Test the generated docker image
• Publish image on the docker hub / github hub on release

Also do we plan to release on docker hub only or we push it to github registry also? It's just a matter of URL in the command in the CI but @ry and @piscisaureus have to create / set credentials in the project.

[hayd]

Ry has denoland on dockerhub.

I think we should solve glibc issue on Alpine before shipping, as I think Alpine is preferred target. #3243
I don't know how to do sccache on docker, the Amazon Linux build (from scratch) is pretty slow on GitHub actions ~45minutes. However this could be a lot quicker if we instead generated a compatible binary as part of regular CI (no need for building or even installing curl just ADD).

On https://github.com/hayd/deno-docker GitHub actions has my docker hub API key and pushes on tags to docker hub, very easy. See release.yml. Deno could do both :latest (every commit) and :VERSION.

[hayd]

Targeting x86_64-unknown-linux-musl might make this work... And also mean the Amazon Linux (Lambda) build becomes trivial (at the moment it's kinda a pain/slow to create a compatible binary).

denoland/rusty_v8#49

[aadamsx]

Currently trying out https://github.com/hayd/deno-docker

[grant]

Currently trying out https://github.com/hayd/deno-docker

This has worked for me too. Docker Hub link: https://hub.docker.com/r/hayd/deno

[andrewmclagan]

Keenly following the progress of this. Alpine Deno image is essential :-)

[gvatn]

Note that several people have had performance problems when using rust on musl: https://andygrove.io/2020/05/why-musl-extremely-slow/

[hayd]

I am happy to help move over hayd/deno-docker to denoland, if that was desired.

Most are a very thin wrapper, extract the binary, create user and deno-dir, set up entrypoint:
https://github.com/hayd/deno-docker/blob/master/alpine.dockerfile
(The alpine ones uses glibc.)

@kesor created a musl built image recently! My suspicion is it makes sense to incorporate some of that into BUILD.gn/gn-args instead of maintaining a patch to apply:
#3711 (comment)

It'd be great to have some deno-specific benchmarks before dismissing musl here... however one concern is currently alpine offers llvm9 which means a larger deno binary (since no_inline_line_tables=false), so atm the musl alpine image is larger than alpine w/ glibc.

[andrewmclagan]

I mean 27MB is quite slim really.

[andrewmclagan]

I think once this is decided on and stable we could PR the official docker repo.

[zekth]

Also we can push to docker registry and also github registry don't you think?

[nihaals]

It's also standard to have the OS part of the tag instead of the repository name. Is it possible to get _/deno? It might be better to have an official image.

[cqh963852]

Deno image based on the buster is needed for development

[cqh963852]

Will we create a new reop just for dockerfile?

[Vanuan]

Just wanted to try this out, found out that there's no official image yet. What's the workaround?
This one? https://hub.docker.com/r/hayd/alpine-deno/

[nihaals]

We can make a proposal to Docker to have an official image and then both deno and Docker will help work on the image to make it as efficient and stable as possible. Updates are also managed by Docker. It's probably better to do it from the start so people don't have to move from a deno image to an official image and there will almost definitely be confusion with the differences.

[lucemans]

Working on this repository atm: https://github.com/lucemans/docker-deno
Currently have smaller images, and different way of installing to avoid having to download extra software etc. Also should auto update to the latest deno version

[nihaals]

That's not how Docker images are meant to be designed... Everything should be installed beforehand not while creating the container

[hayd]

We can make a proposal to Docker to have an official image and then both deno and Docker will help work on the image to make it as efficient and stable as possible.

@nihaals what is the route for such a proposal?

[nihaals]

The first step is to create our final images. I've created a repo which can house the images while they are being developed and reviewed. What will happen is first we will need to create the images (I think @hayd will probably be able to kickstart this). Once we have a set of images that work, I can create a PR to the official images repo. This kickstarts the proposal.

During this time, people who know a lot more about maintaining secure and efficient Docker images will give feedback which will then be implemented (they may PR, I think it would most likely require us to implement the changes ourselves), and once the images are good enough, they will be accepted. Every update the images will be updated and a new PR will be made giving the new GitHub commit with the newer images. These images will again be verified (most likely with less feedback), and then updated on Docker Hub. It's this reason why anyone can host the repo during this phase, as every update will be manually verified by them.

It's usually standard to build from source in images and making sure to delete any kind of caches. I can also implement a testing system to make sure any images actually work as there may be a large number of PRs initially. I personally think we should focus on slim images (using Debian) as these tend to have the least issues (for example Alpine using musl).

Once the first proper images are done, automation just needs to be used to make sure they are updated with the latest version of deno (possibly also updating any checksums). We can take inspiration from the Node and Python images as they're very optimised. The main thing I'm unsure on is architecture support, the rest I've played around with. Part of the proposal is listing the supported architectures and if one requires a different image, listing that too. It may also be useful to move the repo to some kind of docker-deno org (I've currently reserved this name), but right now the images are going to require a lot of fine tuning.

Also, a modification for some of the 3rd party images will be needed to work here: we should be using official base images, as this will increase our likelihood of being accepted.

[nihaals]

We also need a member to confirm whether they are or aren't interested in maintaining the image (either a member needs to be related to the repo or we need to have "contacted" them (assuming this means we need permission)). We shouldn't even really consider moving forward until we get some response as this will block it from happening at some point and it might be that they would prefer to be the main maintainers.

[nathanblair]

I've also got a vested interest in seeing a musl-compatible binary for release on alpine. It could also be extrapolated to be run on void-linux as well. Watching this as well as #3711.

If there's anything I can do to help with troubleshooting the Dockerfile give me a holler.

[andrewmclagan]

Any movement on this?

[nihaals]

Just waiting on a response from a project member

[youngjuning]

Just waiting on a response from a project member +1

[zekth]

@bartlomieju Do you think we can move the @hayd project in the official Deno repo in the future? Seems like it's mature enough.

[nihaals]

I’m not sure if that makes much sense if it doesn’t follow the official docker image guidelines
There’s really not much that can be done until a member gives their input on either hosting the repo themselves or allowing a community member to
Going by the lack of response in ~8 months, Docker doesn’t seem to be a current priority so you should probably stick to either installing it yourself or using the previously mentioned repo

[hayd]

I’m not sure if that makes much sense if it doesn’t follow the official docker image guidelines

Are there some specific things hayd/docker-deno images are doing/not doing? Either way we can address these as GitHub issues ...

Atm it seems like there's been no indication the correct docker team is watching/interested in managing a deno image (yet).

Moving to denoland seems positive way forward.

[nihaals]

I already explained in my write-up.

It's obviously up to the project members to decide, they may not want to even have an official image for whatever reason. The key point is that it's up to them. It's their project, they have the final say, even if it's open source

[hayd]

When you say "project member" do you mean of deno(land)? or a member of the docker official images team?

[nihaals]

I'm quoting Docker's docs, I believe they say member of upstream, which I guess in this case would be a member of the denoland org

[hayd]

I see (the original link you posted)

it is preferable to have upstream software authors maintaining their corresponding Official Images

This reads like it's a good start for dockerfiles to be moved to denoland if we want a deno Official Image in the future.

[nihaals]

I agree, the only reason I'm hesitant is that it could actually delay progress since it would be rewritten from scratch (not all the current main images are built from source and use official base images)
Your images are great for anyone who needs a docker image now and is probably very similar to what people would do in their own custom images (I want to be clear that I think they're definitely a great resource), but official docker images, understandably, have much tighter restrictions as they could end up having billions of pulls, so feels like a drastically different beast. Many people wouldn't consider checksum checks to be very important, but when you're creating images that effectively define what good, safe and secure images should look like, they may be important

As Docker images don't seem to be a current priority, this possible further delay could mean that we don't get official images for a further 6 months due to miscommunication or misunderstandings of the requirements of official images, compared to having a space where these images are already defined, so having them available to everyone could be as simple as getting an OK from a member or transferring the repo

[ry]

I don’t think we (denoland) are going to do an official docker image right now. Mostly because I’m not sure we have the bandwidth to manage it. My attempt to manage the vscode extension, for example, has not been going well. Andy’s image is working fine.

Maybe in the future if it makes sense.

[nihaals]

Does this also mean hold back on a "Docker Official" image or are you giving permission for someone else to work on and submit it? Anyone is allowed to submit one for review as long as they have permission afaik

[hayd]

Permission is not a prerequisite... at least it's not mentioned in the docs.

I'm happy to take any feedback in order for hayd/deno-docker to meet any Official requirements, please create separate GH issues on that repo. That way we could more easily move it to denoland or Docker Official in the future.

(@ry Atm updates are bumping the version number and tagging releases, so docker would be much less arduous than vscode... But it would still be "another thing" to maintain!)

[hayd]

Just to note, if there is some suggestion this is important/useful for Official Images, @lucacasonato is a maintainer of hayd/deno-docker and also a member of denoland.

Anyway, perhaps let's move this discussion there for now.

[jiraguha]

@ry It already makes sense now IMO. I think that having ASAP an official image will remove some frictions to production... As a Kubernetes user, I would be more confident to ship deno to production with an official image that has support for the community. And Kubernetes is only one of several use-cases where we would need an official docker image.

hayd/deno-docker might be at an early stage but why do not we move it to denoland and incrementally improve it from there? People from the community that have time to maintain it, would find it easy to do PR's @ denoland. If there is no official place for it, the community will attend to be scattered and will have hundreds of images of deno to select or improve?

I hope that many others share the same opinion about such an important topic :-)

[ghost]

So what's holding the "Deno" team to publish official image on Docker Hub? or it's not really up to Deno team?

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.

[Tzahile]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.

I don't think it makes sense to close this issue, as it remains an issue.
Furthermore, google search leads here. Closing this issue without resolving it might be confusing.

[LRNZ09]

I totally quote @jiraguha, I'm new to Deno, and I'd like to try it within a Docker container first.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.

[manniL]

Oh No! :(

[Guileas]

Hy,
Any news on this topic?

[bartlomieju]

Huge thanks to @hayd who's been maintaining Docker images for the past two years. Today these images were transferred to denoland org and are available at https://github.com/denoland/deno_docker and thus are now considered official. Dockerhub account will be setup shortly.