Feature request: Alpine-based image

[benglewis]

Hi 👋 ,

I'd really like it if you could provide an alpine based image. It would help us to keep our images slim and consistent

[pavelzw]

The problem with alpine-based images is that they don't ship glibc. Thus, the __glibc virtual package is not available in this image.
Since most core packages in the conda world have a dependency on glibc in some way or another, I'm not sure how much value this will bring.
Do you have a use case where you don't require glibc?
https://github.com/mamba-org/micromamba-docker has alpine with glibc (https://github.com/mamba-org/micromamba-docker/blob/07679e07f2649aa2788b0b83156b8dc89f39bf75/.github/workflows/push_latest.yml#L31-L33) but when using pixi in this container, you still get the error message from pixi about __glibc.

[benglewis]

Right, we are using PyTorch which I see requires glibc (https://stackoverflow.com/questions/76751155/how-to-install-torch-in-alpine), but I also see that micromamba-docker's alpine-based images are approximately 32MB for the base image, as opposed to approximately 98MB for the pixi:latest image.

So, would it be possible to use the same frolvlad/alpine-glibc:alpine-3.19 base image and offer pixi images for this officially? In the meantime, I'm considering rolling my own potentially

Either way, great work on pixi! It is really nice to have a tool that is properly integrated into the Python ecosystem (including things like support for pyproject.toml) and handles locking properly (pixi is sooo much faster than conda-lock which I was using previously with micromamba and it automatically locks on all the commands, which is much cleaner and simpler). I really appreciate the project :)

[benglewis]

Hi @pavelzw 👋

I created a Gist with the Dockerfile for such an image:
https://gist.github.com/benglewis/5f5f1541ef0db6277850a225ea9ad74d

Would it be possible to add this to the repository and publish it alongside the other Pixi images?

[pavelzw]

Did you try out whether pixi init && pixi add python && pixi run python --version works? I remember that pixi doesn't like this image even though glibc is technically installed.

Also, but this might be irrelevant in your use case: you could take a look at multi stage docker builds where you don't include pixi in the final image but only the hook to activate the environment using pixi shell-hook.

[benglewis]

Hi @pavelzw ,

Frustratingly, no, installing Python does not work :'(
I tried various other things including checking permissions and install locations and glibc and gcompat for various versions of Alpine Linux. I even built/compiled pixi on Alpine Linux (which built fine), but it still did not help. It looks like it is probably an issue that pixi just does not work on Alpine Linux (I tried the official package which I came across and it does not work at all either).

I like your suggestion of a multi-stage Docker build, although I am unsure if it will actually work. I will give it a try

[benglewis]

@pavelzw So the multi-stage build did not work for me unfortunately, however, I found that by switching my base image to woahbase/alpine-glibc instead of the frovlad/alpine-glibc, I was able to get pixi working! 🎆
Maybe you can publish a release with this? Here's a new gist:
https://gist.github.com/benglewis/8a379e1dc6ef7c28112d6a4dc52c118e

[pavelzw]

Here is an example of a multi-stage build: https://github.com/prefix-dev/pixi/blob/main/examples/docker/Dockerfile

@wolfv do you know why in frovlad/alpine-glibc __glibc is not recognized by pixi?
We should add proper testing that the images actually work in this scenario.

[benglewis]

I am not entirely sure, I wonder if it is related to the PATH section which I do not see in the frovlad version. I also found that the shell-hook was not working for me in Docker image (neither the new Alpine version that I built nor the one that you have published). I got issues with a scikit-image and gcc being missing (see the log below). I am working on fixing this by installing build-base from alpine's edge release.

109.7   × failed to solve the pypi requirements of 'dev' 'osx-arm64'
109.7   ├─▶ Failed to build: `scikit-image==0.23.2`
109.7   ╰─▶ Build backend failed to build wheel through `build_wheel()` with exit
109.7       status: 1
109.7       --- stdout:
109.7       + meson setup /root/.cache/rattler/cache/uv-cache/built-wheels-v3/pypi/
109.7       scikit-image/0.23.2/KbXEBqrxgirzSwhLcl5j8/scikit_image-0.23.2.tar.gz /
109.7       root/.cache/rattler/cache/uv-cache/built-wheels-v3/pypi/scikit-
109.7       image/0.23.2/KbXEBqrxgirzSwhLcl5j8/scikit_image-0.23.2.tar.gz/.mesonpy-
109.7       5yupibk2 -Dbuildtype=release -Db_ndebug=if-release -Db_vscrt=md
109.7       --native-file=/root/.cache/rattler/cache/uv-cache/built-wheels-
109.7       v3/pypi/scikit-image/0.23.2/KbXEBqrxgirzSwhLcl5j8/scikit_image-
109.7       0.23.2.tar.gz/.mesonpy-5yupibk2/meson-python-native-file.ini
109.7       The Meson build system
109.7       Version: 1.4.0
109.7       Source dir: /root/.cache/rattler/cache/uv-cache/built-wheels-v3/pypi/
109.7       scikit-image/0.23.2/KbXEBqrxgirzSwhLcl5j8/scikit_image-0.23.2.tar.gz
109.7       Build dir: /root/.cache/rattler/cache/uv-cache/built-wheels-
109.7       v3/pypi/scikit-image/0.23.2/KbXEBqrxgirzSwhLcl5j8/scikit_image-
109.7       0.23.2.tar.gz/.mesonpy-5yupibk2
109.7       Build type: native build
109.7       Project name: scikit-image
109.7       Project version: 0.23.2
109.7       
109.7       ../meson.build:1:0: ERROR: Unknown compiler(s): [['cc'], ['gcc'],
109.7       ['clang'], ['nvc'], ['pgcc'], ['icc'], ['icx']]
109.7       The following exception(s) were encountered:
109.7       Running `cc --version` gave "[Errno 2] No such file or directory: 'cc'"
109.7       Running `gcc --version` gave "[Errno 2] No such file or directory:
109.7       'gcc'"
109.7       Running `clang --version` gave "[Errno 2] No such file or directory:
109.7       'clang'"
109.7       Running `nvc --version` gave "[Errno 2] No such file or directory:
109.7       'nvc'"
109.7       Running `pgcc --version` gave "[Errno 2] No such file or directory:
109.7       'pgcc'"
109.7       Running `icc --version` gave "[Errno 2] No such file or directory:
109.7       'icc'"
109.7       Running `icx --version` gave "[Errno 2] No such file or directory:
109.7       'icx'"
109.7       
109.7       A full log can be found at /root/.cache/rattler/cache/uv-cache/built-
109.7       wheels-v3/pypi/scikit-image/0.23.2/KbXEBqrxgirzSwhLcl5j8/scikit_image-
109.7       0.23.2.tar.gz/.mesonpy-5yupibk2/meson-logs/meson-log.txt

[benglewis]

@pavelzw Should I take a further look at the differences between frovlad/alpine-glibc and woahbase/alpine-glibc? Or is official support for an Alpine-based Docker image not on the table for now?

[benglewis]

@pavelzw I would be happy to come back to looking at this more sometime. It would be nice to be able to move to an official image. Also, maybe another alternative that would work for us, is to use distroless, which would have the added advantage of creating an even smaller Docker image. What do you think?
Here's a nice reference article (https://uwekorn.com/2021/03/01/deploying-conda-environments-in-docker-how-to-do-it-right.html)

[pavelzw]

if you want to minimize the docker container size, distroless is the best option (although you lose some easy debugability with it).

in general, i think though, that you just shouldn't ship pixi to production containers since the 30MB pixi binary is pretty large for a container that is supposed to be as small as possible.

please take a look at this blog post i wrote that shows some examples of how to use distroless with pixi. xref https://github.com/pavelzw/pixi-docker-example