Request: Docker image based on Alpine (for ARM64)

[Stolz]

For those of us who would like to run Invidious in our SBC (such Raspberry Pi) ARM computers it would be nice if a Dockerfile based on Alpine image could be provided.

I've tried to create one myself but I get an error while trying to build. I don't know if by installing more dependencies it would be possible to build Invidious for Musl or if glibc is required.

docker run -it --rm alpine /bin/ash

apk --no-cache add crystal shards

git clone https://github.com/omarroth/invidious --depth=1 && cd invidious

shards

crystal build src/invidious.cr --release
/usr/lib/gcc/x86_64-alpine-linux-musl/8.2.0/../../../../x86_64-alpine-linux-musl/bin/ld: cannot find Scrt1.o: No such file or directory
/usr/lib/gcc/x86_64-alpine-linux-musl/8.2.0/../../../../x86_64-alpine-linux-musl/bin/ld: cannot find crti.o: No such file or directory
/usr/lib/gcc/x86_64-alpine-linux-musl/8.2.0/../../../../x86_64-alpine-linux-musl/bin/ld: cannot find -lpthread (this usually means you need to install the development package for libpthread)
/usr/lib/gcc/x86_64-alpine-linux-musl/8.2.0/../../../../x86_64-alpine-linux-musl/bin/ld: cannot find -lrt (this usually means you need to install the development package for librt)
/usr/lib/gcc/x86_64-alpine-linux-musl/8.2.0/../../../../x86_64-alpine-linux-musl/bin/ld: cannot find -lssp_nonshared (this usually means you need to install the development package for libssp_nonshared)
collect2: error: ld returned 1 exit status
Error: execution of command failed with code: 1: `cc "${@}" -o '/root/.cache/crystal/usr-lib-crystal-core-ecr-process.cr/macro_run'  -rdynamic  -lpcre -lgc -lpthread /usr/lib/crystal/core/ext/libcrystal.a -levent -lrt -L/usr/lib -L/usr/local/lib`

Adding extra libraries didn't help

apk --no-cache add openssl libressl-dev libpthread-stubs

[omarroth]

The problem appears to be that it's trying to pull dependencies for x86_64, which likely won't work on ARM.

Unfortunately to my knowledge there isn't an official package of Crystal for arm devices. I've compiled my own version that I'll try to set up as a proper repository when I have time. I'll keep this open but it may be a while before it's supported.

[Stolz]

Sorry, I pasted the output from my x86_64 machine thinking I was on my ARM machine :/

I can now successfully compile it using Alpine on x86_64 (credits). Here is the Dockerfile in case someone is interested

FROM alpine

RUN apk add --no-cache --virtual .build-dependencies crystal g++ gc-dev git libc-dev libevent-dev libxml2-dev llvm llvm-dev llvm-libs llvm-static make openssl openssl-dev pcre-dev readline-dev shards sqlite-dev yaml-dev zlib zlib-dev

ADD . /invidious

WORKDIR /invidious

RUN shards && crystal build src/invidious.cr --release
# TODO build with --static
# TODO Housekeeping: apk del .build-dependencies

CMD ["/invidious/invidious"]

The Alpine image is 50% lighter than the Arch one and it could probably be reduced even more once build time dependencies are removed. It is also significantly faster to build than the Arch one, although to be honest I didn't even tested if the Alpine one works well because I'm only interested on the ARM version.

Using crystal build src/invidious.cr --release --static does not work in spite of being officially supported.

However, for the ARM version of Alpine, as you mentioned, Crystal is not available. Here is a recent discussion about the topic, including a link with instructions for cross-compiling it. I'll try them when I have more time.

[leonklingele]

This has been implemented with #665.

[Stolz]

Great news. To be consistent, it will be also great if the official docker-compose.yml used the Alpine based image of Postgres

[Stolz]

This has been implemented with #665.

Is the Alpine version supposed to work for ARM? I get errors when building

$ uname -a
Linux orangepi 4.19.57-sunxi #5.90 SMP Fri Jul 5 17:58:43 CEST 2019 armv7l GNU/Linux

$ git log -1
commit b63f469110b6d561e06ba7a39a459b71166ef523
Author: Omar Roth <redacted>
Date:   Fri Aug 9 14:09:24 2019 -0500

Fix typo in ConfigPreferences

$ docker-compose build invidious
Building invidious
Step 1/20 : FROM alpine:latest AS builder
---> 962e8b19ad7b
Step 2/20 : RUN apk add -u crystal shards libc-dev     yaml-dev libxml2-dev sqlite-dev sqlite-static zlib-dev openssl-dev
---> Running in 0c9fbabeae7f
fetch http://dl-cdn.alpinelinux.org/alpine/v3.10/main/armv7/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.10/community/armv7/APKINDEX.tar.gz
ERROR: unsatisfiable constraints:
crystal (missing):
required by: world[crystal]
shards (missing):
required by: world[shards]
ERROR: Service 'invidious' failed to build: The command '/bin/sh -c apk add -u crystal shards libc-dev     yaml-dev libxml2-dev sqlite-dev sqlite-static zlib-dev openssl-dev' returned a non-zero code: 2

[leonklingele]

Looks like Crystal is only available for x64 on Alpine: https://pkgs.alpinelinux.org/packages?name=crystal&branch=edge
I've contacted the package maintainer, asking if adding support for other architectures is possible.

[leonklingele]

it will be also great if the official docker-compose.yml used the Alpine based image of Postgres

#687

[igorkulman]

Looks like Crystal is only available for x64 on Alpine: https://pkgs.alpinelinux.org/packages?name=crystal&branch=edge
I've contacted the package maintainer, asking if adding support for other architectures is possible.

Any news on that? I would love to run Invidious on my Raspberry Pi at home.

I found some ways to cross-compile Crystal for ARM but they do not seem exactly trivial.

[fabriciopirini]

+1 here ... I would love to run it in a Docker container on my Raspberry Pi 4! Let me know if any solution/temporary fix is available.

[astrojuanlu]

Crystal is available for ARMv8 on Alpine, but not ARMv7 https://pkgs.alpinelinux.org/packages?name=crystal&branch=edge despite (from what I understand) having actual support for ARMv7 crystal-lang/crystal#3424

Trying to build docker-compose on a Raspberry Pi results in this error at the moment:

$ sudo ~/.local/bin/docker-compose up
Building with native build. Learn about native build in Compose here: https://docs.docker.com/go/compose-native-build/
Building invidious                            
Sending build context to Docker daemon  25.15MB                                                                                                             
                                                                             
Step 1/23 : FROM crystallang/crystal:0.35.1-alpine AS builder          
 ---> 6a09b54a6eb8                                                               
Step 2/23 : RUN apk add --no-cache curl sqlite-static                                                            
 ---> [Warning] The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm/v7) and no specific platform was requested
 ---> Running in 2228066c18bd                                 
standard_init_linux.go:219: exec user process caused: exec format error
The command '/bin/sh -c apk add --no-cache curl sqlite-static' returned a non-zero code: 1
ERROR: Service 'invidious' failed to build

the reason is that there are no official Docker images for Crystal supporting ARMv7 at the moment:

https://hub.docker.com/r/crystallang/crystal/tags?page=1&ordering=last_updated&name=0.35.1-alpine

I was told in the Crystal chat that there are semi-official images available at jhass/crystal, however the Alpine-based ones do not support ARMv7 either:

https://hub.docker.com/r/jhass/crystal/tags?page=1&ordering=last_updated&name=0.35.0-alpine

This means that, with a bit of work, someone could replace the compilation stage of the Dockerfile by one of these semi-official, Debian-based images and make it work.

[astrojuanlu]

arm-linux-musleabihf support is currently broken (see crystal-lang/crystal#5862)

https://github.com/ysbaddaden/crystal-alpine/issues/6#issuecomment-387201907

Unfinished effort at crystal-lang/crystal#7859

[MrAce100]

I would like to run Invidious on my Pi as well- am I correct when I assume we need to wait until someone sorts out that pull request?

[MrAce100]

Any distros you'd recommend? I should mention I'm using a Pi 2 and also running Pihole. I hope to run both at once for home use if possible. Is the Pi 2 even compatible with armv8 at all? Thanks

[unixfox]

Unfortunately the Raspberry Pi 2 can't run ARMv8 and is probably too weak for running Invidious.

[MrAce100]

Even just for home use with 3 clients or so?
Thanks for letting me know- I'll try it when armv7 support is added but will expect failure

[unixfox]

Even just for home use with 3 clients or so?
Thanks for letting me know- I'll try it when armv7 support is added but will expect failure

Well I don't know if you are going to use dash or not but dash requires a "lot of power" even for 3 users.

Apart from that postgres also use a fair amount of CPU and Invidious use a lot of memory due to some memory leak (see #1438) so if you have only 1GB of RAM that might not be enough.

[MrAce100]

Good to know, thanks.

[unixfox]

Hi!

Thanks to the work of @nemunaire, we are finally officially supporting ARM(v8) Docker image!

It is available under this Docker tag: quay.io/invidious/invidious:latest-arm64
In order to test it, you need to use ARM64/ARMv8 on your device. Raspbian doesn't support ARMv8 by default, you may use Ubuntu ARMv8 instead!

Please test it and give your feedback to us in this GitHub issue!

Some Q&A

Why aren't we supporting ARMv7?

Unfortunately, Crystal doesn't really support ARMv7 for musl (a C standard library), see https://crystal-lang.org/reference/platform_support.html, so there is no crystal package for ARMv7 on Alpine Linux.

Why is the Docker image not available under the Docker tag latest?

Building a multi architecture Docker image requires to use the same Dockerfile, but Crystal doesn't officially offer an ARM64 docker image, so we have to rely on the crystal package from Alpine Linux.
The version of this package may change and result in the inability of building the Docker image because Invidious frequently fail to work on newer version of Crystal.
That's why we prefer for the moment to rely on the official Crystal docker image in order to build our Docker image for x86/AMD64 and thus having a separate Dockerfile for each CPU architecture.