Include liblsquic build in dockerfile + add support for ARM64 on Docker #2147
Conversation
|
I'm not going to lie, using APKBUILD in order to build lsquic and boringssl is a very clever idea. Kudos to you. The only issue with Or we can build our own crystal Docker image for aarch64 but this seems to be a very complicated task from what I tried a couple of days ago. Apart from this issue, I think it's a great idea to build lsquic directly into the Dockerfile. In fact I still up to this day don't understand why lsquic-static-alpine was created because we can utilize the Docker cache layer and just build lsquic once. |
|
After testing this PR, on an ARMv8 and x86 device, Invidious will crash when the client load a URL from the videoplayback endpoint. This is easily reproducible by enabling dash or enabling proxy videos. The error message is this one on an ARMv8 device: And on an x86 device: This is related to the internal HTTP client of crystal because turning off QUIC with Not sure how to solve this issue nor what's really causing this issue. |
|
Perhaps the next alpine 3.14 will have crystal-1.0 and we can rely on the version stability. But the real solution is crystal publishing containers for arm and arm64. Meanwhile, we could just pick the liblsquic build, with I got the same error, when using |
|
From what I understand, in the previous Dockerfile, Invidious was compiled with In this second patch, I also keep using |
|
Thank you for fixing this issue, it is now indeed working perfectly on both arm64 and x86. I think until Alpine 3.14 comes out with crystal 1.0.0 you can publish the arm64 Dockerfile under the file name When I've some free time, I'll work to automatically build the arm64 docker image using GitHub actions and Docker buildx. You may also try to work on it if you have some free time too. EDIT: I'm converting this PR to a draft for the moment until we have automated the build of the ARM64 docker image. |
|
I tried lots of thinks but none of them are working 😞 I didn't find sufficient documentation about existing variable to change the To use buildx, we have to split the build process between arm64 and amd64: - name: Build andpush forPush Event (amd64)
if: github.ref = 'refs/heads/master'
uses: docker/build-push-action@v2
with:
context: .
platforms: linux/amd64
file: docker/Dockerfile
labels: quay.expires-after=12w
push: true
tags: quay.io/invidious/invidious:${{github.sha}},quay.io/invidious/invidious:latest
- name: Build andpush forPush Event (arm64)
if: github.ref = 'refs/heads/master'
uses: docker/build-push-action@v2
with:
context: .
platforms: linux/arm64
file: docker/Dockerfile.arm64
labels: quay.expires-after=12w
push: true
tags: quay.io/invidious/invidious:${{github.sha}}-arm64,quay.io/invidious/invidious:latest-arm64This is not very convinient as we can't make I tried with a common image #Define the FROM imagedepending on the target arch
ARG TARGETPLATFORM
ENV CRYSTALBUILDER="${TARGETPLATFORM/linux\/amd64/crystallang\/crystal:1.0.0-alpine}"
ENV CRYSTALBUILDER="${CRYSTALBUILDER:-alpine:edge}"
FROM $CRYSTALBUILDER AS builder
RUN if ["${CRYSTALBUILDER}" = "alpine:edge" ]; then \
apk add --no-cache 'crystal<2' shards sqlite-static yaml-static yaml-dev libxml2-dev zlib-static openssl-libs-static openssl-dev musl-dev; \
else \
apk add --no-cache sqlite-static yaml-static; \
fiBut it didn't work because there is no such variable substitution in The legacy way is to create I am short of solutions now. If you have any advise to continue, I'll take it! |
|
Thanks for trying to make GitHub actions working for ARM. What about building both of the two Docker images with buildx for the same Docker tag I was also thinking that if buildx doesn't allow to do that, we could instead build the ARM docker image with a normal Anyway, if it's not possible to combine the two docker image into one, then having two separate tags is fine for the moment. |
|
i got error at line 45 There is an error in the yaml syntax: YAMLSyntaxError: Document contains trailing content not separated by a ... or --- line |
This PR is not really ready yet but could you explain a bit more how you got this error? While browsing to which page of invidious? |
|
Alpine 3.14 just released: https://pkgs.alpinelinux.org/packages?name=crystal&branch=v3.14. I still want to have two different Docker images because I prefer to keep relying on the official crystal Docker image that we know work well for x86. If after a couple of months, the ARM docker image is stable, we can think of moving to a custom Docker image for the x86 Docker image. |
unixfox
changed the title
Close #361
Hi,
This pull request is an attempt to resolve #361.
First, instead of relying on the static built liblsquic from https://github.com/iv-org/lsquic-static-alpine, it builds it as a first step. It works both for amd64, arm and arm64.
Secondly, instead of using
crystallang/crystal:1.0.0-alpine, which is only available for amd64, it usesalpine:edge, as there are crystal-1.0.0 for both amd64 and arm64, but not for arm.So thanks to this PR, Docker image can be built for amd64 and arm64, but not arm (v6 nor v7, because
alpinedoesn't provide a package for it, and I don't find any Docker image for crystal-1.0.0).All this work was done thanks to @omarroth instructions and its archived repositories.
I have been using Invidious since a few month now on a Pine64 board with 2GB of RAM, and it works fine (but 2GB is not sufficient to build it). Here is a arm64 only image that I built with the exact commit of this PR: https://hub.docker.com/r/nemunaire/invidious