bug: nix install script doesn't work in docker on m1 machines

[Miaxos]

Description

There is no precompiled binary for aarch64, which is troublesome when you are working with a M1, I think it would be useful to add it, there are more and more people working on this architecture.

 > [ 7/14] RUN curl -sSL https://rover.apollo.dev/nix/v0.4.1 | sh:
#10 0.457 ERROR: no precompiled binaries available for CPU architecture: aarch64

Linked issue: #936 #260 #582

Thank you for this awesome crate :-)!

[EverlastingBugstopper]

Hey @Miaxos - we don't have any plans to build native ARM binaries right now, but you should be able to use the precompiled x86_64 binaries for your M1 machines since they come installed with Apple's Rosetta software which allows x86_64 code to be run just fine. Hope this helps!

[EverlastingBugstopper]

also i'm closing this as a duplicate of #260

[Miaxos]

Yeah, but in a docker context within M1 it's really troublesome to run your entire docker stack through rosetta.

[abernix]

Yeah, but in a docker context within M1 it's really troublesome to run your entire docker stack through rosetta.

That would be troublesome, but I don't quite understand how this would be the outcome. Could you elaborate on how that would happen?

(If you run rover on Mac and rover itself runs through Rosetta, that shouldn't cause Docker to be Rosetta'd, only rover. If you need to run rover inside Docker, presumably you'd be using a Linux version of rover and using x86_64 binaries inside Docker on macOS doesn't cause the entirety of Docker to use Rosetta, does it?)

[Miaxos]

Well, if you install rover inside your docker image which is running inside an M1, (if docker isn't running through rosetta, if it is, idk, it's not what I'm doing), you'll have your docker images built for aarch64 and not x86_64, to build x86_64 images you'll have to use buildx, so when you want to install rover inside your docker image, you'll get a :

at 03:26:01 ❯ docker build -t test . && docker run test
[+] Building 9.7s (6/6) FINISHED                                                                                  
 => [internal] load build definition from Dockerfile                                                         0.0s
 => => transferring dockerfile: 226B                                                                         0.0s
 => [internal] load .dockerignore                                                                            0.0s
 => => transferring context: 2B                                                                              0.0s
 => [internal] load metadata for docker.io/library/ubuntu:latest                                             1.1s
 => CACHED [1/3] FROM docker.io/library/ubuntu:latest@sha256:b5a61709a9a44284d88fb12e5c48db0409cfad5b69d4ff  0.0s
 => [2/3] RUN apt update &&     apt install -y apt-transport-https ca-certificates curl gnupg lsb-release    8.1s
 => ERROR [3/3] RUN curl -sSL https://rover.apollo.dev/nix/v0.4.1 | sh                                       0.5s
------                                                                                                            
 > [3/3] RUN curl -sSL https://rover.apollo.dev/nix/v0.4.1 | sh:                                                  
#6 0.446 ERROR: no precompiled binaries available for CPU architecture: aarch64                                   
------                                                                                                            
executor failed running [/bin/sh -c curl -sSL https://rover.apollo.dev/nix/v0.4.1 | sh]: exit code: 1    

Dockerfile:

FROM ubuntu:latest
RUN apt update && \
    apt install -y apt-transport-https ca-certificates curl gnupg lsb-release
RUN curl -sSL https://rover.apollo.dev/nix/v0.4.1 | sh
CMD rover

[EverlastingBugstopper]

my guess is that the docker environment is a bit different than real m1 machines. usually, uname -s returns Darwin on both M1 and x86_64 machines but it looks like it's returning aarch64 for you in docker instead. my guess is that the get_architecture section of our installer script needs to be updated to check for the proper uname -s and uname -m outputs for this particular environment. it's failing because it's returning unexpected values for those commands and it's not even trying to download the binaries from our github releases page that should work as expected.4

we have a few other things in flight at the moment so I'm not sure when this will be addressed but I'm going to reopen this and edit the issue title to reflect that the issue could be addressed by modifying our install script. for now, you could download the install script and modify the get_architecture command to work on your system or download the binaries directly from our releases page and add it to your PATH in Docker and it should work fine.

if you do end up getting the script fixed up we'd love to accept a PR

[mlkiefer]

I think the script is running correctly.
The point Miaxos is trying to make is that a Docker container on an M1 Mac is running Linux on a VM. So the script sees that it is running on Linux on an aarch64 platform and tries to obtain a binary for that one, which doesn't exist.

$ uname -s
Linux
$ uname -m
aarch64

If you tried to override that in the script and run x86_64 binaries in there, the Linux in the VM wouldn't know what to do with it, because it doesn't have something like Rosetta. That would be like running an Intel Linux binary on a Raspberry Pi or a mobile phone.

[EverlastingBugstopper]

ahhh, thanks for the explainer there @mlkiefer, that makes a lot of sense. in that case i'm actually going to close this as a duplicate of #582 (which does seem to include some user-submitted Dockerfiles that may help you!)

it seems like there have been increasing requests for arm64 binaries as of late, I can't make any promises as to if/when those binaries will be provided but do know that it's on our radar.

[EverlastingBugstopper]

actually, it looks like linux M1 builds can use our x86_64 binaries. @garypen pointed this PR out to me that fixed the same thing. going to do the same hack on our installer and get that out to you today

[EverlastingBugstopper]

ok so my own investigation into this is yielding very strange results. no matter what i do i can't seem to get uname -s and uname -m to return anything other than "Linux" and "x86_64" on my M1 machine running docker from ubuntu:latest scratch this, for some reason i thought my work macbook (that i rarely use) was an M1, but now i'm remembering that I have an M1 that I need to ssh into.

Anyways, I need to get Rover v0.4.2 out today and don't feel confident that I can make the same change that Gary made in today's time frame regardless. We will revisit this.

edit: as a workaround, you could try to download the x86_64 binaries directly from our github releases. i imagine it would work!

[mlkiefer]

It probably works for the x86_64 Mac binary in the native Mac environment but it won't work in the Linux VM that runs the docker container.
That's a linux on an ARM (virtual) machine so it doesn't know what to do with an x86_64 binary. In any case, we have removed rover from our setup so at least for me, this is not important any more.
However, thanks for working on Apollo and providing these tools! ❤️

[garypen]

The docker build problem isn't caused by the install script, but by trying to install amd64 rover into the aarch64 (ARM) docker image. From above:

FROM ubuntu:latest
RUN apt update && \
    apt install -y apt-transport-https ca-certificates curl gnupg lsb-release
RUN curl -sSL https://rover.apollo.dev/nix/v0.4.1 | sh
CMD rover

By default, on an M1 machine, that will retrieve an aarch64(ARM) docker image and thus the install of rover will fail. If however, I have this Dockerfile:

FROM amd64/ubuntu:latest
RUN apt update && \
    apt install -y apt-transport-https ca-certificates curl gnupg lsb-release
RUN curl -sSL https://rover.apollo.dev/nix/v0.4.1 | sh
ENTRYPOINT ["/root/.rover/bin/rover"]

(Note the specification of amd64, then all is well. Slow, but well) I changed CMD to entrypoint to avoid PATH type problems.

Build as:

docker build -t rover:0.4.1 .

If I run the docker image this creates on my M1 laptop, then I get:

garypen@Garys-MBP rover % docker run -it --rm rover:0.4.1 
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
Rover 0.4.1

Rover - Your Graph Companion
Read the getting started guide by running:

    $ rover docs open start

To begin working with Rover and to authenticate with Apollo Studio,
run the following command:

    $ rover config auth

This will prompt you for an API Key that can be generated in Apollo Studio.

The most common commands from there are:

    - rover graph fetch: Fetch a graph schema from the Apollo graph registry
    - rover graph check: Check for breaking changes in a local graph schema against a graph schema in the Apollo graph
registry
    - rover graph publish: Publish an updated graph schema to the Apollo graph registry

You can open the full documentation for Rover by running:

    $ rover docs open

USAGE:
    rover [FLAGS] [OPTIONS] <SUBCOMMAND>

FLAGS:
        --insecure-accept-invalid-certs        Accept invalid certificates when performing HTTPS requests
        --insecure-accept-invalid-hostnames    Accept invalid hostnames when performing HTTPS requests
    -h, --help                                 Prints help information
    -V, --version                              Prints version information

OPTIONS:
        --client-timeout <client-timeout>    Configure the timeout length (in seconds) when performing HTTP(S) requests
                                             [default: 30]
    -l, --log <log-level>                    Specify Rover's log level [possible values: error, warn,
                                             info, debug, trace]
        --output <output-type>               Specify Rover's output type [default: plain]  [possible values:
                                             json, plain]

SUBCOMMANDS:
    config        Configuration profile commands
    docs          Interact with Rover's documentation
    explain       Explain error codes
    fed2          Federation 2 Alpha commands
    graph         Graph API schema commands
    help          Prints this message or the help of the given subcommand(s)
    subgraph      Subgraph schema commands
    supergraph    Supergraph schema commands
    update        Commands related to updating rover
garypen@Garys-MBP rover % 

(You can get rid of the WARNING by specifying --platform=linux/amd64)

[EverlastingBugstopper]

AH! thanks for the help on this @garypen - I don't think we'll update our installer code then and folks can use this as a workaround. I'm going to close this and we can continue to track aarch64 binaries in issue #582.