Dockerfile with pnpm and Elixir 1.16

[ndrean]

Match code with Dockerfile.
The Dockerfile builds correctly. I did not use it nor tested the deployment of course.

Why?

You seemed to have had problems with npm in the dev mode and used successfully pnpm to overcome this.

For the Dockerfile, you don't use pnpm and I assume it works.

If you want to match the code with the Dockerfile, you may want to use pnpmtoo.

How?

For this, you need to install a recent Node.js version via cURL (Debian comes with v12 installed whilst pnpm requires >v16).
Note that node comes with npm.

You can use the Dockerfile below.

Note I optionally updated to the most recent Elixir version to match the Elixir 1.16 also optionally updated in the MixProject.

# see https://hub.docker.com/r/hexpm/elixir/tags?page=1
ARG ELIXIR_VERSION=1.16.2
                    ^^^
ARG OTP_VERSION=25.3.2.10
                    ^^^
ARG DEBIAN_VERSION=bullseye-20240130
                    ^^^

ARG BUILDER_IMAGE="hexpm/elixir:${ELIXIR_VERSION}-erlang-${OTP_VERSION}-debian-${DEBIAN_VERSION}"
ARG RUNNER_IMAGE="debian:${DEBIAN_VERSION}"

FROM ${BUILDER_IMAGE} as builder

# install build dependencies (and curl for EXLA)
RUN apt-get update -y && apt-get install -y build-essential git curl -y libmagic-dev && \
  curl -sL https://deb.nodesource.com/setup_18.x | bash - && \
    ^^^
  apt-get install -y nodejs && \
  apt-get clean && rm -f /var/lib/apt/lists/*_* && \
  node --version && \
  npm --version

RUN npm install -g pnpm
  ^^^

# prepare build dir
WORKDIR /app

# install hex + rebar
RUN mix local.hex --force && \
  mix local.rebar --force

# set build ENV
ENV MIX_ENV="prod"

# install mix dependencies
COPY mix.exs mix.lock ./
RUN mix deps.get --only $MIX_ENV
RUN mkdir config

# copy compile-time config files before we compile dependencies
# to ensure any relevant config change will trigger the dependencies
# to be re-compiled.
COPY config/config.exs config/${MIX_ENV}.exs config/
RUN mix deps.compile

COPY priv priv

COPY lib lib

COPY assets assets

# Install dependencies for assets folder
RUN pnpm install --prefix assets
  ^^^
...

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 100.00%. Comparing base (462ef27) to head (0008846).
Report is 16 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff            @@
##              main       #87   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files            5         5           
  Lines          199       199           
=========================================
  Hits           199       199           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[nelsonic]

Curious why the OTP_VERSION downgrade? Elixir 1.16.x is compatible with OTP 26 ... 💭
https://hexdocs.pm/elixir/main/compatibility-and-deprecations.html#between-elixir-and-erlang-otp

[ndrean]

I had the same question ! But no image with OPT26

[LuchoTurtle]

I'm having trouble understanding why we're introducing pnpm in this project. I haven't used it in this project so far. Though I agree pnpm is certainly a better alternative to npm, I don't see why we have to use it in the Dockerfile (and subsequently downgrade the OTP version).

Can you clarify this for me, please? :)

[ndrean]

@LuchoTurtle
I believe we used pnpm because of #36 to install "properly" the dependency "[email protected]" (and I used it as well for "[email protected]"). As you said, pnpm is a much better alternative to simply npm. Bringing it in the Docker image is easy (once you know how to)!.

Then we don't downgrade subsequently the OTP because of this. It is downgraded because of the upgrade to v1.16. You chosed the image from hex.pm and it does not seem to offer 1.16+26 at the moment. However, the "official" https://hub.docker.com/_/elixir/ seems to offer 1.16+26.

[ndrean]

If you prefer, it looks like 1.16.1+26 is available, but 1.16.2 comes with 25. Probably a question of time.

ARG ELIXIR_VERSION=1.16.1
ARG OTP_VERSION=26.2.3
ARG DEBIAN_VERSION=bullseye-20240130

https://hub.docker.com/r/hexpm/elixir/tags?page=1&name=1.16.1
https://github.com/fly-apps/hello_elixir/blob/main/Dockerfile

Then the node version in the Dockerfile runs on Linux, is v12.2 I believe, and you need to bring in npm which is probably good enough to install your package. On the other side, I presume you are running node on OSX in dev mode, which is just different.

[nelsonic]

LGTM. 👌

[ndrean]

How to test a Docker image?

I fought a little bit, and my process ends with an error 137 (out of memory) ?? (I have only 8G RAM, Docker+MacOS)

Hopefully it works, so if you have a better way, I take it.

We can whether use build: . and pass args in the "docker-compose.yml" file or use an image.

1. If Image:

• create a local registry to accelerate the push:

docker run -d -p 5001:5000 --name registry registry:2.7

• build the app image that runs only the release (the "builder"):

docker build --build-args="MIX_ENV=prod" -t localhost:5001/img:latest .

where the Dockerfile uses a build-arg to switch Mix modes:

...
FROM ${BUILDER_IMAGE} as builder
ARG MIX_ENV
ENV MIX_ENV=$MIX_ENV
...
FROM ${RUNNER_IMAGE}
ARG MIX_ENV
ENV MIX_ENV=$MIX_ENV
...

• push to the local registry

docker push localhost:5001/img-[dev/prod]:latest

2. Instantiate the database.

• run App.Release.migrate/0 in the Dockerfile CMD
• or use the Postgres initialisation script located under "/docker-entrypoint-initdb.d/".

The 1st option uses CMD to execute Ecto.Migrator. You firstly run the migration and then start the app:

CMD ["sh", "-c", "/app/bin/migrate && /app/bin/server"]

Check he Fly.io blog on running migrations
Also this thread on Elixirforum helps a lot.

The 2d option works only for a newly created database. Clean the generated file below and save it.

mix ecto.create && mix ecto.migrate --log-migrations-sql > ./init.sql

• create from the previous file the (say) "init-db.sh" file and bind it to a mount volume in the database container init scripts folder "/docker-entrypoint-initdb.d/". Note the usage of psql variables.

# init-db.sh
#!/bin/bash
set -e

psql -v ON_ERROR_STOP=1 --username "$POSTGRES_USER"  -d "$POSTGRES_DB" -v db="$POSTGRES_DB" -v user="$POSTGRES_USER" -e <<-EOSQL

    GRANT ALL PRIVILEGES ON DATABASE :db TO :user;

    CREATE TABLE "images" (
        "id" bigserial,
        "url" varchar(255),
        "description" varchar(255),
        "width" integer,
        "height" integer,
        "inserted_at" timestamp(0) NOT NULL,
        "updated_at" timestamp(0) NOT NULL,
        PRIMARY KEY ("id")
    );

    ALTER TABLE "images"
        ADD COLUMN "idx" bigint,
        ADD COLUMN "sha1" varchar(255);

    CREATE UNIQUE INDEX "images_sha1_index" ON "images" ("sha1");
    
    CREATE UNIQUE INDEX "images_idx_index" ON "images" ("idx");

    CREATE TABLE IF NOT EXISTS "hnswlib_index" (
        "id" bigserial,
        "lock_version" integer DEFAULT 1,
        "file" bytea,
        PRIMARY KEY ("id")
    );
EOSQL

3. Create an ".env-docker" file for docker compose:

# .env-prod-docker
MIX_ENV=prod

POSTGRES_PASSWORD=postgres
POSTGRES_USER=postgres
POSTGRES_DB=app_prod
PGDATA=var/lib/postgresql/data/pg-data

SECRET_KEY_BASE=qhSXOpnkmvYzf3/maxi4/pg+WkVnYkhp
DATABASE_URL=ecto://postgres:postgres@db:5432/app_prod
PHX_SERVER=true

4. Run the "docker-compose.yml". The "pg-data" volume can be pruned to re-instantiate the DB.

docker compose up

# docker-compose.yml
version: "3.9"

volumes:
  pg-data:
    driver: local

services:
  db:
    image: postgres:15.3-bullseye
    env_file:
      - .env-docker
    restart: always
    volumes:
      - pg-data:/var/lib/postgresql/data/pg-data
      - ./init.sql:/docker-entrypoint-initdb.d/init.sh:ro
    ports:
      - 5432:5432
     
  app:
    build:
      context: .
      args:
        - MIX_ENV
     # OR
    image: localhost:5001/img-[dev/prod]:latest

    depends_on:
      - db
    env_file:
      - .env-docker
    ports:
      - 4000:4000

[LuchoTurtle]

Thanks @ndrean !

[nelsonic]

🚀