Can't use apt-get in Dockerfile when using datasetteproj/datasette as base

[brandonrobertz]

The datasette base Docker image is super convenient, but there's one problem: if any of the plugins you install require additional system dependencies (e.g., xz, git, curl) then any attempt to use apt in said Dockerfile results in an explosion:

$ docker-compose build
Building server
[+] Building 9.9s (7/9)
 => [internal] load build definition from Dockerfile                                                                                                                                                                                                     0.0s
 => => transferring dockerfile: 666B                                                                                                                                                                                                                     0.0s
 => [internal] load .dockerignore                                                                                                                                                                                                                        0.0s
 => => transferring context: 34B                                                                                                                                                                                                                         0.0s
 => [internal] load metadata for docker.io/datasetteproject/datasette:latest                                                                                                                                                                             0.6s
 => [base 1/4] FROM docker.io/datasetteproject/datasette@sha256:2250d0fbe57b1d615a8d6df0c9d43deb9533532e00bac68854773d8ff8dcf00a                                                                                                                         0.0s
 => [internal] load build context                                                                                                                                                                                                                        1.8s
 => => transferring context: 2.44MB                                                                                                                                                                                                                      1.8s
 => CACHED [base 2/4] WORKDIR /datasette                                                                                                                                                                                                                 0.0s
 => ERROR [base 3/4] RUN apt-get update     && apt-get install --no-install-recommends -y git ssh curl xz-utils                                                                                                                                          9.2s
------
 > [base 3/4] RUN apt-get update     && apt-get install --no-install-recommends -y git ssh curl xz-utils:
#6 0.446 Get:1 http://security.debian.org/debian-security buster/updates InRelease [65.4 kB]
#6 0.449 Get:2 http://deb.debian.org/debian buster InRelease [121 kB]
#6 0.459 Get:3 http://httpredir.debian.org/debian sid InRelease [157 kB]
#6 0.784 Get:4 http://deb.debian.org/debian buster-updates InRelease [51.9 kB]
#6 0.790 Get:5 http://httpredir.debian.org/debian sid/main amd64 Packages [8626 kB]
#6 1.003 Get:6 http://deb.debian.org/debian buster/main amd64 Packages [7907 kB]
#6 1.180 Get:7 http://security.debian.org/debian-security buster/updates/main amd64 Packages [286 kB]
#6 7.095 Get:8 http://deb.debian.org/debian buster-updates/main amd64 Packages [10.9 kB]
#6 8.058 Fetched 17.2 MB in 8s (2243 kB/s)
#6 8.058 Reading package lists...
#6 9.166 E: flAbsPath on /var/lib/dpkg/status failed - realpath (2: No such file or directory)
#6 9.166 E: Could not open file  - open (2: No such file or directory)
#6 9.166 E: Problem opening
#6 9.166 E: The package lists or status file could not be parsed or opened.

The problem seems to be from completely wiping out /var/lib/dpkg in the upstream Dockerfile:

datasette/Dockerfile

Line 18 in 1b69753

rm -rf /var/lib/dpkg

I've tested without removing the directory and apt works as expected.

[simonw]

This is a good catch, thank you.

[simonw]

root@33c269900821:/# du -h /var/lib/dpkg
6.3M	/var/lib/dpkg/info
24K	/var/lib/dpkg/triggers
4.0K	/var/lib/dpkg/updates
20K	/var/lib/dpkg/alternatives
4.0K	/var/lib/dpkg/parts
6.7M	/var/lib/dpkg

So /var/lib/dpkg/info is most of the space, I'll try just deleting that.

[simonw]

I'm testing tweaks to the Dockerfile by editing it and running:

docker build . -t datasette-057a0 --build-arg VERSION=0.57a0

Then running this to inspect the new image:

docker run -it -p 8001:8001 datasette-057a0 bash

After changing that line to rm -rf /var/lib/dpkg/info/* I can run apt-get successfully from inside that bash prompt:

# apt-get update
# apt-get install ripgrep
# rg --version
ripgrep 12.1.1

[simonw]

Added some new documentation about that here: https://github.com/simonw/datasette/blob/c0a748e5c3f498fa8c139b420d07dd3dea612379/docs/installation.rst#installing-plugins