Upgrade fidesops base image from Debian Buster to Bullseye

[daveqnet]

fidesops is currently based on python:3.x.y-slim-buster. In August 2022 Debian Buster (10) is scheduled to move from official (oldstable) support to LTS community support.

The fidesops base image should be changed to python:3.x.y-slim-bullseye to ensure that timely security and version updates are applied to the application image.

[daveqnet]

Note: from some basic testing of a Docker build based on Bullseye, at least ipython (Enhanced interactive Python 2 shell) is affected. It seems to be unavailable in Debian releases newer than Buster. However, ipython3 (Enhanced interactive Python 3 shell) is available.

I'm going to unassign myself so that a dev can investigate further. This is exactly the same issue encountered in ethyca/fides#935.

username@hostname fidesops % docker build .
[+] Building 28.3s (11/25)                                                                                                                                                                                                                                                   
 => [internal] load build definition from Dockerfile                                                                                                                                                                                                                    0.0s
 => => transferring dockerfile: 2.66kB                                                                                                                                                                                                                                  0.0s
 => [internal] load .dockerignore                                                                                                                                                                                                                                       0.0s
 => => transferring context: 327B                                                                                                                                                                                                                                       0.0s
 => [internal] load metadata for docker.io/library/python:3.9.13-slim-bullseye                                                                                                                                                                                          2.2s
 => [internal] load metadata for docker.io/library/node:16                                                                                                                                                                                                              1.4s
 => [internal] load build context                                                                                                                                                                                                                                       2.3s
 => => transferring context: 13.39MB                                                                                                                                                                                                                                    2.3s
 => [backend  1/13] FROM docker.io/library/python:3.9.13-slim-bullseye@sha256:ea93ec4fbe8ee1c62397410c0d1f342a33199e98cd59adac6964b38e410e8246                                                                                                                          2.1s
 => => resolve docker.io/library/python:3.9.13-slim-bullseye@sha256:ea93ec4fbe8ee1c62397410c0d1f342a33199e98cd59adac6964b38e410e8246                                                                                                                                    0.0s
 => => sha256:d5b2724fe69a2c5a30f2de228be6523c5824a71f395b93f1daa143aae70e4dfa 11.58MB / 11.58MB                                                                                                                                                                        1.2s
 => => sha256:7965a1bc96734b52e0fb5ea62cdc8db7fa40f7ad4dac8d6f17929ac5ebe35e38 233B / 233B                                                                                                                                                                              0.4s
 => => sha256:203fb75034a0c93bcbc6ccec455d8a3c62ed141ae169fc308abf73a5704f20e8 3.17MB / 3.17MB                                                                                                                                                                          1.1s
 => => sha256:ea93ec4fbe8ee1c62397410c0d1f342a33199e98cd59adac6964b38e410e8246 1.86kB / 1.86kB                                                                                                                                                                          0.0s
 => => sha256:d0516f711fb211d961438b17a03e9a2fe4ae41cb67c554d0590872613512e769 1.37kB / 1.37kB                                                                                                                                                                          0.0s
 => => sha256:ae64b82339a839dcc559fcfe204ab23604476e8806c548d22ed7ab9615fb62fc 7.48kB / 7.48kB                                                                                                                                                                          0.0s
 => => extracting sha256:d5b2724fe69a2c5a30f2de228be6523c5824a71f395b93f1daa143aae70e4dfa                                                                                                                                                                               0.4s
 => => extracting sha256:7965a1bc96734b52e0fb5ea62cdc8db7fa40f7ad4dac8d6f17929ac5ebe35e38                                                                                                                                                                               0.0s
 => => extracting sha256:203fb75034a0c93bcbc6ccec455d8a3c62ed141ae169fc308abf73a5704f20e8                                                                                                                                                                               0.3s
 => CACHED [frontend 1/5] FROM docker.io/library/node:16@sha256:2e1b4542d4a06e0e0442dc38af1f4828760aecc9db2b95e7df87f573640d98cd                                                                                                                                        0.0s
 => [frontend 2/5] WORKDIR /fidesops/clients/admin-ui                                                                                                                                                                                                                   0.1s
 => ERROR [backend  2/13] RUN apt-get update &&     apt-get install -y --no-install-recommends     git     make     ipython     vim     curl     g++     gnupg     gcc     python3-wheel     && apt-get clean     && rm -rf /var/lib/apt/lists/*                       22.7s
 => [frontend 3/5] COPY clients/admin-ui/ .                                                                                                                                                                                                                             0.1s
 => CANCELED [frontend 4/5] RUN npm install                                                                                                                                                                                                                            23.6s
------
 > [backend  2/13] RUN apt-get update &&     apt-get install -y --no-install-recommends     git     make     ipython     vim     curl     g++     gnupg     gcc     python3-wheel     && apt-get clean     && rm -rf /var/lib/apt/lists/*:
#6 0.671 Get:1 http://deb.debian.org/debian bullseye InRelease [116 kB]
#6 0.777 Get:2 http://deb.debian.org/debian-security bullseye-security InRelease [48.4 kB]
#6 0.809 Get:3 http://deb.debian.org/debian bullseye-updates InRelease [44.1 kB]
#6 2.270 Get:4 http://deb.debian.org/debian bullseye/main amd64 Packages [8182 kB]
#6 6.786 Get:5 http://deb.debian.org/debian-security bullseye-security/main amd64 Packages [166 kB]
#6 6.934 Get:6 http://deb.debian.org/debian bullseye-updates/main amd64 Packages [2592 B]
#6 13.50 Fetched 8559 kB in 13s (658 kB/s)
#6 13.50 Reading package lists...
#6 17.42 Reading package lists...
#6 21.03 Building dependency tree...
#6 21.72 Reading state information...
#6 21.80 Package ipython is not available, but is referred to by another package.
#6 21.80 This may mean that the package is missing, has been obsoleted, or
#6 21.80 is only available from another source
#6 21.80 
#6 22.19 E: Package 'ipython' has no installation candidate
------
executor failed running [/bin/sh -c apt-get update &&     apt-get install -y --no-install-recommends     git     make     ipython     vim     curl     g++     gnupg     gcc     python3-wheel     && apt-get clean     && rm -rf /var/lib/apt/lists/*]: exit code: 100

[sanders41]

The ipython one is not a big deal. I talk to the team and everyone said they don't use it so we should be able to remove it all together. The bigger issue is mssql also has an issue. The drivers can't be installed in bullseye on M1 Macs. There is an issue about this, but Microsoft closed it without a solution so I'm not sure if they plan to fix it.

 > [backend  8/13] RUN if [ "$SKIP_MSSQL_INSTALLATION" != "true" ] ; then apt-get -y --no-install-recommends install     unixodbc-dev     msodbcsql17     mssql-tools     && apt-get clean     && rm -rf /var/lib/apt/lists/* ; fi:
#12 0.300 Reading package lists...
#12 2.766 Building dependency tree...
#12 3.240 Reading state information...
#12 3.341 Some packages could not be installed. This may mean that you have
#12 3.341 requested an impossible situation or if you are using the unstable
#12 3.341 distribution that some required packages have not yet been created
#12 3.341 or been moved out of Incoming.
#12 3.341 The following information may help to resolve the situation:
#12 3.341 
#12 3.341 The following packages have unmet dependencies:
#12 3.576  libodbc1 : PreDepends: multiarch-support but it is not installable
#12 3.577  odbcinst1debian2 : PreDepends: multiarch-support but it is not installable
#12 3.601 E: Unable to correct problems, you have held broken packages.
------
executor failed running [/bin/sh -c if [ "$SKIP_MSSQL_INSTALLATION" != "true" ] ; then apt-get -y --no-install-recommends install     unixodbc-dev     msodbcsql17     mssql-tools     && apt-get clean     && rm -rf /var/lib/apt/lists/* ; fi]: exit code: 100
make: *** [docker-build] Error 1

[daveqnet]

Thanks for looking at this so quickly @sanders41, and appreciate the quick removal of ipython.

Re mssql, it's odd that an m1 issue would affect the build given that --platform=linux/amd64 is included as a flag in the FROM instruction.

Just checking: did you update the url on line 42 of the Dockerfile to https://packages.microsoft.com/config/debian/11/prod.list before running the mssql build?

The build seems to complete okay for me on an m1 mac just now.

username@hostname fidesops % docker build --build-arg SKIP_MSSQL_INSTALLATION=false .
[+] Building 563.0s (28/28) FINISHED                                                                                                                                               
 => [internal] load build definition from Dockerfile                                                                                                                          0.0s
 => => transferring dockerfile: 37B                                                                                                                                           0.0s
 => [internal] load .dockerignore                                                                                                                                             0.0s
 => => transferring context: 35B                                                                                                                                              0.0s
 => [internal] load metadata for docker.io/library/python:3.9.13-slim-bullseye                                                                                                1.5s
 => [internal] load metadata for docker.io/library/node:16                                                                                                                    1.4s
 => [auth] library/python:pull token for registry-1.docker.io                                                                                                                 0.0s
 => [auth] library/node:pull token for registry-1.docker.io                                                                                                                   0.0s
 => [backend  1/13] FROM docker.io/library/python:3.9.13-slim-bullseye@sha256:ea93ec4fbe8ee1c62397410c0d1f342a33199e98cd59adac6964b38e410e8246                                0.0s
 => [internal] load build context                                                                                                                                             0.1s
 => => transferring context: 55.39kB                                                                                                                                          0.0s
 => [frontend 1/5] FROM docker.io/library/node:16@sha256:2e1b4542d4a06e0e0442dc38af1f4828760aecc9db2b95e7df87f573640d98cd                                                     0.0s
 => CACHED [backend  2/13] RUN apt-get update &&     apt-get install -y --no-install-recommends     git     make     vim     curl     g++     gnupg     gcc     python3-whee  0.0s
 => [backend  3/13] RUN echo "ENVIRONMENT VAR:  SKIP_MSSQL_INSTALLATION false"                                                                                                0.2s
 => CACHED [frontend 2/5] WORKDIR /fidesops/clients/admin-ui                                                                                                                  0.0s
 => CACHED [frontend 3/5] COPY clients/admin-ui/ .                                                                                                                            0.0s
 => CACHED [frontend 4/5] RUN npm install                                                                                                                                     0.0s
 => CACHED [frontend 5/5] RUN npm run export                                                                                                                                  0.0s
 => [backend  4/13] RUN if [ "false" != "true" ] ; then apt-get install -y --no-install-recommends apt-transport-https && apt-get clean && rm -rf /var/lib/apt/lists/* ; fi   0.5s
 => [backend  5/13] RUN if [ "false" != "true" ] ; then curl https://packages.microsoft.com/keys/microsoft.asc | apt-key add - ; fi                                          20.0s 
 => [backend  6/13] RUN if [ "false" != "true" ] ; then curl https://packages.microsoft.com/config/debian/11/prod.list | tee /etc/apt/sources.list.d/msprod.list ; fi        15.5s 
 => [backend  7/13] RUN if [ "false" != "true" ] ; then apt-get update ; fi                                                                                                  12.9s 
 => [backend  8/13] RUN if [ "false" != "true" ] ; then apt-get -y --no-install-recommends install     unixodbc-dev     msodbcsql17     mssql-tools     && apt-get clean      9.7s 
 => [backend  9/13] COPY requirements.txt dev-requirements.txt mssql-requirements.txt ./                                                                                      0.0s 
 => [backend 10/13] RUN pip install -U pip      && pip --no-cache-dir install -r requirements.txt -r dev-requirements.txt     && if [ "false" != "true" ] ; then pip --no-  451.6s 
 => [backend 11/13] COPY . /fidesops                                                                                                                                          0.1s 
 => [backend 12/13] WORKDIR /fidesops                                                                                                                                         0.0s 
 => [backend 13/13] RUN pip install -e .                                                                                                                                     48.2s 
 => [app 1/2] RUN mkdir -p /fidesops/src/fidesops/build/static/                                                                                                               0.2s 
 => [app 2/2] COPY --from=frontend /fidesops/clients/admin-ui/out/ /fidesops/src/fidesops/build/static/                                                                       0.0s 
 => exporting to image                                                                                                                                                        2.4s 
 => => exporting layers                                                                                                                                                       2.4s 
 => => writing image sha256:9baf0d1476f02599e3eb84c75847edb3d846dcb51ba24a011f842bbabd655a26                                                                                  0.0s 

[sanders41]

@daveqnet I did't notice that line. I'll give that a try.

[sanders41]

Updating line 42 got it to work.