Security optimizations for the container base image #10672

poikilotherm · 2024-07-08T13:32:19Z

What this PR does / why we need it:
This issue addressed some flaws I found that are security related as well as enabling better compatibility with OpenShift and Kubernetes.

Which issue(s) this PR closes:

Special notes for your reviewer:
None

Suggestions on how to test this:
Run them locally in Docker:

mvn -Pct -f modules/container-base clean package -Dbase.image.tag=10508
mvn -Pct clean package docker:run -Dbase.image.tag=10508

To test changing the passwords, you need to add the documented settings to the compose file as env vars.

Does this PR introduce a user interface change? If mockups are available, please link/include them here:
Nope

Is there a release notes update needed for this change?:
Included.

Additional documentation:
🔋 included

The Ubuntu specific wrappers "adduser" and "addgroup" have been removed with Ubuntu 24.04. Also, lets be more compatible with LSB this way.

It was left as "root" by accident, but should obviously be not.

Also delete some unused packages like gpg and dirmngr

- The entrypoint now defines two locations, which can be overridden by a user _without_ implicitely trying to execute these scripts. - The entrypoint now _removes_ any files found at these locations to _always_ start with a clean slate. Otherwise stale files might be looped over and over again. - A consequence of this: any kind of commands to be included must be provided via a script and cannot be provided by some initial file. - The configuration scripts no longer leave temporary files dangling and avoid these files if possible. Instead, we are injecting statements into these files while checking for duplicates as a safety measure.

…application Setting these env vars when reloading are highly application specific and shall not reside in the base image.

- Provide env vars for admin, linux user and domain master password. - These are set to the publicly known values, good enough for development or demo purposes. - For production purposes, these variables will be used to change passwords at run/boottime of the container. - As of this commit, do _not_ leave any password lying around in files, which could be exploited.

A new init script allows to set passwords at boot time of the container. If the passwords are not changed, there will be warnings logged about the default in use. Slightly modifying the startInForeground.sh script to avoid keeping password files or sensitive passwords around after starting the server.

coveralls · 2024-07-15T12:06:55Z

coverage: 20.623% (+0.7%) from 19.882%
when pulling aba7ab4 on 10508-base-image-fixes
into 5ba74e8 on develop.

Aligning configbaker and base image with same tool. wait4x has much more features to wait for different services.

As per https://docs.docker.com/reference/dockerfile/#automatic-platform-args-in-the-global-scope BuildKit / buildx will expose the target architecture. It requires adding an ARG in the Dockerfile to inject the data.

jattach binary is now available for ARM64 and AMD64, but requires special handling with download URLs and checksums.

Variable names related to user, password, and domain in Dockerfile and scripts have been modified for better clarity and consistency. This includes changing the names of admin user and password, domain master password, and Linux password and user.

Also includes a release note

poikilotherm · 2024-07-15T13:31:27Z

I think this is ready for review! Putting it in ready for sprint, as I don't know how fast we can act on it.

pdurbin · 2024-07-25T13:18:17Z

Unfortunately, I get the dreaded [INFO] DOCKER> unknown flag: --driver error when I try to run mvn -Pct -f modules/container-base clean package -Dbase.image.tag=10508 on my Mac.

We saw this error before:

DOCKER> unknown flag: --driver with M1 Mac #9771

To fix this, I bumped to this version: <fabric8-dmp.version>0.44.0</fabric8-dmp.version>. See 146c927.

poikilotherm · 2024-07-25T13:19:13Z

~~We can bump the DMP version in this PR, too.~~ You already did 😄

pdurbin · 2024-07-25T13:20:01Z

Hmm, when I run mvn -Pct clean package docker:run -Dbase.image.tag=10508 I get an error:

[INFO] --- docker:0.44.0:build (default-build) @ dataverse ---
[INFO] Reading assembly descriptor: /Users/PDurbin/github/iqss/dataverse/src/main/docker/assembly.xml
[INFO] Copying files to /Users/PDurbin/github/iqss/dataverse/target/docker/gdcc/dataverse/unstable/build/maven
[INFO] Building tar: /Users/PDurbin/github/iqss/dataverse/target/docker/gdcc/dataverse/unstable/tmp/docker-build.tar
[INFO] DOCKER> [gdcc/dataverse:unstable] "dev_dataverse": Created docker-build.tar in 1 second 
[INFO] DOCKER> [gdcc/dataverse:unstable] "dev_dataverse": Built image sha256:45db6
[WARNING] DOCKER> gdcc/dataverse:unstable: Unable to remove image [sha256:56818] : {"message":"conflict: unable to delete 568185d0af61 (cannot be forced) - image is being used by running container e4fdb985b8bf"} (Conflict: 409) (dangling image) [{"message":"conflict: unable to delete 568185d0af61 (cannot be forced) - image is being used by running container e4fdb985b8bf"} (Conflict: 409)]
[INFO] Reading assembly descriptor: /Users/PDurbin/github/iqss/dataverse/modules/container-configbaker/assembly.xml
[INFO] Copying files to /Users/PDurbin/github/iqss/dataverse/target/docker/gdcc/configbaker/unstable/build/maven
[INFO] Building tar: /Users/PDurbin/github/iqss/dataverse/target/docker/gdcc/configbaker/unstable/tmp/docker-build.tar
[INFO] DOCKER> [gdcc/configbaker:unstable] "dev_bootstrap": Created docker-build.tar in 36 milliseconds
[ERROR] DOCKER> Unable to inspect image [solr:] : {"message":"invalid reference format"} (Bad Request: 400) [{"message":"invalid reference format"} (Bad Request: 400)]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  18.042 s
[INFO] Finished at: 2024-07-25T09:18:57-04:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal io.fabric8:docker-maven-plugin:0.44.0:build (default-build) on project dataverse: Unable to inspect image [solr:] : {"message":"invalid reference format"} (Bad Request: 400) -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException

This reverts commit 146c927. We can't use 0.44 because of this error: "Unable to inspect image [solr:]" See https://github.com/fabric8io/docker-maven-plugin/issues/1756e

github-actions · 2024-07-25T15:11:37Z

📦 Pushed preview images as

ghcr.io/gdcc/dataverse:10508-base-image-fixes

ghcr.io/gdcc/configbaker:10508-base-image-fixes

🚢 See on GHCR. Use by referencing with full name as printed above, mind the registry name.

pdurbin

I tested this on Rocky 8 (my dev3 server) and it seems to work fine. Merging.

poikilotherm · 2024-07-25T16:49:28Z

Thanks for testing and merging! Much appreciated, as always! ❤️

pdurbin · 2024-07-25T17:58:22Z

Thanks for the PR! There's a lot in here! 😄

poikilotherm added 7 commits July 8, 2024 11:01

refactor(ct): migrate to useradd style commands in base image

d35a474

The Ubuntu specific wrappers "adduser" and "addgroup" have been removed with Ubuntu 24.04. Also, lets be more compatible with LSB this way.

fix(ct): security fix to use the unprivileged user in base image

4a49552

It was left as "root" by accident, but should obviously be not.

refactor(ct): use pkg names compatible with Ubuntu 22 and 24

7213c43

Also delete some unused packages like gpg and dirmngr

refactor(ct): move Dataverse specific tweaks for reload from base to …

bd0832a

…application Setting these env vars when reloading are highly application specific and shall not reside in the base image.

poikilotherm added Feature: Installation Guide Component: Containers Anything related to cloudy Dataverse, shipped in containers. Size: 3 A percentage of a sprint. 2.1 hours. Feature: Container Guide labels Jul 8, 2024

poikilotherm self-assigned this Jul 8, 2024

This comment has been minimized.

Sign in to view

Merge branch 'develop' into 10508-base-image-fixes

aba7ab4

This comment has been minimized.

Sign in to view

poikilotherm added 5 commits July 15, 2024 14:15

chore(ct): replace wait-for with wait4x

119d9eb

Aligning configbaker and base image with same tool. wait4x has much more features to wait for different services.

build(ct): make target architecture available in base image build

2176516

As per https://docs.docker.com/reference/dockerfile/#automatic-platform-args-in-the-global-scope BuildKit / buildx will expose the target architecture. It requires adding an ARG in the Dockerfile to inject the data.

chore(ct): upgrade base image with jattach v2.2

904229f

jattach binary is now available for ARM64 and AMD64, but requires special handling with download URLs and checksums.

docs(ct): add documentation about changing passwords and some more

5c7a91c

Also includes a release note

poikilotherm marked this pull request as ready for review July 15, 2024 13:31

This comment has been minimized.

Sign in to view

pdurbin unassigned poikilotherm Jul 15, 2024

cmbz added this to the 6.4 milestone Jul 18, 2024

poikilotherm mentioned this pull request Jul 24, 2024

Upgrade from Ubuntu 22 to 24 breaks container images #10722

Closed

This was referenced Jul 25, 2024

no-op to check status of mvn package and "base container image" workflow #10725

Closed

add local_lib mvn repo and primefaces themes, pin base image to Ubuntu 22 #10721

Closed

pdurbin self-assigned this Jul 25, 2024

bump to prevent "unknown flag: --driver" on Mac #10508 #9771

146c927

poikilotherm and others added 2 commits July 25, 2024 15:25

Merge branch 'develop' into 10508-base-image-fixes

610c184

Revert "bump to prevent "unknown flag: --driver" on Mac #10508 #9771"

4258900

This reverts commit 146c927. We can't use 0.44 because of this error: "Unable to inspect image [solr:]" See https://github.com/fabric8io/docker-maven-plugin/issues/1756e

pdurbin approved these changes Jul 25, 2024

View reviewed changes

pdurbin merged commit 1e7e9f1 into develop Jul 25, 2024
19 checks passed

pdurbin removed their assignment Jul 25, 2024

pdurbin deleted the 10508-base-image-fixes branch July 25, 2024 16:32

pdurbin mentioned this pull request Oct 24, 2024

Fixes dead Dataverse after server restart #10915

Closed

pdurbin mentioned this pull request Nov 4, 2024

Dead Dataverse after server restart #10998

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Security optimizations for the container base image #10672

Security optimizations for the container base image #10672

poikilotherm commented Jul 8, 2024 •

edited by pdurbin

Loading

This comment has been minimized.

This comment has been minimized.

coveralls commented Jul 15, 2024

poikilotherm commented Jul 15, 2024 •

edited

Loading

This comment has been minimized.

pdurbin commented Jul 25, 2024

poikilotherm commented Jul 25, 2024 •

edited

Loading

pdurbin commented Jul 25, 2024

github-actions bot commented Jul 25, 2024

pdurbin left a comment

poikilotherm commented Jul 25, 2024

pdurbin commented Jul 25, 2024

Security optimizations for the container base image #10672

Security optimizations for the container base image #10672

Conversation

poikilotherm commented Jul 8, 2024 • edited by pdurbin Loading

This comment has been minimized.

This comment has been minimized.

coveralls commented Jul 15, 2024

poikilotherm commented Jul 15, 2024 • edited Loading

This comment has been minimized.

pdurbin commented Jul 25, 2024

poikilotherm commented Jul 25, 2024 • edited Loading

pdurbin commented Jul 25, 2024

github-actions bot commented Jul 25, 2024

pdurbin left a comment

Choose a reason for hiding this comment

poikilotherm commented Jul 25, 2024

pdurbin commented Jul 25, 2024

poikilotherm commented Jul 8, 2024 •

edited by pdurbin

Loading

poikilotherm commented Jul 15, 2024 •

edited

Loading

poikilotherm commented Jul 25, 2024 •

edited

Loading