Skip to content

Commit

Permalink
Merge pull request #295 from graalvm/tiny-containers-multi-stage
Browse files Browse the repository at this point in the history 
Tiny containers multi stage
  • Loading branch information
@shaunsmith
shaunsmith authored Oct 29, 2024
2 parents 228695d + 2b69cb8 commit 9bb3aad
Show file tree
Hide file tree
Showing 25 changed files with 247 additions and 253 deletions.
45 changes: 31 additions & 14 deletions .github/workflows/tiny-java-containers.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -23,24 +23,12 @@ jobs:
github-token: ${{ secrets.GITHUB_TOKEN }}
- name: Run 'tiny-java-containers'
run: |
cd tiny-java-containers
sleep_period=5
#
# Setup musl toolchain
#
./setup-musl.sh
export PATH="$PWD/musl-toolchain/bin:$PATH"
#
# Download upx
#
./setup-upx.sh
#
cd tiny-java-containers
# Hello World
#
cd helloworld
./build.sh
./hello
./hello.upx
docker run --rm hello:upx
cd ..
#
Expand All @@ -64,9 +52,12 @@ jobs:
curl "http://localhost:8000"
docker stop $container_id
#
# Static Scratch
# Static
#
./build-static.sh
#
# Static Scratch
#
container_id=$(docker run -d --rm -p8000:8000 jwebserver:scratch.static)
sleep $sleep_period
curl "http://localhost:8000"
Expand Down Expand Up @@ -100,3 +91,29 @@ jobs:
sleep $sleep_period
curl "http://localhost:8000"
docker stop $container_id
#
# JDK Build
#
./build-jvm.sh
#
# Debian JDK
#
container_id=$(docker run -d --rm -p8000:8000 jwebserver:debian)
sleep $sleep_period
curl "http://localhost:8000"
docker stop $container_id
#
# Eclipse Temurin Static
#
container_id=$(docker run -d --rm -p8000:8000 jwebserver:temurin)
sleep $sleep_period
curl "http://localhost:8000"
docker stop $container_id
#
# Distroless Java
#
./build-jlink.sh
container_id=$(docker run -d --rm -p8000:8000 jwebserver:distroless-java)
sleep $sleep_period
curl "http://localhost:8000"
docker stop $container_id
197 changes: 76 additions & 121 deletions tiny-java-containers/README.md
View file
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Tiny Java Containers

This example shows how a simple Java application and a simple web
This demo shows how a simple Java application and a simple web
server can be compiled to produce very small Docker container images.

The smallest container images contains just an executable. But since there's
Expand All @@ -13,128 +13,80 @@ To support static linking of `libc`, GraalVM Native Image supports using the
implementation.

You can watch a [Devoxx 2022](https://devoxx.be/) session that walks through
this example on YouTube.
an earlier version of this example on YouTube.

[![A 1.5MB Java Container
App](images/youtube.png)](https://youtu.be/6wYrAtngIVo)

## Prerequisites

* x86 Linux (but the few binary dependencies could easily be changed for aarch64)
* Docker installed and running. It should work fine with [podman](https://podman.io/) but it has not been tested.
* [GraalVM for JDK 21](https://www.graalvm.org/downloads/)
* Docker installed and running. It should work fine with
[podman](https://podman.io/) but it has not been tested.

> We recommend Oracle GraalVM for the best experience. It is licensed under the [GraalVM Free Terms and Conditions (GFTC)](https://www.oracle.com/downloads/licenses/graal-free-license.html) license, which permits use by any user including commercial and production use.
GraalVM Community Edition for JDK 21 works too, but Native Image generated executables sizes will differ.

> These instructions have only been tested on Linux x64.
> NOTE: These instructions have only been tested on Linux x64.
## Setup

You need the following zlib packages installed:
* zlib.x86_64
* zlib-devel.x86_64
* zlib-static.x86_64

On Oracle Linux, you can install with:
```sh
sudo yum install -y zlib.x86_64
sudo yum install -y zlib-devel.x86_64
sudo yum install -y zlib-static.x86_64
```

Clone this Git repo and in your Linux shell type the following to download and
configure the `musl` toolchain.

![](images/keyboard.jpg) `./setup-musl.sh`

Download [upx](https://upx.github.io/):

![](images/keyboard.jpg) `./setup-upx.sh`
Clone this Git repo. Everything runs in Docker so no need to install anything
on your machine.

## Hello World

With the `musl` toolchain installed, cd in to the `helloworld` folder.
Let's start with a simple Hello World example.

![](images/keyboard.jpg) `cd helloworld`
Change the directory to `helloworld`.

Using the `build.sh` script, compile a simple single Java class Hello World
application with `javac`, compile the generated .class file into a fully
statically linked native Linux executable named `hello`, compress the executable
with [upx](https://upx.github.io/) to create the executable `hello.upx`, and
package the compressed static `hello.upx` executable into a `scratch`
Docker container image:
![](images/keyboard.jpg) `cd helloworld`

![](images/keyboard.jpg) `./build.sh`
Use the `build.sh` script to run a Docker build that:
1. compiles a simple single Java class Hello World application with `javac`
2. compiles the generated .class file with GraalVM Native Image into a fully
statically linked native Linux executable named `hello`
3. compresses the executable with [upx](https://upx.github.io/) to create the
executable `hello.upx`
4. packages the compressed static `hello.upx` executable into a `scratch` Docker
container image

You'll see two executables were built:
In a terminal, run:

![](images/keyboard.jpg) `ls -lh hello*`
![](images/keyboard.jpg) `./build.sh`

### Native Executables

Running either of the `hello` executables you can see they are functionally
equivalent. They just print "Hello World". But there are a few points worth
noting:

1. The executable generated by GraalVM Native Image using the
`--static --libc=musl` options is a fully self-contained executable which can be
confirmed by examining it with `ldd`:

![](images/keyboard.jpg) `ldd hello`

should result in:

```shell
not a dynamic executable
```

This means that it does not rely on any libraries in the host operating system
environment making it easier to package in a variety of Docker container images.

Unfortunately `upx` compression renders `ldd` unable to list the shared
libraries of an executable, but since you compressed the statically linked
executable, you can be confident it is also statically linked.

2. Both executables are the result of compiling a Java bytecode application into
native machine code. The uncompressed executable is only ~6.3MB! There's no
JVM, no JARs, no JIT compiler and none of the overhead it imposes. Both
start extremely fast as there is minimal startup cost.

3. The `upx` compressed executable is over 70% smaller, 1.7MB vs. 6.3MB! With
`upx` the application self-extracts quickly but does incur a cost of about
100ms for decompression. See this blog for a deep dive on [GraalVM Native
Image and
1. The `hello` executable generated by GraalVM Native Image in the Dockerfile
using the `--static --libc=musl` options is a fully self-contained
executable. This means that it does not rely on any libraries in the host
operating system environment. This makes it easier to package in a variety of
container images.

2. You can see in the output of the Dockerfile build that `ls -lh` reports the
`hello` executable is ~4.9MB. There's no JVM, no JARs, no JIT compiler and
none of the overhead it imposes. It starts extremely fast as there is minimal
startup cost.

3. The `upx` compressed `hello.upx` executable is over 70% smaller, 1.3MB vs.
4.9MB! A `upx` compressed application self-extracts quickly but does incur a
cost of about 100ms for decompression. See this blog for a deep dive on
[GraalVM Native Image and
UPX](https://medium.com/graalvm/compressed-graalvm-native-images-4d233766a214).

### Container Images

The size of the `scratch`-based container image is slightly more than the `hello.upx`
executable.
The size of the `scratch`-based container image is about the same size as the
`hello.upx` executable since it adds little overhead.

![](images/keyboard.jpg) `docker images hello`

```shell
REPOSITORY TAG IMAGE ID CREATED SIZE
hello upx 4d122bd39a8a About a minute ago 1.78 MB
hello upx b69a5d79e8dc 1 second ago 1.3MB
```

This is a tiny container image and yet it contains a fully functional and
deployable (although fairly useless 😉) application. The Dockerfile that
generated it simply copies the executable into the container image and sets the
executable as the `ENTRYPOINT`.

A better way to build these images is with a multi-stage build, but to keep the
focus on the final result, build on a host machine and copy the binary into
the container image. E.g.,

```docker
FROM scratch
COPY hello.upx /
ENTRYPOINT ["/hello.upx"]
```
deployable (although fairly useless 😉) application.

Running the container image is straight forward:
Running the executable in the container image is straight forward:

![](images/keyboard.jpg) `docker run --rm hello:upx`

Expand All @@ -153,7 +105,7 @@ introduced in JDK 18 and build a containerized executable that serves up web
pages.

How small can a containerized Java web server be? Would you believe a measly
5.5MB? Let's see.
3.9 MB? Let's see.

Let's move from the `helloworld` folder over to the `jwebserver` folder.

Expand All @@ -171,62 +123,65 @@ custome runtime image for comparison.

![](images/keyboard.jpg) `./build-all.sh`

The various Dockerfiles simply copy the executable or `jlink` generated custom
runtime image folder into the container image along with an `index.html` file to
serve, and set the `ENTRYPOINT`. E.g.,
The various Dockerfiles simply copy the compiled executable or `jlink` generated
custom runtime image folder into the deployment container image along with an
`index.html` file to serve, and set the `ENTRYPOINT`.

```docker
FROM scratch
COPY jwebserver.static /
COPY index.html /web/index.html
ENTRYPOINT ["/jwebserver.static", "-b", "0.0.0.0", "-d", "/web"]
```
The Distroless Java, Eclipse Temurin, and Debian container images include a full
JDK, which includes jwebserver.

When complete you can see the sizes of the various versions:
When complete you can see the sizes of the various variants:

![](images/keyboard.jpg) `$ docker images jwebserver`

```shell
REPOSITORY TAG IMAGE ID CREATED SIZE
jwebserver distroless-java-base.jlink 414d84f8b7c7 22 minutes ago 132 MB
jwebserver scratch.static-upx 47aabdd14c04 22 minutes ago 4.71 MB
jwebserver alpine.static 783ab3a60248 22 minutes ago 23.4 MB
jwebserver distroless-static.static c894f14d4068 22 minutes ago 18.7 MB
jwebserver scratch.static 034cfbdf3577 22 minutes ago 15.7 MB
jwebserver distroless-base.mostly e99811e574d3 22 minutes ago 37.6 MB
jwebserver distroless-java-base.dynamic 72a210e3c705 23 minutes ago 50.6 MB
jwebserver distroless-java de7f7efb6df4 4 minutes ago 192MB
jwebserver temurin 643203bf8168 4 minutes ago 451MB
jwebserver debian fa5bfa4b2e5e 4 minutes ago 932MB
jwebserver distroless-java-base.jlink c3113c2400ea 5 minutes ago 122MB
jwebserver scratch.static-upx 75b3bb3249f3 5 minutes ago 3.9MB
jwebserver alpine.static 178081760470 6 minutes ago 21.6MB
jwebserver distroless-static.static 84053f6323c1 6 minutes ago 15.8MB
jwebserver scratch.static 98061f48037c 6 minutes ago 13.8MB
jwebserver distroless-base.mostly b33fc99fbe2a 7 minutes ago 34.3MB
jwebserver distroless-java-base.dynamic 1aceeabbb329 7 minutes ago 46.9MB
```

Sorting by size, it's clear that the fully statically linked GraalVM Native
Image generated executable that's compressed and packaged on `scratch`
(`scratch.static-upx`) is the smallest at just 4.71MB, less than 4% of the size
(`scratch.static-upx`) is the smallest at just 3.9 MB, less than 4% of the size
of the `jlink` version (`distroless-java-base.jlink`) running on the JVM.

| Base Image | App Version | Size (MB) |
| -------------------- | ---------------------------------- | --------- |
| Distroless Java Base | jlink | 132.00 |
| Distroless Java Base | native *dynamic* linked | 50.60 |
| Distroless Base | native *mostly* static linked | 37.60 |
| Alpine | native *fully* static linked | 23.40 |
| Distroless Static | native *fully* static linked | 18.70 |
| Scratch | native *fully* static linked | 15.70 |
| Scratch | *compressed* native *fully* static | 4.71 |

Running a container image is straight forward, just remember to map the ports, e.g.:

![](images/keyboard.jpg) `docker run --rm -p8000:8000 jwebserver:scratch.static`
| Debian Slim + JDK | jwebserver included in the JDK | 932.00 |
| Eclipse Temurin | jwebserver included in the JDK | 451.00 |
| Distroless Java | jwebserver included in the JDK | 192.00 |
| Distroless Java Base | jlink | 122.00 |
| Distroless Java Base | native *dynamic* linked | 46.90 |
| Distroless Base | native *mostly* static linked | 34.30 |
| Alpine | native *fully* static linked | 21.60 |
| Distroless Static | native *fully* static linked | 15.80 |
| Scratch | native *fully* static linked | 13.80 |
| Scratch | *compressed* native *fully* static | 3.90 |

Running a container image is once again straight forward, just remember to map
the server port, e.g.:

![](images/keyboard.jpg) `docker run --init --rm -p8000:8000 jwebserver:scratch.static`

or

![](images/keyboard.jpg) `docker run --rm -p8000:8000 jwebserver:scratch.static-upx`
![](images/keyboard.jpg) `docker run --init --rm -p8000:8000 jwebserver:scratch.static-upx`

Using `curl` or your favourite tool you can hit `http://localhost:8000` to fetch
the index.html file.

## Wrapping Up

A fully functional, albeit minimal, Java "microservice" was compiled
into a native Linux executable and packaged into Distroless, Alpine, and
A fully functional, albeit minimal, Java "microservice" was compiled into a
native Linux executable and packaged into Distroless, Alpine, and
`scratch`-based container images thanks to GraalVM Native Image's support for
various linking options including fully static linking with the `musl` libc.

Expand Down
1 change: 0 additions & 1 deletion tiny-java-containers/clean.sh
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@

set +e

rm -rf x86_64-linux-musl-native zlib-*
cd helloworld
./clean.sh || true
cd ../jwebserver
Expand Down
28 changes: 27 additions & 1 deletion tiny-java-containers/helloworld/Dockerfile
View file
Original file line number Diff line number Diff line change
@@ -1,3 +1,29 @@

# Build in a container with Oracle GraalVM Native Image and MUSL
FROM container-registry.oracle.com/graalvm/native-image:23-muslib AS nativebuild
WORKDIR /build
# Install UPX
ARG UPX_VERSION=4.2.2
ARG UPX_ARCHIVE=upx-${UPX_VERSION}-amd64_linux.tar.xz
RUN microdnf -y install wget xz && \
wget -q https://github.com/upx/upx/releases/download/v${UPX_VERSION}/${UPX_ARCHIVE} && \
tar -xJf ${UPX_ARCHIVE} && \
rm -rf ${UPX_ARCHIVE} && \
mv upx-${UPX_VERSION}-amd64_linux/upx . && \
rm -rf upx-${UPX_VERSION}-amd64_linux

# Compile the Hello class to Java bytecode
COPY Hello.java Hello.java
RUN javac Hello.java
# Build a native executable with native-image
RUN native-image -Os --static --libc=musl Hello -o hello
RUN ls -lh hello

# Compress the executable with UPX
RUN ./upx --lzma --best -o hello.upx hello
RUN ls -lh hello.upx

# Copy the compressed executable into a scratch container
FROM scratch
COPY hello.upx /
COPY --from=nativebuild /build/hello.upx /hello.upx
ENTRYPOINT ["/hello.upx"]
Loading

0 comments on commit 9bb3aad

Please sign in to comment.