Optimize tomcat-embed-core footprint

[sdeleuze]

tomcat-embed-programmatic is an experimental Tomcat dependency designed to lower the memory footprint, but it seems possible to reach similar or even lower footprint with the default tomcat-embed-core with a refined native configuration and an additional substitution.

My analysis tends to show that unused protocols are the main source of inefficiency, and more generally optional features should probably be removed from default Tomcat native configuration. As a consequence, I propose to modify tomcat-reflection.json as following:

• Move DefaultServlet entry to a new tomcat-reflection-default-servlet.json file
• Move Http11Nio2Protocol entry to a new tomcat-reflection-nio2.json file
• Move Http11AprProtocol & AprEndpoint entries to a new tomcat-reflection-apr.json file
• Move OpenSSLImplementation, SSLHostConfig, SSLHostConfigCertificate, OpenSSLConf, AbstractJsseEndpoint entries to a new tomcat-reflection-ssl.json file
• Add a { "name":"org.apache.coyote.AbstractProtocol", "methods" : [{"name": "setPort","parameterTypes":["int"] }] } entry (mandatory and previously covered as a side effect by Http11AprProtocol entry)

Those optional feature could then be enabled by users on demand with Args = -H:ReflectionConfigurationResources=/META-INF/native-image/org.apache.tomcat.embed/tomcat-embed-core/tomcat-reflection-nio2.json for example in a /META-INF/native-image/native-image.properties file. This principle is already use for tomcat-jni.json which is not included by default.

This will require on Spring side the following substitution because Tomcat ProtocolHandler completly defeats GraalVM static analysis:

@TargetClass(className = "org.apache.coyote.ProtocolHandler", onlyWith = { OnlyIfPresent.class })
final class Target_ProtocolHandler {

	@Substitute
	public static ProtocolHandler create(String protocol, boolean apr)
			throws ClassNotFoundException, InstantiationException, IllegalAccessException,
			IllegalArgumentException, InvocationTargetException, NoSuchMethodException, SecurityException {
		return new org.apache.coyote.http11.Http11NioProtocol();
	}
}

Notice that we can't move forward without a Tomcat update because --exclude-config in native-image.properties is likely broken, see also this NBT issue.

Unrelated, but likely worth to fix, org.apache.tomcat.util.threads.res.LocalStrings should probably be removed from tomcat-resource.json since there is an error message saying it does not exists.

@fhanik Could you please provide your feedback on the proposed changes, and if it is possible to perform them on Tomcat 9 branch or if we will have to wait Tomcat 10.1?

[fhanik]

@sdeleuze I've spent some time on this, and I agree with the general direction. I think it's important to capture what has taken place. Here is a review (all using the same tomcat binaries built with Java 8 sha 338d05d

Graal 19.2 / Java 8

Jar File | RSS Memory | Image Size | Build Time | Build Memory
progr:   | 17.9M      | 16.8M      | 24s        | ???G
core :   | 20.8M      | 21.8M      | 30s        | ???G

GraalVM Version 19.2.1 CE
OpenJDK 64-Bit GraalVM CE 19.2.1 (build 25.232-b07-jvmci-19.2-b03, mixed mode)

Graal 19.3 / Java 8

Jar File | RSS Memory | Image Size | Build Time | Build Memory
progr:   | 18.0M      | 17.6M      | 26s        | ???G
core :   | 21.0M      | 23.0M      | 33s        | ???G

GraalVM Version 19.3.1 CE
OpenJDK 64-Bit GraalVM CE 19.3.1 (build 25.242-b06-jvmci-19.3-b07, mixed mode)

Graal 19.3 / Java 11

Jar File | RSS Memory | Image Size | Build Time | Build Memory
progr:   | 20.6M      | 21.6M      | 32s        | ???G
core :   | 23.8M      | 27.2M      | 36s        | ???G

GraalVM Version 19.3.1 CE
OpenJDK 64-Bit Server VM GraalVM CE 19.3.1 (build 11.0.6+9-jvmci-19.3-b07, mixed mode, sharing)

Graal 20.2 / Java 11

Jar File | RSS Memory | Image Size | Build Time | Build Memory
progr:   | 24.4M      | 31.3M      | 38s        | 5.5G
core :   | 27.0M      | 36.6M      | 44s        | 5.8G

GraalVM Version 20.2.0 (Java Version 11.0.8)
OpenJDK 64-Bit Server VM GraalVM CE 20.2.0 (build 11.0.8+10-jvmci-20.2-b03, mixed mode, sharing)

Graal 21.3 / Java 11

Jar File | RSS Memory | Image Size | Build Time | Build Memory
progr:   | 30.2M      | 25.0M      | 25s        | 4.4G
core :   | 36.2M      | 40.3M      | 38s        | 6.4G

GraalVM 21.3.0 Java 11 CE (Java Version 11.0.13+7-jvmci-21.3-b05)
OpenJDK 64-Bit Server VM GraalVM CE 21.3.0 (build 11.0.13+7-jvmci-21.3-b05, mixed mode, sharing)

Graal 22.0 / Java 17

Jar File | RSS Memory | Image Size | Build Time | Build Memory
progr:   | 30.0M      | 26.3M      | 30s        | 6.4G
core :   | 35.2M      | 41.6M      | 34s        | 7.7G

GraalVM 22.0.0.2 Java 17 CE (Java Version 17.0.2+8-jvmci-22.0-b05)
OpenJDK 64-Bit Server VM GraalVM CE 22.0.0.2 (build 17.0.2+8-jvmci-22.0-b05, mixed mode, sharing)

[fhanik]

The path forward that I believe can start on is

1. Remove tomcat-embed-programmatic, it seems we can achieve this with graal precompiler solutions
2. Clean up tomcat-embed-xxx graal json files
3. Start working on @Substitute optimization (these should then be contributed to a JAR file called tomcat-embed-graal for Tomcat 10.1 while we can develop them inside of Spring native for Tomcat 9)

[sdeleuze]

How can we move forward on it, could you work on a Tomcat fork with the JSON refinements I proposed in the issue description? Are JSON refinements ok to be pushed on Tomcat 9 (this will likely break some native users) or should that be 10.1 only (so Spring Boot 3 only)?

About the figures, Java 8 to Java 11 RSS memory increase is well know, but I am surprised by the RSS increase on latest versions. Maybe something we want to analyze more closely.

[sdeleuze]

@fhanik After more thoughts, we should be able to generate the substitution on Spring side at AOT level based on Spring Boot configuration, so on Tomcat side you can let it as it is (especially given the fact Tomcat 10 removed some old protocols I think).

So I think you can focus on:

• My proposed refactoring of the config in the description of the issue
• Trying to identify the reason for the increased memory consumption on recent GraalVM versions.

[sdeleuze]

For protocols, we may be able to use conditional configuration and just rely on the framework level subtitution to trigger or not the config.

Something like:

{
   "condition" : { "typeReachable" : "org.apache.coyote.http11.Http11Nio2Protocol" },
   "name":"org.apache.coyote.http11.Http11Nio2Protocol",
   "methods" : [{"name": "<init>","parameterTypes":[]}]
},

Not sure, but something to test (I don't remember why those reflection entries are needed if ProtocolHandler create the instances programmatically).

[sdeleuze]

Ok so after a meeting with @fhanik and GraalVM team, we agreed on the following path:

1. Introduce an utility class in Tomcat designed like Spring Framework NativeDetector (source code) to provide system properties based boolean methods, for example:

public abstract class TomcatFeatureDetector {

	private static final boolean HTTP_11_NIO_PROTOCOL = (System.getProperty("tomcat.http11.nio.protocol") != null);

	private static final boolean HTTP_11_APR_PROTOCOL = (System.getProperty("tomcat.http11.apr.protocol") != null);

	public static boolean isHttp11NioProtocolEnabled() {
		return HTTP_11_NIO_PROTOCOL;
	}

	public static boolean isHttp11AprProtocolEnabled() {
		return HTTP_11_APR_PROTOCOL;
	}

}

2. Initialize this class at build time by shipping a /META-INF/native-image/{groupId}/{artifactId}/native-image.properties file with something like:

Args = --initialize-at-build-time=my.tomcat.package.TomcatFeatureDetector

(works for build time evaluation thanks to the inlining now enabled by default).
3) Modify ProtocolHandler#create (and other use cases) to something like:

public static ProtocolHandler create(String protocol, boolean apr)
            throws ClassNotFoundException, InstantiationException, IllegalAccessException,
            IllegalArgumentException, InvocationTargetException, NoSuchMethodException, SecurityException {
        if (protocol == null || "HTTP/1.1".equals(protocol)
                || (!apr && org.apache.coyote.http11.Http11NioProtocol.class.getName().equals(protocol))
                || (apr && org.apache.coyote.http11.Http11AprProtocol.class.getName().equals(protocol))) {
            if (apr && TomcatFeatureDetector.isHttp11AprProtocolEnabled()) {
                return new org.apache.coyote.http11.Http11AprProtocol();
            } else if (TomcatFeatureDetector.isHttp11NioProtocolEnabled()) {
                return new org.apache.coyote.http11.Http11NioProtocol();
            }
        ...
        } else {
            // Instantiate protocol handler
            Class<?> clazz = Class.forName(protocol);
            return (ProtocolHandler) clazz.getConstructor().newInstance();
        }
    }

4. Tomcat documentaton is updated to mention the system properties to use to control the removal of features at build time (and Spring Boot uses that).

5. @gradinac provides a documentation to explain how to use GraalVM 22.1 nightlies to generate automatically the reflection configuration automatically based on running the unit tests, and @fhanik create locally a modified Tomcat that uses Gradle or Maven to run the unit tests on native via https://github.com/graalvm/native-build-tools. The generated configuration with conditional configuration could then be shipped with Tomcat.

[sdeleuze]

@fhanik Any update?

[fhanik]

tomcat-embed-programmatic

Packaging webmvc-tomcat with Maven (native)
SUCCESS
Testing webmvc-tomcat
SUCCESS
Build memory: 7.07GB
Image build time: 49.3s
RSS memory: 67.3M
Image size: 54.1M
Startup time: 0.029 (JVM running for 0.03)
Lines of reflective config: 2487

tomcat-embed-core

=== Building webmvc-tomcat sample ===
Packaging webmvc-tomcat with Maven (native)
SUCCESS
Testing webmvc-tomcat
SUCCESS
Build memory: 7.80GB
Image build time: 49.0s
RSS memory: 69.4M
Image size: 58.9M
Startup time: 0.027 (JVM running for 0.028)
Lines of reflective config: 2487

There is a diminishing rate of return here.

I'm currently working on the reflection files, but I won't go into separation and optimization if it truly doesn't yield more than 2.1MB in a Spring MVC application. Strangely, the difference between the two JAR files is 6MB in a Hello World style Apache Tomcat application.

[cforce]

How far is this in context for spring boot and spring-boot-starter-web when using profile native? Will i get out of the box a smart decision?
What is about this number crunching when i use the dep spring-cloud-function-web?

[mhalbritter]

When you use Spring Boot 3 and starter-web, you'll not get the tomcat-embed-programmatic. It uses the default Tomcat. But you can switch to tomcat-embed-programmatic if you want, it's documented in the wiki.

[sdeleuze]

Notice Tomcat 8.0.3 will ship 1.5M less resources after this optimization I contirbuted. I plan to send other PRs to the Tomcat project to make tomcat-embed-programmatic non experimental anymore since it will probably be the best way to implement more aggressive optimizations. Since my PR has been merge and since we can't optimize much more tomcat-embed-core, I am closing this issue. Further optimizations will be made available via Spring Boot 3.