Bash embedded jar as repackaging option

[aantono]

When trying to run the application from a jar that has been embedded into an executable bash script, I get ClassNotFoundException: org.springframework.context.support.LiveBeansView

java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:483)
    at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:53)
    at java.lang.Thread.run(Thread.java:744)
Caused by: java.lang.NoClassDefFoundError: org/springframework/context/support/LiveBeansView
    at org.springframework.context.support.AbstractApplicationContext.doClose(AbstractApplicationContext.java:876)
    at org.springframework.boot.context.embedded.EmbeddedWebApplicationContext.doClose(EmbeddedWebApplicationContext.java:152)
    at org.springframework.context.support.AbstractApplicationContext.close(AbstractApplicationContext.java:841)
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:335)
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:944)
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:933)
    at com.example.myproject.Application.main(Application.java:14)
    ... 6 more
Caused by: java.lang.ClassNotFoundException: org.springframework.context.support.LiveBeansView
    at java.net.URLClassLoader$1.run(URLClassLoader.java:372)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:360)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at org.springframework.boot.loader.LaunchedURLClassLoader.doLoadClass(LaunchedURLClassLoader.java:161)
    at org.springframework.boot.loader.LaunchedURLClassLoader.loadClass(LaunchedURLClassLoader.java:131)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    ... 13 more

When starting the application by using java -jar <jar_name> everything starts just fine.

The command to generate the executable bash is:
cat stub.sh build/libs/spring-boot-sandbox.jar > test.run

stub.sh:

#!/bin/sh
MYSELF=`which "$0" 2>/dev/null`
[ $? -gt 0 -a -f "$0" ] && MYSELF="./$0"
java=java
if test -n "$JAVA_HOME"; then
    java="$JAVA_HOME/bin/java"
fi
#jvm_opts=`unzip -p $MYSELF jvm.options 2>/dev/null`
#echo "Starting application using $jvm_opts"
#exec "$java" $java_args $jvm_opts -jar $MYSELF "$@"
exec "$java" $java_args -jar $MYSELF "$@"
exit 1

It seems that the problem stems with setting a custom SecurityManager

https://www.dropbox.com/s/tlo43vlqtcnh5bb/spring-boot-sandbox.jar - works fine
https://www.dropbox.com/s/dbyb1cd8yxfr3li/test.run - creates errors at startup

[philwebb]

Do you have the source code? I'm not too keen to run a random jar on my computer :)

[aantono]

Phil, can't share the code for this project, will see if I can create a single project that will replicate this issue. So far trying to move the NopSecurityManager out of the dependent jar into the spring-boot-sandbox build does not produce the error. So seems like the issue is even more convoluted and only revels itself when I try to do System.setSecurityManager from the static block declared inside the class from an inner dependency jars (one that lives inside the boot jar/lib directory)

[aantono]

Here are the relevant security classes that I can share the source of:
NopSecurityManager.java

package com.example.myproject.security;

import java.io.FileDescriptor;
import java.net.InetAddress;
import java.security.AccessController;
import java.security.Permission;

/**
 * NoOpSecurityManager has empty implementations of check methods that return void.  For other methods
 * the implementation is similar to that of java.lang.SecurityManager.
 *
 * The purpose of this class is to attempt to minimize Thread synchronization contention that we've observed
 * during load tests.
 */
public class NopSecurityManager extends SecurityManager {

    static {
        TrivialPolicy.init(); }

    public boolean getInCheck() {
        return false;
    }

    public Object getSecurityContext() {
        return AccessController.getContext();
    }

    /**
     * Update tests when writing logic for these methods.
     */

    public void checkPermission(Permission perm) {
    }

    public void checkPermission(Permission perm, Object context) {
    }

    public void checkCreateClassLoader() {
    }

    public void checkAccess(Thread t) {
    }

    public void checkAccess(ThreadGroup g) {
    }

    public void checkExit(int status) {
    }

    public void checkExec(String cmd) {
    }

    public void checkLink(String lib) {
    }

    public void checkRead(FileDescriptor fd) {
    }

    public void checkRead(String file) {
    }

    public void checkRead(String file, Object context) {
    }

    public void checkWrite(FileDescriptor fd) {
    }

    public void checkWrite(String file) {
    }

    public void checkDelete(String file) {
    }

    public void checkConnect(String host, int port) {
    }

    public void checkConnect(String host, int port, Object context) {
    }

    public void checkListen(int port) {
    }

    public void checkAccept(String host, int port) {
    }

    public void checkMulticast(InetAddress maddr) {
    }

    public void checkMulticast(InetAddress maddr, byte ttl) {
    }

    public void checkPropertiesAccess() {
    }

    public void checkPropertyAccess(String key) {
    }

    public boolean checkTopLevelWindow(Object window) {
        return true;
    }

    public void checkPrintJobAccess() {
    }

    public void checkSystemClipboardAccess() {
    }

    public void checkAwtEventQueueAccess() {
    }

    public void checkPackageAccess(String pkg) {
    }

    public void checkPackageDefinition(String pkg) {
    }

    public void checkSetFactory() {
    }

    public void checkMemberAccess(Class<?> clazz, int which) {
    }

    public void checkSecurityAccess(String target) {
    }

    public ThreadGroup getThreadGroup() {
        return Thread.currentThread().getThreadGroup();
    }

}

TrivialPolicy.java

package com.example.myproject.security;

import java.security.CodeSource;
import java.security.PermissionCollection;
import java.security.Policy;
import java.security.ProtectionDomain;

/**
 * Simple security policy implementation that allows all actions.
 *
 * This should be used in lieu of the Sun's PolicyFile class because it avoid lock contentions.  (COR-258)
 *
 * @author belliott
 */
public class TrivialPolicy extends Policy {

    private static final PermissionCollection permissionCollection = new AllPermissionCollection();

    public PermissionCollection getPermissions(CodeSource codesource) {
        return permissionCollection;
    }


    public PermissionCollection getPermissions(ProtectionDomain domain) {
        return permissionCollection;
    }

    public void refresh() {
        // no-op.
    }

    public static void init() {
        Policy.setPolicy(new TrivialPolicy());
    }

}

AllPermissionCollection.java

package com.example.myproject.security;

import sun.security.util.SecurityConstants;

import java.security.Permission;
import java.security.PermissionCollection;
import java.util.Enumeration;

/**
 * Simple permission collection granting the AllPermission.
 */
public class AllPermissionCollection extends PermissionCollection {

    private static final class AllPermissionEnumeration implements Enumeration<Permission> {
        public boolean hasMoreElements() {
            return false;
        }

        public Permission nextElement() {
            return SecurityConstants.ALL_PERMISSION;
        }
    }

    private static final long serialVersionUID = 1L;

    public AllPermissionCollection() {
        super();
        setReadOnly();
    }

    public void add(final Permission permission) {
        // add permission is a no-op.
    }

    public boolean implies(final Permission permission) {
        // always allow.
        return true;
    }

    /**
     * Return enumeration consisting of only the "all permission."
     *
     * @return Enumeration
     */
    public Enumeration<Permission> elements() {
        return new AllPermissionEnumeration();
    }
}

It seems that something weird only happens when I have

static {
        System.setSecurityManager(new NopSecurityManager());
    }

inside a class that lives in a different jar from where the NopSecurityManager class is packaged in.

[philwebb]

I've done a bit of digging. I think this might be the root cause of the problem:

http://hg.openjdk.java.net/jdk8/jdk8/jdk/file/687fd7c7986d/src/share/classes/sun/misc/URLClassPath.java#l707

[philwebb]

Try adding -Dsun.misc.URLClassPath.disableJarChecking=true on the line that invokes java -jar

[aantono]

Phil, that's exactly the cause. When setting the property, the test.run with embedded jar inside works just fine (as well as when calling explicitly java -jar spring-boot-sandbox.jar). What is VERY STRANGE, is why does java -jar spring-boot-sandbox.jar works just fine without setting any properties, but when the same jar content is being embedded into a test.run shell script, it errors out if the -Dsun.misc.URLClassPath.disableJarChecking=true is not set and only works if its present!???

Any thoughts or theories on what could cause this?

[philwebb]

I'm guessing that zipAccess.startsWithLocHeader(jar) just blindly looks at the first bytes of the JAR to see if it has a "Local file header signature". With spring-boot-sandbox.jar this is true, but when the shell script is added to the front it is not.

I didn't pick it up with any of the regular samples because they don't have a SecurityManager

[aantono]

Just verified that this indeed only happens when running under JDK 1.8, when starting under 1.7 (I've rebuilt the jar using 1.7) there is no classloading problem, it only occurs in 1.8 when the system prop is not set. :(

[philwebb]

Interesting. I wonder if we are able to programmatically add the property?

[aantono]

Would there be any possible undesirable side effects by not having the jar checking taking place?

[philwebb]

I can't foresee any, I wonder why the check was added in the first place?

[aantono]

This is what I found when looking for the implementation of the checker - http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/8-b132/java/util/zip/ZipFile.java#780
Apparently once can be set/overridden via static call to SharedSecrets.setJavaUtilZipFileAccess(JavaUtilZipFileAccess access) (http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/8-b132/sun/misc/SharedSecrets.java#162)

[aantono]

The check might try to do some security validation to try to prevent malicious jars that don't conform with strict format rules JDK imposes.

I guess, since this only applies to the embedded jars, we can always just hard-code the -Dsun.misc.URLClassPath.disableJarChecking=true into the stub.sh template.

So the template can look like this:

#!/bin/sh
MYSELF=`which "$0" 2>/dev/null`
[ $? -gt 0 -a -f "$0" ] && MYSELF="./$0"
java=java
if test -n "$JAVA_HOME"; then
    java="$JAVA_HOME/bin/java"
fi
jvm_opts=`unzip -p $MYSELF jvm.options 2>/dev/null`
echo "Starting application using the following jvm.options: $jvm_opts"
exec "$java" $JAVA_OPTS -Dsun.misc.URLClassPath.disableJarChecking=true $jvm_opts -jar $MYSELF "$@"
exit 1

BTW, the template above does on-the-fly extraction of jvm.options file from the root of the jar, so I've managed to successfully embed the startup jvm arguments inside the jar and then use them within the shell script to construct the cmd of the java exec. With this, I can just wire up the script to init.d and have the program automatically launch upon machine boot (vm, docker, physical, etc).

Would there be any interest in backing this into a gradle task as part of the gradle boot plugin to also be able to create not only executable jar but also an embedded runnable shell?

[dsyer]

Definitely there is interest. Maybe using some if the extended features of the Boot CLI shell wrapper for detecting JAVA_HOME etc (which we copied from groovy.sh). We would need Maven feature parity as well so pulling the code out into the shared tools would be a good plan.

[wilkinsona]

Via @spencergibb: https://github.com/brianm/really-executable-jars-maven-plugin

[spencergibb]

@wilkinsona I first found it while looking at Netflix/denominator build a long time ago, then searched google for a maven version.

[philwebb]

Tests are currently breaking on Windows:

build   10-Apr-2015 09:38:06    java.lang.UnsupportedOperationException: null
build   10-Apr-2015 09:38:06        at sun.nio.fs.WindowsFileSystemProvider.readAttributes(WindowsFileSystemProvider.java:192)
build   10-Apr-2015 09:38:06        at java.nio.file.Files.readAttributes(Files.java:1686)
build   10-Apr-2015 09:38:06        at java.nio.file.Files.getPosixFilePermissions(Files.java:1953)
build   10-Apr-2015 09:38:06        at org.springframework.boot.loader.tools.RepackagerTests.addLauncherScript(RepackagerTests.java:405)