Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Native image does not inherit system time zone #5016

Closed
sgerr opened this issue Oct 30, 2019 · 29 comments
Closed

Native image does not inherit system time zone #5016

sgerr opened this issue Oct 30, 2019 · 29 comments
Labels
triage/out-of-date This issue/PR is no longer valid or relevant

Comments

@sgerr
Copy link

sgerr commented Oct 30, 2019

Native image does ignore system time zone, assumes GMT instead. Code snippet to verify:

@ApplicationScoped
public class Startup {
    
    public void startup(@Observes StartupEvent ev) {
        System.out.println("--- default time zone id is: " + ZoneId.systemDefault());
    }
    
}

Output (no matter system timezone is):

--- default time zone id is: GMT

@geoand
Copy link
Contributor

geoand commented Oct 30, 2019

@dmlloyd is this something you have seen before?

It sounds like a GraalVM bug, not a Quarkus one.

@dmlloyd
Copy link
Member

dmlloyd commented Oct 30, 2019

No, I have never seen this. I agree this looks like a GraalVM bug. I wonder if it doesn't load the zone database by default to save space.

@sgerr
Copy link
Author

sgerr commented Oct 31, 2019

Is there any workaround except of explicit specifying time zone every time it is needed?

@jaikiran
Copy link
Member

I can't reproduce this. I get the right timezone. I'm on Graal 19.2.0.1 (I haven't yet tested on 19.2.1). What version of Graal VM are you on and which Java vendor and version?

Also, can you paste the exact command you use to generate the native image?

@jaikiran
Copy link
Member

Tried on 19.2.1 of GraalVM. Can't reproduce there either - I get my system specific timezone as the value.

@sgerr
Copy link
Author

sgerr commented Oct 31, 2019

Hello, @jaikiran

I'm building native image using maven with docker on Windows 10, running it on Oracle Linux 7 or even Fedora 30. To reproduce:

  1. Create project

mvn io.quarkus:quarkus-maven-plugin:0.27.0:create -DprojectGroupId=com.sgerr -DprojectArtifactId=tztest -Dextensions="undertow" -Dpath=tz

  1. Create class Starter, use code snippet above.
  2. Build native image

mvnw package -Pnative -Dquarkus.native.container-build=true

Docker command is:

[INFO] [io.quarkus.deployment.pkg.steps.NativeImageBuildStep] docker run -v C:\work\sandbox\SandboxProjects\tztest\target\tztest-1.0-SNAPSHOT-native-image-source-jar:/project:z --rm quay.io/quarkus/ubi-quarkus-native-image:19.2.1 -J-Dsu
n.nio.ch.maxUpdateArraySize=100 -J-Djava.util.logging.manager=org.jboss.logmanager.LogManager -J-Dvertx.logger-delegate-factory-class-name=io.quarkus.vertx.core.runtime.VertxLogDelegateFactory -J-Dvertx.disableDnsResolver=true -J-Dio.ne
tty.leakDetection.level=DISABLED -J-Dio.netty.allocator.maxOrder=1 --initialize-at-build-time= -H:InitialCollectionPolicy=com.oracle.svm.core.genscavenge.CollectionPolicy$BySpaceAndTime -jar tztest-1.0-SNAPSHOT-runner.jar -J-Djava.util.
concurrent.ForkJoinPool.common.parallelism=1 -H:FallbackThreshold=0 -H:+ReportExceptionStackTraces -H:-AddAllCharsets -H:EnableURLProtocols=http -H:-JNI -H:-UseServiceLoaderFeature -H:+StackTrace tztest-1.0-SNAPSHOT-runner
  1. Copy and run it on Linux.
# date +%z && ./tztest -Dquarkus.http.port=9999
+0300
--- default time zone id is: GMT
2019-10-31 15:04:57,611 INFO  [io.quarkus] (main) tztest 1.0-SNAPSHOT (running on Quarkus 0.27.0) started in 0.016s. Listening on: http://0.0.0.0:9999
2019-10-31 15:04:57,611 INFO  [io.quarkus] (main) Profile prod activated.
2019-10-31 15:04:57,611 INFO  [io.quarkus] (main) Installed features: [cdi, resteasy, servlet]

@gsmet
Copy link
Member

gsmet commented Oct 31, 2019 via email

@dmlloyd
Copy link
Member

dmlloyd commented Oct 31, 2019

I'm looking at the source code and it looks like this:

@AutomaticFeature
final class TimeZoneFeature implements Feature {
    static class Options {
        @Option(help = "When true, all time zones will be pre-initialized in the image.")//
        public static final HostedOptionKey<Boolean> IncludeAllTimeZones = new HostedOptionKey<>(false);
    }

    @Override
    public void afterRegistration(AfterRegistrationAccess access) {
        TimeZone defaultZone = TimeZone.getDefault();
        String[] supportedZoneIDs = Options.IncludeAllTimeZones.getValue() ? TimeZone.getAvailableIDs() : new String[]{"GMT", "UTC", defaultZone.getID()};
        Map<String, TimeZone> supportedZones = Arrays.stream(supportedZoneIDs)
                        .map(TimeZone::getTimeZone)
                        .collect(toMap(TimeZone::getID, tz -> tz, (tz1, tz2) -> tz1));
        ImageSingletons.add(TimeZoneSupport.class, new TimeZoneSupport(supportedZones, defaultZone));
    }
}

So it looks to me like it's only including, by default, GMT, UTC, and whatever the build host's time zone is. So if the run host has a different time zone, you won't get it and it'll fall back to UTC. This should be verifiable by building a native image with a different value set for the TZ env. variable and then running it to see if it falls back to UTC.

Setting -H:+IncludeAllTimeZones should "fix" the problem but might inflate the image significantly.

@sgerr
Copy link
Author

sgerr commented Nov 1, 2019

Hello, @dmlloyd

Thank you for ypur reply. Please clarify, how to add -H:+IncludeAllTimeZones switch when maven build?

@sgerr
Copy link
Author

sgerr commented Nov 1, 2019

Oh, yes. Including in pom properties

<properties>
...
        <quarkus.native.additional-build-args>-H:+IncludeAllTimeZones</quarkus.native.additional-build-args>
    </properties>

will do that.

Command now is:

[INFO] [io.quarkus.deployment.pkg.steps.NativeImageBuildStep] docker run -v C:\work\sandbox\SandboxProjects\tztest\target\tztest-1.0-SNAPSHOT-native-image-source-jar:/project:z --rm quay.io/quarkus/ubi-quarkus-native-image:19.2.1 -J-Dsu
n.nio.ch.maxUpdateArraySize=100 -J-Djava.util.logging.manager=org.jboss.logmanager.LogManager -J-Dvertx.logger-delegate-factory-class-name=io.quarkus.vertx.core.runtime.VertxLogDelegateFactory -J-Dvertx.disableDnsResolver=true -J-Dio.ne
tty.leakDetection.level=DISABLED -J-Dio.netty.allocator.maxOrder=1 -H:+IncludeAllTimeZones --initialize-at-build-time= -H:InitialCollectionPolicy=com.oracle.svm.core.genscavenge.CollectionPolicy$BySpaceAndTime -jar tztest-1.0-SNAPSHOT-r
unner.jar -J-Djava.util.concurrent.ForkJoinPool.common.parallelism=1 -H:FallbackThreshold=0 -H:+ReportExceptionStackTraces -H:-AddAllCharsets -H:EnableURLProtocols=http -H:-JNI -H:-UseServiceLoaderFeature -H:+StackTrace tztest-1.0-SNAPS
HOT-runner

But still no effect:

date +%z && ./tztest -Dquarkus.http.port=9999
+0300
--- default time zone id is: GMT
2019-11-01 06:13:34,096 INFO  [io.quarkus] (main) tztest 1.0-SNAPSHOT (running on Quarkus 0.27.0) started in 0.019s. Listening on: http://0.0.0.0:9999
2019-11-01 06:13:34,097 INFO  [io.quarkus] (main) Profile prod activated.
2019-11-01 06:13:34,097 INFO  [io.quarkus] (main) Installed features: [cdi, resteasy, servlet]
2019-11-01 06:13:39,201 INFO  [io.quarkus] (main) tztest stopped in 0.005s

Is it because Windows build?

@sgerr
Copy link
Author

sgerr commented Nov 1, 2019

I just rebuild project on Fedora 30 (maven, opendjk 1.8, docker).

docker run -v /root/tztest/target/tztest-1.0-SNAPSHOT-native-image-source-jar:/project:z --user 0:0 --rm quay.io/quarkus/ubi-quarkus-native-image:19.2.1 -J-Dsun.nio.ch.maxUpdateArraySize=100 -J-Djava.util.logging.manager=org.jboss.logmanager.LogManager -J-Dvertx.logger-delegate-factory-class-name=io.quarkus.vertx.core.runtime.VertxLogDelegateFactory -J-Dvertx.disableDnsResolver=true -J-Dio.netty.leakDetection.level=DISABLED -J-Dio.netty.allocator.maxOrder=1 -H:+IncludeAllTimeZones --initialize-at-build-time= -H:InitialCollectionPolicy=com.oracle.svm.core.genscavenge.CollectionPolicy$BySpaceAndTime -jar tztest-1.0-SNAPSHOT-runner.jar -J-Djava.util.concurrent.ForkJoinPool.common.parallelism=1 -H:FallbackThreshold=0 -H:+ReportExceptionStackTraces -H:-AddAllCharsets -H:EnableURLProtocols=http -H:-JNI --no-server -H:-UseServiceLoaderFeature -H:+StackTrace tztest-1.0-SNAPSHOT-runner

The same result:

./target/tztest-1.0-SNAPSHOT-runner
--- default time zone id is: GMT
2019-11-01 07:33:45,352 INFO  [io.quarkus] (main) tztest 1.0-SNAPSHOT (running on Quarkus 0.27.0) started in 0.020s. Listening on: http://0.0.0.0:8080
2019-11-01 07:33:45,353 INFO  [io.quarkus] (main) Profile prod activated.
2019-11-01 07:33:45,353 INFO  [io.quarkus] (main) Installed features: [cdi, resteasy, servlet]

Build host's time zone is neither GMT no UTC.

@dmlloyd
Copy link
Member

dmlloyd commented Nov 1, 2019

It seems that the default time zone is not reset by GraalVM at native image start, so the time zone captured at build is retained. To fix it, there would have to be some startup hook which causes redetection of the default zone.

Furthermore there's a bug in that if you set the default time zone to null, you'll end up with lots of NPEs because the null handling logic is substituted away by GraalVM.

The class with the substitutions references an internal (secret) GraalVM bug ID GR-11844. I'm not sure what the contents of that bug ID are, but there are at least two major problems with their substitution class...

@sgerr
Copy link
Author

sgerr commented Nov 1, 2019

so the time zone captured at build is retained.

Unfortunatelly, it does not. At build time zone was neither GMT no UTC too.

So, no workaround on that except to specify time zone explicitly at start time or invoke date +%z on *NIX system and collect output.

@dmlloyd
Copy link
Member

dmlloyd commented Nov 1, 2019

Did you not say that you were building in a docker container? Are you setting the time zone env var in docker?

@sgerr
Copy link
Author

sgerr commented Nov 1, 2019

Are you setting the time zone env var in docker?

No. I'm using standard maven build procedure as I mentioned above:

mvnw package -Pnative -Dquarkus.native.container-build=true

@dmlloyd
Copy link
Member

dmlloyd commented Nov 1, 2019

I believe docker does not inherit the time zone from the host by default. See https://forums.docker.com/t/synchronize-timezone-from-host-to-container/39116/2

@sgerr
Copy link
Author

sgerr commented Nov 1, 2019

May be. But it does not matter because the point of our interest is the TZ of host machine at runtime.

@geoand
Copy link
Contributor

geoand commented Nov 1, 2019

@sgerr what @dmlloyd is saying essentially is that if you do mvnw package -Pnative (and thus don't use Docker) you should see at runtime the TZ of the machine that built the native binary.

Due to the fact that you are using docker to build the binary, you don't see that behavior because Docker (seemingly) doesn't use the hosts default TZ.

@sgerr
Copy link
Author

sgerr commented Nov 1, 2019

if you do mvnw package -Pnative (and thus don't use Docker) you should see at runtime the TZ of the machine that built the native binary.

@geoand, yes. But is it the behaviour you expected? Of course -- not. You are expecting to get host machine time zone at runtime. Moreover in some cases, for example storing timeseries to DB, you do not want to think about time zones at all because Hibernate is clever enought to do this work for you behind the scenes.

@geoand
Copy link
Contributor

geoand commented Nov 1, 2019

@sgerr so yes, of course the behavior is not expected.

What we are trying to do however is set the scene for what the actual problem can be and what is peripheral to the discussion.
@dmlloyd's analysis and your testing make it clear that the bug is in GraalVM, so I would propose you open an relevant issue in the project's issue tracker.

@sgerr
Copy link
Author

sgerr commented Nov 1, 2019

@geoand yes, sure. Meanwhile we can use quarkus in JVM mode where it works as expected. Thank you.

@geoand
Copy link
Contributor

geoand commented Nov 1, 2019

Thanks!

@gastaldi
Copy link
Contributor

gastaldi commented Nov 8, 2019

Related to #1842

@jaikiran
Copy link
Member

jaikiran commented Nov 8, 2019

I've raised a graal PR oracle/graal#1819 to see if we can have timezone support enhanced to allow more fine grained control on how users or Quarkus can configure the available timezones in native-image.

@gastaldi
Copy link
Contributor

gastaldi commented May 1, 2021

Can someone confirm that this is still an issue?

@aruffie
Copy link
Contributor

aruffie commented May 14, 2021

I confirm too, I have the bug here: quarkiverse/quarkus-univocity-parsers@5321516 the timezone is different between my non-native test phase and the native test phase. I think the "-H:+IncludeAllTimeZones" isn't take in account

@geoand
Copy link
Contributor

geoand commented Jan 11, 2022

@zakkak do you know if this is still an issue?

@zakkak
Copy link
Contributor

zakkak commented Jan 11, 2022

@zakkak do you know if this is still an issue?

This doesn't look like an issue any more:

import java.time.ZoneId;

public class Reproducer {
	public static void main(String[] args) {
		System.out.println("--- default time zone id is: " + ZoneId.systemDefault());
	}
}
$ javac Reproducer.java
# build with timezone set to `Europe/Paris`
$ TZ=Europe/Paris /opt/jvms/graalvm-ce-java11-21.3.0/bin/native-image Reproducer
# Run with host's timezone
$ ./reproducer
--- default time zone id is: Europe/London
# Run with various timezones
$ TZ=Europe/Dublin ./reproducer
--- default time zone id is: Europe/Dublin
$ TZ=Europe/Dublin23 ./reproducer
--- default time zone id is: GMT
$ TZ=Europe/London ./reproducer
--- default time zone id is: Europe/London
$ TZ=Europe/Paris ./reproducer
--- default time zone id is: Europe/Paris

@zakkak
Copy link
Contributor

zakkak commented May 6, 2022

Closing this. Please comment if you are still seeing the issue.

@zakkak zakkak closed this as completed May 6, 2022
@geoand geoand added the triage/out-of-date This issue/PR is no longer valid or relevant label May 6, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
triage/out-of-date This issue/PR is no longer valid or relevant
Projects
None yet
Development

No branches or pull requests

8 participants