6.7 BWC layer for Java time formats in mappings does not default timezone to UTC

[droberts195]

It is documented that timestamps parsed by mappings that do not explicitly specify a timezone are considered to be UTC. In the master branch this behaviour from the days of Joda time seems to have been preserved in the new Java time parsing. However, in the 6.7 BWC layer trying to rely on the default of UTC timestamps parsed by mappings results in an exception.

Steps to reproduce.

First, to show it works as expected in master:

• git checkout master
• Start an ES test instance using ./gradlew run
• Run the following in a different terminal:

curl -H "Content-Type: application/json" -XPUT "localhost:9200/with_tz" -d '{
  "mappings": {
    "properties": {
      "my_time": {
        "type": "date",
        "format": "yyyy-MM-dd HH:mm:ssXX"
      }
    }
  }
}'

curl -H "Content-Type: application/json" -XPOST "localhost:9200/with_tz/_doc/1" -d '{
  "my_time": "2018-02-18 17:47:17Z"
}'

curl -H "Content-Type: application/json" -XPUT "localhost:9200/without_tz" -d '{
  "mappings": {
    "properties": {
      "my_time": {
        "type": "date",
        "format": "yyyy-MM-dd HH:mm:ss"
      }
    }
  }
}'

curl -H "Content-Type: application/json" -XPOST "localhost:9200/without_tz/_doc/1" -d '{
  "my_time": "2018-02-18 17:47:17"
}'

• Note that there are no exceptions and it is possible to index documents both with and without timezones in their timestamps when the Java 8 formats are set in the mappings.

Now to show that things are different in 6.7 when Java 8 timestamp formats are used:

• git checkout 6.7
• Start an ES test instance using ./gradlew run
• Run the following in a different terminal:

curl -H "Content-Type: application/json" -XPUT "localhost:9200/with_tz?include_type_name=false" -d '{
  "mappings": {
    "properties": {
      "my_time": {
        "type": "date",
        "format": "8yyyy-MM-dd HH:mm:ssXX"
      }
    }
  }
}'

curl -H "Content-Type: application/json" -XPOST "localhost:9200/with_tz/_doc/1" -d '{
  "my_time": "2018-02-18 17:47:17Z"
}'

curl -H "Content-Type: application/json" -XPUT "localhost:9200/without_tz?include_type_name=false" -d '{
  "mappings": {
    "properties": {
      "my_time": {
        "type": "date",
        "format": "8yyyy-MM-dd HH:mm:ss"
      }
    }
  }
}'

curl -H "Content-Type: application/json" -XPOST "localhost:9200/without_tz/_doc/1" -d '{
  "my_time": "2018-02-18 17:47:17"
}'

• The last command returns this error:

{"error":{"root_cause":[{"type":"mapper_parsing_exception","reason":"failed to parse field [my_time] of type [date] in document with id '1'"}],"type":"mapper_parsing_exception","reason":"failed to parse field [my_time] of type [date] in document with id '1'","caused_by":{"type":"date_time_exception","reason":"Unable to obtain Instant from TemporalAccessor: {},ISO resolved to 2018-02-18T17:47:17 of type java.time.format.Parsed","caused_by":{"type":"unsupported_temporal_type_exception","reason":"Unsupported field: InstantSeconds"}}},"status":400}

• The ES log contains this error:

[elasticsearch] Caused by: java.time.temporal.UnsupportedTemporalTypeException: Unsupported field: InstantSeconds
[elasticsearch]         at java.time.format.Parsed.getLong(Parsed.java:203) ~[?:?]
[elasticsearch]         at java.time.Instant.from(Instant.java:373) ~[?:?]
[elasticsearch]         at org.elasticsearch.common.time.DateFormatter.parseMillis(DateFormatter.java:51) ~[elasticsearch-6.7.0-SNAPSHOT.jar:6.7.0-SNAPSHOT]
[elasticsearch]         at org.elasticsearch.index.mapper.DateFieldMapper$DateFieldType.parse(DateFieldMapper.java:246) ~[elasticsearch-6.7.0-SNAPSHOT.jar:6.7.0-SNAPSHOT]
[elasticsearch]         at org.elasticsearch.index.mapper.DateFieldMapper.parseCreateField(DateFieldMapper.java:454) ~[elasticsearch-6.7.0-SNAPSHOT.jar:6.7.0-SNAPSHOT]
[elasticsearch]         at org.elasticsearch.index.mapper.FieldMapper.parse(FieldMapper.java:297) ~[elasticsearch-6.7.0-SNAPSHOT.jar:6.7.0-SNAPSHOT]
[elasticsearch]         ... 42 more

Unless I'm making a silly mistake I think this is another bug that needs to be fixed before 6.7 GA because otherwise the Java 8 timestamp BWC layer is not robust enough to use for other components of the stack that need to work with many different timestamp formats.

[elasticmachine]

Pinging @elastic/es-core-infra