XContent: Trying to use a writer with date formats results in serialization issues #31853

spinscale · 2018-07-06T08:09:35Z

fails under 6.3.1/6.3.0.

PUT foo/bar/1?refresh=true
{
  "key" : 123
}

GET foo/_search
{
  "script_fields": {
    "custom": {
      "script": {
        "source" : "return Instant.ofEpochMilli(123)"
      }
    }
  }
}

results in

# GET foo/_search
{
  "error": {
    "root_cause": [
      {
        "type": "illegal_argument_exception",
        "reason": "cannot write xcontent for unknown value of type class java.time.Instant"
      }
    ],
    "type": "illegal_argument_exception",
    "reason": "cannot write xcontent for unknown value of type class java.time.Instant"
  },
  "status": 400
}

exact stack trace

java.lang.IllegalArgumentException: cannot write xcontent for unknown value of type class java.time.Instant
	at org.elasticsearch.common.xcontent.XContentBuilder.unknownValue(XContentBuilder.java:755)
	at org.elasticsearch.common.xcontent.XContentBuilder.value(XContentBuilder.java:726)
	at org.elasticsearch.common.document.DocumentField.toXContent(DocumentField.java:131)
	at org.elasticsearch.search.SearchHit.toInnerXContent(SearchHit.java:456)
	at org.elasticsearch.search.SearchHit.toXContent(SearchHit.java:398)
	at org.elasticsearch.search.SearchHits.toXContent(SearchHits.java:117)
	at org.elasticsearch.action.search.SearchResponseSections.toXContent(SearchResponseSections.java:106)
	at org.elasticsearch.action.search.SearchResponse.innerToXContent(SearchResponse.java:241)
	at org.elasticsearch.action.search.SearchResponse.toXContent(SearchResponse.java:221)
	at org.elasticsearch.rest.action.RestStatusToXContentListener.buildResponse(RestStatusToXContentListener.java:57)
	at org.elasticsearch.rest.action.RestStatusToXContentListener.buildResponse(RestStatusToXContentListener.java:33)
	at org.elasticsearch.rest.action.RestToXContentListener.buildResponse(RestToXContentListener.java:42)
	at org.elasticsearch.rest.action.RestToXContentListener.buildResponse(RestToXContentListener.java:34)
	at org.elasticsearch.rest.action.RestResponseListener.processResponse(RestResponseListener.java:37)
	at org.elasticsearch.rest.action.RestActionListener.onResponse(RestActionListener.java:47)
	at org.elasticsearch.action.support.TransportAction$1.onResponse(TransportAction.java:85)
	at org.elasticsearch.action.support.TransportAction$1.onResponse(TransportAction.java:81)
	at org.elasticsearch.action.search.AbstractSearchAsyncAction.onResponse(AbstractSearchAsyncAction.java:308)
	at org.elasticsearch.action.search.AbstractSearchAsyncAction.onResponse(AbstractSearchAsyncAction.java:48)
	at org.elasticsearch.action.search.FetchSearchPhase$3.run(FetchSearchPhase.java:212)
	at org.elasticsearch.action.search.AbstractSearchAsyncAction.executePhase(AbstractSearchAsyncAction.java:155)
	at org.elasticsearch.action.search.AbstractSearchAsyncAction.executeNextPhase(AbstractSearchAsyncAction.java:148)
	at org.elasticsearch.action.search.ExpandSearchPhase.run(ExpandSearchPhase.java:118)
	at org.elasticsearch.action.search.AbstractSearchAsyncAction.executePhase(AbstractSearchAsyncAction.java:155)
	at org.elasticsearch.action.search.AbstractSearchAsyncAction.executeNextPhase(AbstractSearchAsyncAction.java:148)
	at org.elasticsearch.action.search.FetchSearchPhase.moveToNextPhase(FetchSearchPhase.java:205)
	at org.elasticsearch.action.search.FetchSearchPhase.lambda$innerRun$2(FetchSearchPhase.java:104)
	at org.elasticsearch.action.search.CountedCollector.countDown(CountedCollector.java:53)
	at org.elasticsearch.action.search.CountedCollector.onResult(CountedCollector.java:64)
	at org.elasticsearch.action.search.FetchSearchPhase$2.innerOnResponse(FetchSearchPhase.java:165)
	at org.elasticsearch.action.search.FetchSearchPhase$2.innerOnResponse(FetchSearchPhase.java:162)
	at org.elasticsearch.action.search.SearchActionListener.onResponse(SearchActionListener.java:45)
	at org.elasticsearch.action.search.SearchActionListener.onResponse(SearchActionListener.java:29)
	at org.elasticsearch.action.ActionListenerResponseHandler.handleResponse(ActionListenerResponseHandler.java:46)
	at org.elasticsearch.action.search.SearchTransportService$ConnectionCountingHandler.handleResponse(SearchTransportService.java:518)
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleResponse(TransportService.java:1096)
	at org.elasticsearch.transport.TransportService$DirectResponseChannel.processResponse(TransportService.java:1165)
	at org.elasticsearch.transport.TransportService$DirectResponseChannel.sendResponse(TransportService.java:1155)
	at org.elasticsearch.transport.TransportService$DirectResponseChannel.sendResponse(TransportService.java:1144)
	at org.elasticsearch.transport.TaskTransportChannel.sendResponse(TaskTransportChannel.java:54)
	at org.elasticsearch.action.search.SearchTransportService$11.messageReceived(SearchTransportService.java:441)
	at org.elasticsearch.action.search.SearchTransportService$11.messageReceived(SearchTransportService.java:437)
	at org.elasticsearch.xpack.security.transport.SecurityServerTransportInterceptor$ProfileSecuredRequestHandler$1.doRun(SecurityServerTransportInterceptor.java:259)
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
	at org.elasticsearch.xpack.security.transport.SecurityServerTransportInterceptor$ProfileSecuredRequestHandler.messageReceived(SecurityServerTransportInterceptor.java:317)
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:66)
	at org.elasticsearch.transport.TransportService$7.doRun(TransportService.java:664)
	at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:725)
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
	at org.elasticsearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:41)
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1135)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:844)
"

The text was updated successfully, but these errors were encountered:

elasticmachine · 2018-07-06T08:09:36Z

Pinging @elastic/es-core-infra

spinscale · 2018-07-06T15:40:57Z

the issue for this is, that the date transformers (where there is one for Instant) only are checked if you use writeTimeField(), however this one is just a regular writer, thus this fails, because nothing is registered for Instant.

This used to work under 6.2 and fails under 6.3

spinscale · 2018-07-08T12:42:08Z

Note that this change affects ML watches created via the Kibana UI, see elastic/kibana#18376

shahid21st · 2018-07-27T03:56:53Z

Similar thing happens in Java API, if I build a query with RangeQueryBuilder with to/from parameters using LocalDate and then do a toString() to get the translated query. This happens as XContentElasticsearchExtension has no transformers for LocalDate and also XContentBuilder has no writer for LocalDate.
For LocalDate or related classes it might be possible to check whether the object is an implementation of Temporal interface and invoke toString(). But looking at #32395 , it seems, it would need a proper solution to address all of these.

spinscale · 2018-08-22T12:16:33Z

this also affects users with watches that were created and properly running in 5.x and now fail under 6.3

This ensures that the java time class exposed by painless have proper serialization/string representations. Closes elastic#31853

BobBlank12 · 2018-08-28T16:45:44Z

Would this also cause watches created by an ML job in 6.3 to not execute properly as well? Error is:
[2018-08-27T20:40:51,497][ERROR][org.elasticsearch.xpack.watcher.input.search.ExecutableSearchInput] failed to execute [search] input for watch [ml-ml-test], reason [cannot write xcontent for unknown value of type class java.time.LocalDateTime]

BobBlank12 · 2018-08-28T17:23:36Z

Just answered my own question by reading more carefully in the thread.

This ensures that the java time class exposed by painless have proper serialization/string representations. Closes #31853

spinscale added >bug :Data Management/Watcher labels Jul 6, 2018

spinscale assigned hub-cap Jul 6, 2018

spinscale unassigned hub-cap Jul 6, 2018

spinscale changed the title ~~Watcher: Returning specific classes in a transform results in an exception~~ XContent: Trying to use a writer with date formats returns in serialization issues Jul 6, 2018

spinscale added :Core/Infra/Core Core issues without another label >regression and removed :Data Management/Watcher labels Jul 6, 2018

spinscale changed the title ~~XContent: Trying to use a writer with date formats returns in serialization issues~~ XContent: Trying to use a writer with date formats results in serialization issues Jul 8, 2018

ohadravid mentioned this issue Jul 26, 2018

Exception when using source filtering on a large integer field #32395

Closed

spinscale added the team-discuss label Aug 22, 2018

spinscale removed the team-discuss label Aug 24, 2018

spinscale added a commit to spinscale/elasticsearch that referenced this issue Aug 24, 2018

Core: Add java time xcontent serializers

e3b6f49

This ensures that the java time class exposed by painless have proper serialization/string representations. Closes elastic#31853

spinscale mentioned this issue Aug 24, 2018

Core: Add java time xcontent serializers #33120

Merged

spinscale closed this as completed in #33120 Aug 29, 2018

spinscale added a commit that referenced this issue Aug 29, 2018

Core: Add java time xcontent serializers (#33120)

48b388c

This ensures that the java time class exposed by painless have proper serialization/string representations. Closes #31853

spinscale added a commit that referenced this issue Aug 29, 2018

Core: Add java time xcontent serializers (#33120)

5c24164

This ensures that the java time class exposed by painless have proper serialization/string representations. Closes #31853

spinscale added a commit that referenced this issue Aug 29, 2018

Core: Add java time xcontent serializers (#33120)

f95f9df

This ensures that the java time class exposed by painless have proper serialization/string representations. Closes #31853

aravind-musigumpula mentioned this issue Apr 12, 2024

Exception while creating a PIT with high level client opensearch-project/OpenSearch#13144

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

XContent: Trying to use a writer with date formats results in serialization issues #31853

XContent: Trying to use a writer with date formats results in serialization issues #31853

spinscale commented Jul 6, 2018 •

edited

Loading

elasticmachine commented Jul 6, 2018

spinscale commented Jul 6, 2018 •

edited

Loading

spinscale commented Jul 8, 2018

shahid21st commented Jul 27, 2018

spinscale commented Aug 22, 2018 •

edited

Loading

BobBlank12 commented Aug 28, 2018

BobBlank12 commented Aug 28, 2018

XContent: Trying to use a writer with date formats results in serialization issues #31853

XContent: Trying to use a writer with date formats results in serialization issues #31853

Comments

spinscale commented Jul 6, 2018 • edited Loading

elasticmachine commented Jul 6, 2018

spinscale commented Jul 6, 2018 • edited Loading

spinscale commented Jul 8, 2018

shahid21st commented Jul 27, 2018

spinscale commented Aug 22, 2018 • edited Loading

BobBlank12 commented Aug 28, 2018

BobBlank12 commented Aug 28, 2018

spinscale commented Jul 6, 2018 •

edited

Loading

spinscale commented Jul 6, 2018 •

edited

Loading

spinscale commented Aug 22, 2018 •

edited

Loading