Date format inconsistent between the Fields API and search_after

[majagrubic]

When discussing #67063 and its effects on the usage of the Fields API in Discover, we had agreed with Nik to explicitly request the date format with each date field.

This works fine for date_nanos mapping, but for date_nanos mapping with a custom date format, there seems to be an issue when passing the same date back in search_after. It looks like the same format is not accepted in search_after.

Eg:
For the mapping: https://github.com/elastic/kibana/blob/c42d23347a901ae7b7af5855eaddc8c2972762cc/test/functional/fixtures/es_archiver/date_nanos_custom/mappings.json#L18

If we read the date from the fields section when displaying the context view, we use the search_after in the query. The exception we get in this case is:

"statusCode":400,"attributes":{"root_cause":[{"type":"parse_exception","reason":"failed to parse date field [2019-10-21T08:30:04.828733Z] with format [yyyy-MM-dd HH:mm:ss.SSSSSS]

Specifying a format is one thing, but Kibana should not be doing a "normalization" of dates between taking an ES response and providing its values back to ES.

[elasticmachine]

Pinging @elastic/es-search (Team:Search)

[dnhatn]

Relates #67063

[jimczi]

The exception you're getting is expected. The format yyyy-MM-dd HH:mm:ss.SSSSSS cannot parse 2019-10-21T08:30:04.828733Z. If you want to parse with optional T, you should use strict_date_optional_time_nanos.
I tested the same example with search_after and the fields option and it worked fine with strict_date_optional_time_nanos so I hope you don't mind if I close this issue.