timestamp data type date nanoseconds causes rule to fail

[clwo]

Kibana version:
v8.6.2
Elasticsearch version:
v8.6.2
Describe the bug:
If a es-query rule is executed against a data view containing timestamps with data type Date nanoseconds instead of type Date the rule will fail if it matches documents. If no documents are matched the rule will succeed.

The following error is present in the Kibana logs:

[2023-05-04T10:50:22.808+00:00][ERROR][plugins.alerting.es-query] Executing Rule default:.es-query:fae14d00-e99d-11ed-be8c-a55fc9d78ed8 has resulted in Error: Invalid time value - RangeError: Invalid time value
    at Date.toISOString (<anonymous>)
    at tryToParseAsDate (/usr/share/kibana/x-pack/plugins/stack_alerts/server/rule_types/es_query/executor.js:191:31)
    at getValidTimefieldSort (/usr/share/kibana/x-pack/plugins/stack_alerts/server/rule_types/es_query/executor.js:182:22)
    at /usr/share/kibana/x-pack/plugins/stack_alerts/server/rule_types/es_query/executor.js:107:119
    at Array.find (<anonymous>)
    at executor (/usr/share/kibana/x-pack/plugins/stack_alerts/server/rule_types/es_query/executor.js:107:107)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at Object.executor (/usr/share/kibana/x-pack/plugins/stack_alerts/server/rule_types/es_query/rule_type.js:153:14)
    at /usr/share/kibana/x-pack/plugins/alerting/server/task_runner/task_runner.js:226:24
    at TaskRunnerTimer.runWithTimer (/usr/share/kibana/x-pack/plugins/alerting/server/task_runner/task_runner_timer.js:49:20)
    at TaskRunner.runRule (/usr/share/kibana/x-pack/plugins/alerting/server/task_runner/task_runner.js:193:9)
    at TaskRunner.run (/usr/share/kibana/x-pack/plugins/alerting/server/task_runner/task_runner.js:517:49)
    at TaskManagerRunner.run (/usr/share/kibana/x-pack/plugins/task_manager/server/task_running/task_runner.js:266:22)

Steps to reproduce:

1. Create index with @timestamp data type mapping to Date nanoseconds
2. In Kibana: create Elasticsearch query rule in the UI with KQL or Lucene query type matching documents in the data view for the prior created index.
3. Rule execution fails with the error above.

Expected behavior:
Rule execution should succeed and trigger alerts.

[elasticmachine]

Pinging @elastic/response-ops (Team:ResponseOps)

[elasticmachine]

Pinging @elastic/kibana-data-discovery (Team:DataDiscovery)

[ersin-erdal]

Hi,
This issue has been solved in version 8.7 with this PR.
Would it be possible for you to upgrade your Kibana to v8.7?

[kertal]

@ersin-erdal great that date_nanos support was integrated 👍

just wanted to ask, looking at the failure of here, are you sure this part of the code works with any data nanos value?

kibana/x-pack/plugins/stack_alerts/server/rule_types/es_query/executor.ts

Lines 126 to 128 in a1923c5

	const firstValidTimefieldSort = getValidTimefieldSort(
	result.hits.find((hit) => getValidTimefieldSort(hit.sort))?.sort
	);

It's using this function under the hood:

kibana/x-pack/plugins/stack_alerts/server/rule_types/es_query/executor.ts

Lines 225 to 230 in a1923c5

	export function tryToParseAsDate(sortValue?: string | number | null): undefined | string {
	const sortDate = typeof sortValue === 'string' ? Date.parse(sortValue) : sortValue;
	if (sortDate && !isNaN(sortDate)) {
	return new Date(sortDate).toISOString();
	}
	}

And this implementation high can high likely cause troubles with date_nanos, given the lack of support of 64bit values in JavaScript #40183

[ersin-erdal]

Hi @kertal,
We tested it with 1676480400000 (which was the "current" day back then :)) in the PR and it worked.
Could it be bigger than that?

[clwo]

Hi, This issue has been solved in version 8.7 with this PR. Would it be possible for you to upgrade your Kibana to v8.7?

Hi @ersin-erdal, I've upgraded a cluster to 8.7.1 and can confirm that it is working as expected. Thanks a lot :)

[ersin-erdal]

@clwo
I'm glad it worked.

@kertal
I realised that the number we tried was milliseconds not nano, but tried with nano as well : 1599999999999999999,
still works.

[kertal]

I see, how do you test it?
new Date(1599999999999999999).toISOString();
or
new Date(1676480400000000000).toISOString();

I'm getting an invalid date

[ersin-erdal]

You are right, i think i copied the wrong output...
Doesn't work with nanoseconds, we need to divide by 100000.
I will push a fix in 8.9

[ersin-erdal]

@kertal
This is interesting but i think ES does not support indexing nanoseconds!

When i create a test index with the below mappings and try to push data that has more than 13 digits to date_nanos filed i get a illegal_argument_exception error.

mapping:

{
  "mappings": {
    "properties": {
      "date": {
        "type": "date"
      },
      "date_nanos": {
        "type": "date_nanos"
      }
    }
  }
}

data:

PUT test-index-date/_bulk?refresh
{ "index" : { "_id" : "1" } }
{ "date": "2023-06-05T22:00:00.000Z", "date_nanos": 16366080300000 } 
{ "index" : { "_id" : "2" } }
{ "date": "2023-06-05T22:00:00.000Z", "date_nanos": 1636608030000 } 

error:

"caused_by": {
            "type": "illegal_argument_exception",
            "reason": "date[2488-08-14T05:25:00Z] is after 2262-04-11T23:47:16.854775807 and cannot be stored in nanosecond resolution"
          }

So the data format has to be milliseconds. That's why it worked for @clwo.
Therefore (IMHO) there is no need to support nanoseconds in the parser.

[ersin-erdal]

As shown in the documentation: https://www.elastic.co/guide/en/elasticsearch/reference/current/date_nanos.html

date_nanos field does not take a date as nanoseconds.
All the possible formats are: https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-date-format.html

As the possible biggest number that we can get is milliseconds there is no need to support nanoseconds (aka BigInt) in the code.

Closing the issue as the user's problem has also been solved by upgrading to 8.7.