Add date_nanos support to Time field #151374
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ersin-erdal
added
release_note:fix
Team:ResponseOps
|
Pinging @elastic/response-ops (Team:ResponseOps) |
|
When creating and running the ES query rule with the date_nanos timefield, I get this error after a few executions: |
ersin-erdal
commented
Feb 20, 2023
There was a problem hiding this comment.
As i can't test with KQL due to #151609, blocking the merge until the bug is resolved.
💚 Build Succeeded
Metrics [docs]Async chunks
Page load bundle
History
To update your PR or re-run it, just comment with: |
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this pull request
Feb 21, 2023
Fixes: elastic#145855 `Time field` in rule update/create forms shows only the fields with `date` data type. This PR adds `date_nanos` type as well. ## To verify: - Create an index that has fields with `date` and `date_nanos` types. ``` PUT test-index-date { "mappings": { "properties": { "date": { "type": "date" }, "timestamp": { "type": "date_nanos" } } } } ``` - Add data with `date` and `date_nanos` data types Note: You can use https://www.site24x7.com/tools/time-stamp-converter.html address to generate date_nanos timestamps with a date you desired. ``` PUT test-index-date/_bulk?refresh { "index" : { "_id" : "1" } } { "date": "2023-02-15T18:00:00.000Z", "timestamp": 1676480400000} { "index" : { "_id" : "2" } } { "date": "2023-02-15T18:00:00.000Z", "timestamp": 1676480400000 } ``` - Open a Rule create form and select `Elasticsearch query` - Select `Query DSL` - Click on `select an index` - Type the above index name (test-index-date) in `Indices to query` input - Select `timestamp` from time field - Click on `Test Query` button and expect to see `Query matched 2 documents in the last 5m.` (cherry picked from commit 58204cf)
💚 All backports created successfully
Note: Successful backport PRs will be merged automatically after passing CI. Questions ?Please refer to the Backport tool documentation |
kibanamachine
added a commit
that referenced
this pull request
Feb 21, 2023
# Backport This will backport the following commits from `main` to `8.7`: - [Add date_nanos support to Time field (#151374)](#151374) <!--- Backport version: 8.9.7 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Ersin Erdal","email":"[email protected]"},"sourceCommit":{"committedDate":"2023-02-21T15:25:09Z","message":"Add date_nanos support to Time field (#151374)\n\nFixes: #145855 \r\n\r\n`Time field` in rule update/create forms shows only the fields with\r\n`date` data type. This PR adds `date_nanos` type as well.\r\n\r\n## To verify:\r\n\r\n- Create an index that has fields with `date` and `date_nanos` types.\r\n```\r\nPUT test-index-date\r\n{\r\n \"mappings\": {\r\n \"properties\": {\r\n \"date\": {\r\n \"type\": \"date\"\r\n },\r\n \"timestamp\": {\r\n \"type\": \"date_nanos\"\r\n }\r\n }\r\n }\r\n}\r\n```\r\n- Add data with `date` and `date_nanos` data types\r\nNote: You can use\r\nhttps://www.site24x7.com/tools/time-stamp-converter.html address to\r\ngenerate date_nanos timestamps with a date you desired.\r\n```\r\nPUT test-index-date/_bulk?refresh\r\n{ \"index\" : { \"_id\" : \"1\" } }\r\n{ \"date\": \"2023-02-15T18:00:00.000Z\", \"timestamp\": 1676480400000} \r\n{ \"index\" : { \"_id\" : \"2\" } }\r\n{ \"date\": \"2023-02-15T18:00:00.000Z\", \"timestamp\": 1676480400000 } \r\n```\r\n- Open a Rule create form and select `Elasticsearch query`\r\n- Select `Query DSL`\r\n- Click on `select an index`\r\n- Type the above index name (test-index-date) in `Indices to query`\r\ninput\r\n- Select `timestamp` from time field\r\n- Click on `Test Query` button and expect to see `Query matched 2\r\ndocuments in the last 5m.`","sha":"58204cf2a0fd2b921d5f5c3e138b725bce1c5229","branchLabelMapping":{"^v8.8.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:fix","Team:ResponseOps","v8.7.0","v8.8.0"],"number":151374,"url":"https://github.com/elastic/kibana/pull/151374","mergeCommit":{"message":"Add date_nanos support to Time field (#151374)\n\nFixes: #145855 \r\n\r\n`Time field` in rule update/create forms shows only the fields with\r\n`date` data type. This PR adds `date_nanos` type as well.\r\n\r\n## To verify:\r\n\r\n- Create an index that has fields with `date` and `date_nanos` types.\r\n```\r\nPUT test-index-date\r\n{\r\n \"mappings\": {\r\n \"properties\": {\r\n \"date\": {\r\n \"type\": \"date\"\r\n },\r\n \"timestamp\": {\r\n \"type\": \"date_nanos\"\r\n }\r\n }\r\n }\r\n}\r\n```\r\n- Add data with `date` and `date_nanos` data types\r\nNote: You can use\r\nhttps://www.site24x7.com/tools/time-stamp-converter.html address to\r\ngenerate date_nanos timestamps with a date you desired.\r\n```\r\nPUT test-index-date/_bulk?refresh\r\n{ \"index\" : { \"_id\" : \"1\" } }\r\n{ \"date\": \"2023-02-15T18:00:00.000Z\", \"timestamp\": 1676480400000} \r\n{ \"index\" : { \"_id\" : \"2\" } }\r\n{ \"date\": \"2023-02-15T18:00:00.000Z\", \"timestamp\": 1676480400000 } \r\n```\r\n- Open a Rule create form and select `Elasticsearch query`\r\n- Select `Query DSL`\r\n- Click on `select an index`\r\n- Type the above index name (test-index-date) in `Indices to query`\r\ninput\r\n- Select `timestamp` from time field\r\n- Click on `Test Query` button and expect to see `Query matched 2\r\ndocuments in the last 5m.`","sha":"58204cf2a0fd2b921d5f5c3e138b725bce1c5229"}},"sourceBranch":"main","suggestedTargetBranches":["8.7"],"targetPullRequestStates":[{"branch":"8.7","label":"v8.7.0","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v8.8.0","labelRegex":"^v8.8.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/151374","number":151374,"mergeCommit":{"message":"Add date_nanos support to Time field (#151374)\n\nFixes: #145855 \r\n\r\n`Time field` in rule update/create forms shows only the fields with\r\n`date` data type. This PR adds `date_nanos` type as well.\r\n\r\n## To verify:\r\n\r\n- Create an index that has fields with `date` and `date_nanos` types.\r\n```\r\nPUT test-index-date\r\n{\r\n \"mappings\": {\r\n \"properties\": {\r\n \"date\": {\r\n \"type\": \"date\"\r\n },\r\n \"timestamp\": {\r\n \"type\": \"date_nanos\"\r\n }\r\n }\r\n }\r\n}\r\n```\r\n- Add data with `date` and `date_nanos` data types\r\nNote: You can use\r\nhttps://www.site24x7.com/tools/time-stamp-converter.html address to\r\ngenerate date_nanos timestamps with a date you desired.\r\n```\r\nPUT test-index-date/_bulk?refresh\r\n{ \"index\" : { \"_id\" : \"1\" } }\r\n{ \"date\": \"2023-02-15T18:00:00.000Z\", \"timestamp\": 1676480400000} \r\n{ \"index\" : { \"_id\" : \"2\" } }\r\n{ \"date\": \"2023-02-15T18:00:00.000Z\", \"timestamp\": 1676480400000 } \r\n```\r\n- Open a Rule create form and select `Elasticsearch query`\r\n- Select `Query DSL`\r\n- Click on `select an index`\r\n- Type the above index name (test-index-date) in `Indices to query`\r\ninput\r\n- Select `timestamp` from time field\r\n- Click on `Test Query` button and expect to see `Query matched 2\r\ndocuments in the last 5m.`","sha":"58204cf2a0fd2b921d5f5c3e138b725bce1c5229"}}]}] BACKPORT--> Co-authored-by: Ersin Erdal <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes: #145855
Time fieldin rule update/create forms shows only the fields withdatedata type. This PR addsdate_nanostype as well.To verify:
dateanddate_nanostypes.dateanddate_nanosdata typesNote: You can use https://www.site24x7.com/tools/time-stamp-converter.html address to generate date_nanos timestamps with a date you desired.
Elasticsearch queryQuery DSLselect an indexIndices to queryinputtimestampfrom time fieldTest Querybutton and expect to seeQuery matched 2 documents in the last 5m.