Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Advise against dates with decimal points (backport of #71578) #71653

Merged
merged 1 commit into from
Apr 13, 2021
Merged

Advise against dates with decimal points (backport of #71578) #71653

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
79 changes: 73 additions & 6 deletions docs/reference/mapping/types/date.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,11 +7,11 @@
JSON doesn't have a date data type, so dates in Elasticsearch can either be:

* strings containing formatted dates, e.g. `"2015-01-01"` or `"2015/01/01 12:10:30"`.
* a long number representing _milliseconds-since-the-epoch_.
* an integer representing _seconds-since-the-epoch_.
* a number representing _milliseconds-since-the-epoch_.
* a number representing _seconds-since-the-epoch_ (<<date-epoch-seconds, configuration>>).

NOTE: Values for _milliseconds-since-the-epoch_ and _seconds-since-the-epoch_
must be non-negative. Use a formatted date to represent dates before 1970.
NOTE: Values for _milliseconds-since-the-epoch_ must be non-negative. Use a
formatted date to represent dates before 1970.

Internally, dates are converted to UTC (if the time-zone is specified) and
stored as a long number representing milliseconds-since-the-epoch.
Expand All @@ -26,15 +26,19 @@ supplied as a long in the JSON document.
Date formats can be customised, but if no `format` is specified then it uses
the default:

[source,js]
----
"strict_date_optional_time||epoch_millis"
----
// NOTCONSOLE

This means that it will accept dates with optional timestamps, which conform
to the formats supported by <<strict-date-time,`strict_date_optional_time`>>
or milliseconds-since-the-epoch.

For instance:

[source,console]
[source,console,id=date-example]
--------------------------------------------------
PUT my-index-000001
{
Expand Down Expand Up @@ -68,6 +72,16 @@ GET my-index-000001/_search
<4> This document uses milliseconds-since-the-epoch.
<5> Note that the `sort` values that are returned are all in milliseconds-since-the-epoch.

[WARNING]
====
Dates
// tag::decimal-warning[]
will accept numbers with a decimal point like `{"date": 1618249875.123456}`
but there are some cases ({es-issue}70085[#70085]) where we'll lose precision
on those dates so should avoid them.
// end::decimal-warning[]
====

[[multiple-date-formats]]
==== Multiple date formats

Expand All @@ -76,7 +90,7 @@ Each format will be tried in turn until a matching format is found. The first
format will be used to convert the _milliseconds-since-the-epoch_ value back
into a string.

[source,console]
[source,console,id=date-format-example]
--------------------------------------------------
PUT my-index-000001
{
Expand Down Expand Up @@ -144,3 +158,56 @@ The following parameters are accepted by `date` fields:
<<mapping-field-meta,`meta`>>::

Metadata about the field.


[[date-epoch-seconds]]
==== Epoch seconds

If you need to send dates as _seconds-since-the-epoch_ then make sure the
`format` lists `epoch_second`:

[source,console,id=date-epoch-seconds-example]
----
PUT my-index-000001
{
"mappings": {
"properties": {
"date": {
"type": "date",
"format": "strict_date_optional_time||epoch_second"
}
}
}
}

PUT my-index-000001/_doc/example?refresh
{ "date": 1618321898 }

POST my-index-000001/_search
{
"fields": [ {"field": "date"}],
"_source": false
}
----
// TEST[s/_search/_search?filter_path=hits.hits/]

Which will reply with a date like:

[source,console-result]
----
{
"hits": {
"hits": [
{
"_id": "example",
"_index": "my-index-000001",
"_type": "_doc",
"_score": 1.0,
"fields": {
"date": ["2021-04-13T13:51:38.000Z"]
}
}
]
}
}
----
10 changes: 10 additions & 0 deletions docs/reference/mapping/types/date_nanos.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,11 @@ back to a string depending on the date format that is associated with the field.
Date formats can be customised, but if no `format` is specified then it uses
the default:

[source,js]
----
"strict_date_optional_time||epoch_millis"
----
// NOTCONSOLE

This means that it will accept dates with optional timestamps, which conform
to the formats supported by
Expand Down Expand Up @@ -135,6 +139,12 @@ Use <<strict-date-time-nanos,`strict_date_optional_time_nanos`>> or you'll get a
You can also specify multiple date formats separated by `||`. The
same mapping parameters than with the `date` field can be used.

[WARNING]
====
Date nanoseconds
include::date.asciidoc[tag=decimal-warning]
====

[[date-nanos-limitations]]
==== Limitations

Expand Down