Advise against dates with decimal points (backport of elastic#71578)

We accept dates with a decimal point like `2113413.13241324` and parse them *somehow*. But there are cases where we'll lose precision on those dates, see elastic#70085. This advises folks not to use that format. We'll continue to accept those dates for backwards compatibility but you should avoid using them. Co-authored-by: Adrien Grand <[email protected]>
nik9000 · Apr 13, 2021 · e8d2c73 · e8d2c73
1 parent 29a1f7c
commit e8d2c73
Show file tree

Hide file tree

Showing 2 changed files with 83 additions and 6 deletions.
diff --git a/docs/reference/mapping/types/date.asciidoc b/docs/reference/mapping/types/date.asciidoc
@@ -7,11 +7,11 @@
 JSON doesn't have a date data type, so dates in Elasticsearch can either be:
 
 * strings containing formatted dates, e.g. `"2015-01-01"` or `"2015/01/01 12:10:30"`.
-* a long number representing _milliseconds-since-the-epoch_.
-* an integer representing _seconds-since-the-epoch_.
+* a number representing _milliseconds-since-the-epoch_.
+* a number representing _seconds-since-the-epoch_ (<<date-epoch-seconds, configuration>>).
 
-NOTE: Values for _milliseconds-since-the-epoch_ and _seconds-since-the-epoch_
-must be non-negative. Use a formatted date to represent dates before 1970.
+NOTE: Values for _milliseconds-since-the-epoch_ must be non-negative. Use a
+formatted date to represent dates before 1970.
 
 Internally, dates are converted to UTC (if the time-zone is specified) and
 stored as a long number representing milliseconds-since-the-epoch.
@@ -26,15 +26,19 @@ supplied as a long in the JSON document.
 Date formats can be customised, but if no `format` is specified then it uses
 the default:
 
+[source,js]
+----
     "strict_date_optional_time||epoch_millis"
+----
+// NOTCONSOLE
 
 This means that it will accept dates with optional timestamps, which conform
 to the formats supported by <<strict-date-time,`strict_date_optional_time`>>
 or milliseconds-since-the-epoch.
 
 For instance:
 
-[source,console]
+[source,console,id=date-example]
 --------------------------------------------------
 PUT my-index-000001
 {
@@ -68,6 +72,16 @@ GET my-index-000001/_search
 <4> This document uses milliseconds-since-the-epoch.
 <5> Note that the `sort` values that are returned are all in milliseconds-since-the-epoch.
 
+[WARNING]
+====
+Dates
+// tag::decimal-warning[]
+will accept numbers with a decimal point like `{"date": 1618249875.123456}`
+but there are some cases ({es-issue}70085[#70085]) where we'll lose precision
+on those dates so should avoid them.
+// end::decimal-warning[]
+====
+
 [[multiple-date-formats]]
 ==== Multiple date formats
 
@@ -76,7 +90,7 @@ Each format will be tried in turn until a matching format is found. The first
 format will be used to convert the _milliseconds-since-the-epoch_ value back
 into a string.
 
-[source,console]
+[source,console,id=date-format-example]
 --------------------------------------------------
 PUT my-index-000001
 {
@@ -144,3 +158,56 @@ The following parameters are accepted by `date` fields:
 <<mapping-field-meta,`meta`>>::
 
     Metadata about the field.
+
+
+[[date-epoch-seconds]]
+==== Epoch seconds
+
+If you need to send dates as _seconds-since-the-epoch_ then make sure the
+`format` lists `epoch_second`:
+
+[source,console,id=date-epoch-seconds-example]
+----
+PUT my-index-000001
+{
+  "mappings": {
+    "properties": {
+      "date": {
+        "type":   "date",
+        "format": "strict_date_optional_time||epoch_second"
+      }
+    }
+  }
+}
+
+PUT my-index-000001/_doc/example?refresh
+{ "date": 1618321898 }
+
+POST my-index-000001/_search
+{
+  "fields": [ {"field": "date"}],
+  "_source": false
+}
+----
+// TEST[s/_search/_search?filter_path=hits.hits/]
+
+Which will reply with a date like:
+
+[source,console-result]
+----
+{
+  "hits": {
+    "hits": [
+      {
+        "_id": "example",
+        "_index": "my-index-000001",
+        "_type": "_doc",
+        "_score": 1.0,
+        "fields": {
+          "date": ["2021-04-13T13:51:38.000Z"]
+        }
+      }
+    ]
+  }
+}
+----
diff --git a/docs/reference/mapping/types/date_nanos.asciidoc b/docs/reference/mapping/types/date_nanos.asciidoc
@@ -18,7 +18,11 @@ back to a string depending on the date format that is associated with the field.
 Date formats can be customised, but if no `format` is specified then it uses
 the default:
 
+[source,js]
+----
     "strict_date_optional_time||epoch_millis"
+----
+// NOTCONSOLE
 
 This means that it will accept dates with optional timestamps, which conform
 to the formats supported by
@@ -135,6 +139,12 @@ Use <<strict-date-time-nanos,`strict_date_optional_time_nanos`>> or you'll get a
 You can also specify multiple date formats separated by `||`. The
 same mapping parameters than with the `date` field can be used.
 
+[WARNING]
+====
+Date nanoseconds
+include::date.asciidoc[tag=decimal-warning]
+====
+
 [[date-nanos-limitations]]
 ==== Limitations