Skip to content

Commit

Permalink
type conversion functions
Browse files Browse the repository at this point in the history
  • Loading branch information
fang-xing-esql committed Apr 18, 2024
1 parent 7cd40e5 commit b34f3a1
Show file tree
Hide file tree
Showing 17 changed files with 111 additions and 215 deletions.
6 changes: 6 additions & 0 deletions docs/reference/esql/functions/examples/to_datetime.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,12 @@ include::{esql-specs}/date.csv-spec[tag=to_datetime-str]
|===
include::{esql-specs}/date.csv-spec[tag=to_datetime-str-result]
|===

NOTE: Note that in this example, the last value in the source multi-valued field has not been converted. The reason being that if the date format is not respected, the conversion will result in a *null* value. When this happens a _Warning_ header is added to the response. The header will provide information on the source of the failure:
`"Line 1:112: evaluation of [TO_DATETIME(string)] failed, treating result as null. Only first 20 failures recorded."`
A following header will contain the failure reason and the offending value:
`"java.lang.IllegalArgumentException: failed to parse date field [1964-06-02 00:00:00] with format [yyyy-MM-dd'T'HH:mm:ss.SSS'Z']"`

If the input parameter is of a numeric type, its value will be interpreted as milliseconds since the {wikipedia}/Unix_time[Unix epoch]. For example:
[source.merge.styled,esql]
----
Expand Down
6 changes: 6 additions & 0 deletions docs/reference/esql/functions/examples/to_double.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -11,3 +11,9 @@ include::{esql-specs}/floats.csv-spec[tag=to_double-str]
include::{esql-specs}/floats.csv-spec[tag=to_double-str-result]
|===

NOTE: Note that in this example, the last conversion of the string isn't possible. When this happens, the result is a *null* value. In this case a _Warning_ header is added to the response. The header will provide information on the source of the failure:
`"Line 1:115: evaluation of [TO_DOUBLE(str2)] failed, treating result as null. Only first 20 failures recorded."`
A following header will contain the failure reason and the offending value:
`"java.lang.NumberFormatException: For input string: \"foo\""`


6 changes: 6 additions & 0 deletions docs/reference/esql/functions/examples/to_integer.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -11,3 +11,9 @@ include::{esql-specs}/ints.csv-spec[tag=to_int-long]
include::{esql-specs}/ints.csv-spec[tag=to_int-long-result]
|===

NOTE: Note that in this example, the last value of the multi-valued field cannot be converted as an integer. When this happens, the result is a *null* value. In this case a _Warning_ header is added to the response. The header will provide information on the source of the failure:
`"Line 1:61: evaluation of [TO_INTEGER(long)] failed, treating result as null. Only first 20 failures recorded."`
A following header will contain the failure reason and the offending value:
`"org.elasticsearch.xpack.ql.InvalidArgumentException: [501379200000] out of [integer] range"`


6 changes: 6 additions & 0 deletions docs/reference/esql/functions/examples/to_ip.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -11,3 +11,9 @@ include::{esql-specs}/ip.csv-spec[tag=to_ip]
include::{esql-specs}/ip.csv-spec[tag=to_ip-result]
|===

NOTE: Note that in this example, the last conversion of the string isn't possible. When this happens, the result is a *null* value. In this case a _Warning_ header is added to the response. The header will provide information on the source of the failure:
`"Line 1:68: evaluation of [TO_IP(str2)] failed, treating result as null. Only first 20 failures recorded."`
A following header will contain the failure reason and the offending value:
`"java.lang.IllegalArgumentException: 'foo' is not an IP string literal."`


6 changes: 6 additions & 0 deletions docs/reference/esql/functions/examples/to_long.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -11,3 +11,9 @@ include::{esql-specs}/ints.csv-spec[tag=to_long-str]
include::{esql-specs}/ints.csv-spec[tag=to_long-str-result]
|===

NOTE: Note that in this example, the last conversion of the string isn't possible. When this happens, the result is a *null* value. In this case a _Warning_ header is added to the response. The header will provide information on the source of the failure:
`"Line 1:113: evaluation of [TO_LONG(str3)] failed, treating result as null. Only first 20 failures recorded."`
A following header will contain the failure reason and the offending value:
`"java.lang.NumberFormatException: For input string: \"foo\""`


Original file line number Diff line number Diff line change
Expand Up @@ -11,3 +11,9 @@ include::{esql-specs}/ints.csv-spec[tag=to_unsigned_long-str]
include::{esql-specs}/ints.csv-spec[tag=to_unsigned_long-str-result]
|===

NOTE: Note that in this example, the last conversion of the string isn't possible. When this happens, the result is a *null* value. In this case a _Warning_ header is added to the response. The header will provide information on the source of the failure:
`"Line 1:133: evaluation of [TO_UL(str3)] failed, treating result as null. Only first 20 failures recorded."`
A following header will contain the failure reason and the offending value:
`"java.lang.NumberFormatException: Character f is neither a decimal digit number, decimal point, nor \"e\" notation exponential mark."`


62 changes: 0 additions & 62 deletions docs/reference/esql/functions/to_datetime.asciidoc

This file was deleted.

50 changes: 0 additions & 50 deletions docs/reference/esql/functions/to_double.asciidoc

This file was deleted.

50 changes: 0 additions & 50 deletions docs/reference/esql/functions/to_integer.asciidoc

This file was deleted.

41 changes: 0 additions & 41 deletions docs/reference/esql/functions/to_ip.asciidoc

This file was deleted.

12 changes: 6 additions & 6 deletions docs/reference/esql/functions/type-conversion-functions.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -32,15 +32,15 @@ include::layout/to_base64.asciidoc[]
include::layout/to_boolean.asciidoc[]
include::layout/to_cartesianpoint.asciidoc[]
include::layout/to_cartesianshape.asciidoc[]
include::to_datetime.asciidoc[]
include::layout/to_datetime.asciidoc[]
include::layout/to_degrees.asciidoc[]
include::to_double.asciidoc[]
include::layout/to_double.asciidoc[]
include::layout/to_geopoint.asciidoc[]
include::layout/to_geoshape.asciidoc[]
include::to_integer.asciidoc[]
include::to_ip.asciidoc[]
include::to_long.asciidoc[]
include::layout/to_integer.asciidoc[]
include::layout/to_ip.asciidoc[]
include::layout/to_long.asciidoc[]
include::layout/to_radians.asciidoc[]
include::layout/to_string.asciidoc[]
include::to_unsigned_long.asciidoc[]
include::layout/to_unsigned_long.asciidoc[]
include::layout/to_version.asciidoc[]
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,19 @@ public class ToDatetime extends AbstractConvertFunction {
+ "A string will only be successfully converted if it's respecting the format `yyyy-MM-dd'T'HH:mm:ss.SSS'Z'`. "
+ "To convert dates in other formats, use <<esql-date_parse>>.",
examples = {
@Example(file = "date", tag = "to_datetime-str"),
@Example(
file = "date",
tag = "to_datetime-str",
note = "Note that in this example, the last value in the source multi-valued field has not been converted. "
+ "The reason being that if the date format is not respected, the conversion will result in a *null* value. "
+ "When this happens a _Warning_ header is added to the response. "
+ "The header will provide information on the source of the failure:\n"
+ "`\"Line 1:112: evaluation of [TO_DATETIME(string)] failed, treating result as null. "
+ "Only first 20 failures recorded.\"`\n"
+ "A following header will contain the failure reason and the offending value:\n"
+ "`\"java.lang.IllegalArgumentException: failed to parse date field [1964-06-02 00:00:00] "
+ "with format [yyyy-MM-dd'T'HH:mm:ss.SSS'Z']\"`"
),
@Example(
description = "If the input parameter is of a numeric type, "
+ "its value will be interpreted as milliseconds since the {wikipedia}/Unix_time[Unix epoch]. For example:",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,17 @@ public class ToDouble extends AbstractConvertFunction {
description = "Converts an input value to a double value.\nIf the input parameter is of a date type, "
+ "its value will be interpreted as milliseconds since the {wikipedia}/Unix_time[Unix epoch], "
+ "converted to double. Boolean *true* will be converted to double *1.0*, *false* to *0.0*.",
examples = @Example(file = "floats", tag = "to_double-str")
examples = @Example(
file = "floats",
tag = "to_double-str",
note = "Note that in this example, the last conversion of the string isn't possible. "
+ "When this happens, the result is a *null* value. In this case a _Warning_ header is added to the response. "
+ "The header will provide information on the source of the failure:\n"
+ "`\"Line 1:115: evaluation of [TO_DOUBLE(str2)] failed, treating result as null. "
+ "Only first 20 failures recorded.\"`\n"
+ "A following header will contain the failure reason and the offending value:\n"
+ "`\"java.lang.NumberFormatException: For input string: \\\"foo\\\"\"`"
)
)
public ToDouble(
Source source,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,17 @@ public class ToIP extends AbstractConvertFunction {
@FunctionInfo(
returnType = "ip",
description = "Converts an input string to an IP value.",
examples = @Example(file = "ip", tag = "to_ip")
examples = @Example(
file = "ip",
tag = "to_ip",
note = "Note that in this example, the last conversion of the string isn't possible. "
+ "When this happens, the result is a *null* value. In this case a _Warning_ header is added to the response. "
+ "The header will provide information on the source of the failure:\n"
+ "`\"Line 1:68: evaluation of [TO_IP(str2)] failed, treating result as null. "
+ "Only first 20 failures recorded.\"`\n"
+ "A following header will contain the failure reason and the offending value:\n"
+ "`\"java.lang.IllegalArgumentException: 'foo' is not an IP string literal.\"`"
)
)
public ToIP(
Source source,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,17 @@ public class ToInteger extends AbstractConvertFunction {
+ "If the input parameter is of a date type, its value will be interpreted as milliseconds "
+ "since the {wikipedia}/Unix_time[Unix epoch], converted to integer.\n"
+ "Boolean *true* will be converted to integer *1*, *false* to *0*.",
examples = @Example(file = "ints", tag = "to_int-long")
examples = @Example(
file = "ints",
tag = "to_int-long",
note = "Note that in this example, the last value of the multi-valued field cannot be converted as an integer. "
+ "When this happens, the result is a *null* value. In this case a _Warning_ header is added to the response. "
+ "The header will provide information on the source of the failure:\n"
+ "`\"Line 1:61: evaluation of [TO_INTEGER(long)] failed, treating result as null. "
+ "Only first 20 failures recorded.\"`\n"
+ "A following header will contain the failure reason and the offending value:\n"
+ "`\"org.elasticsearch.xpack.ql.InvalidArgumentException: [501379200000] out of [integer] range\"`"
)
)
public ToInteger(
Source source,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,17 @@ public class ToLong extends AbstractConvertFunction {
+ "its value will be interpreted as milliseconds since the {wikipedia}/Unix_time[Unix epoch], "
+ "converted to long.\n"
+ "Boolean *true* will be converted to long *1*, *false* to *0*.",
examples = @Example(file = "ints", tag = "to_long-str")
examples = @Example(
file = "ints",
tag = "to_long-str",
note = "Note that in this example, the last conversion of the string isn't possible. "
+ "When this happens, the result is a *null* value. In this case a _Warning_ header is added to the response. "
+ "The header will provide information on the source of the failure:\n"
+ "`\"Line 1:113: evaluation of [TO_LONG(str3)] failed, treating result as null. "
+ "Only first 20 failures recorded.\"`\n"
+ "A following header will contain the failure reason and the offending value:\n"
+ "`\"java.lang.NumberFormatException: For input string: \\\"foo\\\"\"`"
)
)
public ToLong(
Source source,
Expand Down
Loading

0 comments on commit b34f3a1

Please sign in to comment.