You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Elasticsearch version (bin/elasticsearch --version): 6.4.2 / 6.8.9 / 7.9.0
Plugins installed: [none]
JVM version (java -version): 1.8
OS version (uname -a if on a Unix-like system): CentOS 7.7
Description of the problem including expected versus actual behavior:
Hello, ES team!
There is a precision losing problem when I'm using java client of ElasticSearch.
Will you fix this bug recently? We do need a big decimal in developing financial system.
Thanks very much!
[Mapping]
PUT test_idx/_doc/_mapping
{
"properties": {
"balance":{"type": "double"}
}
}
[Main Codes & Results -- version of java client and ES server: 6.4.2 / 6.8.9]
BigDecimal val = new BigDecimal("12345678901234567890.12345");
JSONObject jsonObject = new JSONObject();
jsonObject.put("balance", val);
/* Single Put */
// The field 'balance' in curl result by shell showed the correct value '12345678901234567890.12345'
client.index(new IndexRequest("test_idx", "_doc", "1").source(jsonObject.toJSONString(), XContentType.JSON)
,RequestOptions.DEFAULT);
/* Bulk Load */
// The field 'balance' in curl result by shell showed the precision losing value '1.2345678901234567E19'
BulkRequest bulkRequest = new BulkRequest();
bulkRequest.add(new IndexRequest("test_idx", "_doc", "2").source(jsonObject.toJSONString(), XContentType.JSON));
client.bulk(bulkRequest, RequestOptions.DEFAULT);
[Main Codes & Results -- version of java client and ES server: 7.9.0]
BigDecimal val = new BigDecimal("12345678901234567890.12345");
JSONObject jsonObject = new JSONObject();
jsonObject.put("balance", val);
/* Single Put */
// The field 'balance' in curl result by shell showed the correct value '12345678901234567890.12345'
client.index(new IndexRequest("test_idx").source(jsonObject.toJSONString(), XContentType.JSON)
,RequestOptions.DEFAULT);
/* Bulk Load */
// The field 'balance' in curl result by shell showed the precision losing value '1.2345678901234567E19'
BulkRequest bulkRequest = new BulkRequest();
bulkRequest.add(new IndexRequest("test_idx").source(jsonObject.toJSONString(), XContentType.JSON));
client.bulk(bulkRequest, RequestOptions.DEFAULT);
[Debug and Modify code in java client of ES 7.9.0]
From the results above in different version we can see that it'll losing precision when bulk loading data into ES.
So I debugged code, and I found this bug occured in function '_parseSlowFloat(int expType)' of class 'jackon-core-2.10.4.jar/com.fasterxml.jackson.core.base.ParserBase', the value of variable 'expType' is always 0(int value) from begging, it havn't changged, so the non-int numeric value will be regarded as type 'double' always.
And then, I done these:
[1] downloaded the source code 'jackson-core-2.10.4-sources.jar' and decompressed it into my project.
[2] altered code
/********* before /
if (expType == NR_BIGDECIMAL) {
_numberBigDecimal = _textBuffer.contentsAsDecimal();
_numTypesValid = NR_BIGDECIMAL;
} else {
// Otherwise double has to do
_numberDouble = _textBuffer.contentsAsDouble();
_numTypesValid = NR_DOUBLE;
}
/ after *********/
_numberBigDecimal = _textBuffer.contentsAsDecimal();
_numTypesValid = NR_BIGDECIMAL;
[3] rerun bulk load code to ES, field 'balance' in curl result by shell showed the correct value '12345678901234567890.12345'
The text was updated successfully, but these errors were encountered:
Closing in favor of #46934. We don't provide big decimals data type but the feature is discussed in the linked issue.
Hi, jimczi.
I'm still considering it as a bug. Did you see my description above?
When I downloaded the source code 'jackson-core-2.10.4-sources.jar' and decompressed it into my project, and then altered codes in java class 'ParserBase.java' as below:
/********* codes before *********/
if (expType == NR_BIGDECIMAL) {
_numberBigDecimal = _textBuffer.contentsAsDecimal();
_numTypesValid = NR_BIGDECIMAL;
} else {
// Otherwise double has to do
_numberDouble = _textBuffer.contentsAsDouble();
_numTypesValid = NR_DOUBLE;
}
/********* codes after *********/
_numberBigDecimal = _textBuffer.contentsAsDecimal();
_numTypesValid = NR_BIGDECIMAL;
After finished altering codes, I reran codes of putting single big-decimal record and bulk loaded multi big-decimal records, both of them worked fine, big-decimal value stored correctly into ES(should be queryed by curl shell, not in kibana or web browser).
Elasticsearch version (
bin/elasticsearch --version
): 6.4.2 / 6.8.9 / 7.9.0Plugins installed: [none]
JVM version (
java -version
): 1.8OS version (
uname -a
if on a Unix-like system): CentOS 7.7Description of the problem including expected versus actual behavior:
Hello, ES team!
There is a precision losing problem when I'm using java client of ElasticSearch.
Will you fix this bug recently? We do need a big decimal in developing financial system.
Thanks very much!
[Development Environment]
JDK: 1.8
ElasticSearch Server Version: 6.4.2 / 6.8.9 / 7.9.0
ElasticSearch Client Version: 6.4.2 / 6.8.9 / 7.9.0
[Mapping]
PUT test_idx/_doc/_mapping
{
"properties": {
"balance":{"type": "double"}
}
}
[Main Codes & Results -- version of java client and ES server: 6.4.2 / 6.8.9]
BigDecimal val = new BigDecimal("12345678901234567890.12345");
JSONObject jsonObject = new JSONObject();
jsonObject.put("balance", val);
[Main Codes & Results -- version of java client and ES server: 7.9.0]
BigDecimal val = new BigDecimal("12345678901234567890.12345");
JSONObject jsonObject = new JSONObject();
jsonObject.put("balance", val);
[Debug and Modify code in java client of ES 7.9.0]
From the results above in different version we can see that it'll losing precision when bulk loading data into ES.
So I debugged code, and I found this bug occured in function '_parseSlowFloat(int expType)' of class 'jackon-core-2.10.4.jar/com.fasterxml.jackson.core.base.ParserBase', the value of variable 'expType' is always 0(int value) from begging, it havn't changged, so the non-int numeric value will be regarded as type 'double' always.
And then, I done these:
[1] downloaded the source code 'jackson-core-2.10.4-sources.jar' and decompressed it into my project.
[2] altered code
/********* before /
if (expType == NR_BIGDECIMAL) {
_numberBigDecimal = _textBuffer.contentsAsDecimal();
_numTypesValid = NR_BIGDECIMAL;
} else {
// Otherwise double has to do
_numberDouble = _textBuffer.contentsAsDouble();
_numTypesValid = NR_DOUBLE;
}
/ after *********/
_numberBigDecimal = _textBuffer.contentsAsDecimal();
_numTypesValid = NR_BIGDECIMAL;
[3] rerun bulk load code to ES, field 'balance' in curl result by shell showed the correct value '12345678901234567890.12345'
The text was updated successfully, but these errors were encountered: