Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

big decimal value losed precision #64094

Closed
vpzlin opened this issue Oct 23, 2020 · 2 comments
Closed

big decimal value losed precision #64094

vpzlin opened this issue Oct 23, 2020 · 2 comments
Labels
>bug needs:triage Requires assignment of a team area label

Comments

@vpzlin
Copy link

vpzlin commented Oct 23, 2020

Elasticsearch version (bin/elasticsearch --version): 6.4.2 / 6.8.9 / 7.9.0

Plugins installed: [none]

JVM version (java -version): 1.8

OS version (uname -a if on a Unix-like system): CentOS 7.7

Description of the problem including expected versus actual behavior:

Hello, ES team!
There is a precision losing problem when I'm using java client of ElasticSearch.
Will you fix this bug recently? We do need a big decimal in developing financial system.
Thanks very much!

[Development Environment]
JDK: 1.8
ElasticSearch Server Version: 6.4.2 / 6.8.9 / 7.9.0
ElasticSearch Client Version: 6.4.2 / 6.8.9 / 7.9.0

[Mapping]
PUT test_idx/_doc/_mapping
{
"properties": {
"balance":{"type": "double"}
}
}

[Main Codes & Results -- version of java client and ES server: 6.4.2 / 6.8.9]
BigDecimal val = new BigDecimal("12345678901234567890.12345");
JSONObject jsonObject = new JSONObject();
jsonObject.put("balance", val);

/* Single Put */
// The field 'balance' in curl result by shell showed the correct value '12345678901234567890.12345'
client.index(new IndexRequest("test_idx", "_doc", "1").source(jsonObject.toJSONString(), XContentType.JSON)
        ,RequestOptions.DEFAULT);

/* Bulk Load */
// The field 'balance' in curl result by shell showed the precision losing value '1.2345678901234567E19'
BulkRequest bulkRequest = new BulkRequest();
bulkRequest.add(new IndexRequest("test_idx", "_doc", "2").source(jsonObject.toJSONString(), XContentType.JSON));
client.bulk(bulkRequest, RequestOptions.DEFAULT);

[Main Codes & Results -- version of java client and ES server: 7.9.0]
BigDecimal val = new BigDecimal("12345678901234567890.12345");
JSONObject jsonObject = new JSONObject();
jsonObject.put("balance", val);

/* Single Put */
// The field 'balance' in curl result by shell showed the correct value '12345678901234567890.12345'
client.index(new IndexRequest("test_idx").source(jsonObject.toJSONString(), XContentType.JSON)
        ,RequestOptions.DEFAULT);

/* Bulk Load */
// The field 'balance' in curl result by shell showed the precision losing value '1.2345678901234567E19'
BulkRequest bulkRequest = new BulkRequest();
bulkRequest.add(new IndexRequest("test_idx").source(jsonObject.toJSONString(), XContentType.JSON));
client.bulk(bulkRequest, RequestOptions.DEFAULT);

[Debug and Modify code in java client of ES 7.9.0]
From the results above in different version we can see that it'll losing precision when bulk loading data into ES.
So I debugged code, and I found this bug occured in function '_parseSlowFloat(int expType)' of class 'jackon-core-2.10.4.jar/com.fasterxml.jackson.core.base.ParserBase', the value of variable 'expType' is always 0(int value) from begging, it havn't changged, so the non-int numeric value will be regarded as type 'double' always.
And then, I done these:
[1] downloaded the source code 'jackson-core-2.10.4-sources.jar' and decompressed it into my project.
[2] altered code
/********* before /
if (expType == NR_BIGDECIMAL) {
_numberBigDecimal = _textBuffer.contentsAsDecimal();
_numTypesValid = NR_BIGDECIMAL;
} else {
// Otherwise double has to do
_numberDouble = _textBuffer.contentsAsDouble();
_numTypesValid = NR_DOUBLE;
}
/
after *********/
_numberBigDecimal = _textBuffer.contentsAsDecimal();
_numTypesValid = NR_BIGDECIMAL;
[3] rerun bulk load code to ES, field 'balance' in curl result by shell showed the correct value '12345678901234567890.12345'

@vpzlin vpzlin added >bug needs:triage Requires assignment of a team area label labels Oct 23, 2020
@jimczi
Copy link
Contributor

jimczi commented Oct 26, 2020

Closing in favor of #46934. We don't provide big decimals data type but the feature is discussed in the linked issue.

@jimczi jimczi closed this as completed Oct 26, 2020
@vpzlin
Copy link
Author

vpzlin commented Oct 27, 2020

Closing in favor of #46934. We don't provide big decimals data type but the feature is discussed in the linked issue.

Hi, jimczi.
I'm still considering it as a bug. Did you see my description above?
When I downloaded the source code 'jackson-core-2.10.4-sources.jar' and decompressed it into my project, and then altered codes in java class 'ParserBase.java' as below:

/********* codes before *********/
if (expType == NR_BIGDECIMAL) {
_numberBigDecimal = _textBuffer.contentsAsDecimal();
_numTypesValid = NR_BIGDECIMAL;
} else {
// Otherwise double has to do
_numberDouble = _textBuffer.contentsAsDouble();
_numTypesValid = NR_DOUBLE;
}

/********* codes after *********/
_numberBigDecimal = _textBuffer.contentsAsDecimal();
_numTypesValid = NR_BIGDECIMAL;

After finished altering codes, I reran codes of putting single big-decimal record and bulk loaded multi big-decimal records, both of them worked fine, big-decimal value stored correctly into ES(should be queryed by curl shell, not in kibana or web browser).

So, don't you think this is a bug?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
>bug needs:triage Requires assignment of a team area label
Projects
None yet
Development

No branches or pull requests

2 participants