Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Transform] percentiles and sparse data can fail transform #54201

Closed
hendrikmuhs opened this issue Mar 25, 2020 · 1 comment · Fixed by #54202
Closed

[Transform] percentiles and sparse data can fail transform #54201

hendrikmuhs opened this issue Mar 25, 2020 · 1 comment · Fixed by #54202
Labels
:ml/Transform Transform >non-issue

Comments

@hendrikmuhs
Copy link

originally reported by @peteharverson:

If percentiles is used but data is sparse - meaning that no percentiles could be calculated - transform can fail with a BulkIndexingException.

If percentiles can not be calculated it returns NaN, this causes an indexing error as NaN can not be indexed.

When using percentiles in search, it returns null. Transform should do the same, this is also consistent with other aggregations, e.g. single value aggregations.

Because percentiles support for transforms has not been released, I label this as non-issue, technically it is a bug.
@elasticmachine
Copy link
Collaborator

Pinging @elastic/ml-core (:ml/Transform)

@hendrikmuhs hendrikmuhs mentioned this issue Mar 25, 2020
Merged
hendrikmuhs pushed a commit that referenced this issue Mar 25, 2020
[Transform] fix transform failure case for percentiles and spa… (#54202)
995bed2 
index null if percentiles could not be calculated due to sparse data

fixes #54201
hendrikmuhs pushed a commit that referenced this issue Mar 25, 2020
[Transform] fix transform failure case for percentiles and spa… (#54202)
cb0ecaf 
index null if percentiles could not be calculated due to sparse data

fixes #54201
hendrikmuhs pushed a commit that referenced this issue Mar 25, 2020
[Transform] fix transform failure case for percentiles and spa… (#54202)
e724b86 
index null if percentiles could not be calculated due to sparse data

fixes #54201
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
:ml/Transform Transform >non-issue
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[Transform] fix transform failure case for percentiles and sparse data hendrikmuhs/elasticsearch
2 participants
@hendrikmuhs @elasticmachine