Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Term vectors API crashes for documents that have no term vectors in nested fields #32652

Closed
rafaelglater opened this issue Aug 6, 2018 · 4 comments
Assignees
Labels
>bug :Search/Search Search-related issues that do not fall into other categories

Comments

@rafaelglater
Copy link

Elasticsearch version: 6.3.2

JVM version: 1.8.0_181

OS version: KDE Neon 5.13.4

Description of the problem including expected versus actual behavior:
When requesting the termvectors of a document with empty terms in a nested field, we get a null pointer exception.

Steps to reproduce:
1.
PUT /my-index { "settings": { "number_of_shards": 1, "similarity": { "default": { "type": "BM25" } }, "analysis": { "filter": { "bigram_filter": { "type": "shingle", "min_shingle_size": 2, "max_shingle_size": 2, "output_unigrams": false }, "stopwords": { "type": "stop", "stopwords": "_english_" }, "kstemmer": { "type": "stemmer", "language": "kstem" }, "possessive": { "type": "stemmer", "language": "possessive_english" } }, "analyzer": { "bigram_analyzer": { "type": "custom", "tokenizer": "standard", "filter": [ "lowercase", "stopwords", "possessive", "kstemmer", "bigram_filter" ] }, "standard": { "tokenizer": "standard", "filter": [ "lowercase", "stopwords", "possessive", "kstemmer" ] } } } } }

PUT /my-index/_mapping/my_doc { "my_doc": { "properties": { "text": {"type":"text", "analyzer":"standard", "term_vector": "yes", "fields": { "bigrams": { "type": "text", "term_vector": "yes", "analyzer": "bigram_analyzer" } } }, "links": {"type":"nested", "properties":{ "page_id": {"type":"text", "analyzer":"standard"}, "page_name": {"type":"text", "analyzer":"standard", "fields": { "bigrams": { "type": "text", "analyzer": "bigram_analyzer" } } } } } } } }

Insert documents with different terms configurations:

PUT /my-index/my_doc/111 { "text": "a bigram text", "links": [{"page_id": "1", "page_name":"two terms"}, {"page_id": "2", "page_name":"unique"} ] }

PUT /my-index/my_doc/222 { "text": "unigram", "links": [{"page_id": "3", "page_name":"one"}, {"page_id": "2", "page_name":"unique"} ] }

PUT /my-index/my_doc/333 { "text": "", "links": [{"page_id": "4", "page_name":""}] }

PUT /my-index/my_doc/444 { "text": "", "links": [{"page_id": "4", "page_name":""}, {"page_id": "2", "page_name":"unique"} ] }

If I request the termvectors of documents 111 and 222 it's ok. But when requesting termvectors of document 333 and 444, I get null pointer exception because I have an empty nested field "page_name":

GET /my-index/my_doc/444/_termvectors { "fields" : ["text", "text.bigrams", "links.page_name", "links.page_name.bigrams"], "offsets" : false, "payloads" : false, "positions" : false, "term_statistics" : true, "field_statistics" : true }

Provide logs (if relevant):
[2018-08-06T16:04:28,002][DEBUG][o.e.a.t.TransportTermVectorsAction] [T3sBXHE] null: failed to execute [org.elasticsearch.action.termvectors.TermVectorsRequest@307d2b7b]
org.elasticsearch.transport.RemoteTransportException: [T3sBXHE][127.0.0.1:9300][indices:data/read/tv[s]]
Caused by: org.elasticsearch.ElasticsearchException: failed to execute term vector request
at org.elasticsearch.index.termvectors.TermVectorsService.getTermVectors(TermVectorsService.java:150) ~[elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.index.termvectors.TermVectorsService.getTermVectors(TermVectorsService.java:77) ~[elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.action.termvectors.TransportTermVectorsAction.shardOperation(TransportTermVectorsAction.java:89) ~[elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.action.termvectors.TransportTermVectorsAction.shardOperation(TransportTermVectorsAction.java:43) ~[elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.action.support.single.shard.TransportSingleShardAction$ShardTransportHandler.messageReceived(TransportSingleShardAction.java:287) ~[elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.action.support.single.shard.TransportSingleShardAction$ShardTransportHandler.messageReceived(TransportSingleShardAction.java:280) ~[elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.transport.TransportRequestHandler.messageReceived(TransportRequestHandler.java:30) ~[elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.xpack.security.transport.SecurityServerTransportInterceptor$ProfileSecuredRequestHandler$1.doRun(SecurityServerTransportInterceptor.java:259) [x-pack-security-6.3.2.jar:6.3.2]
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.xpack.security.transport.SecurityServerTransportInterceptor$ProfileSecuredRequestHandler.messageReceived(SecurityServerTransportInterceptor.java:317) [x-pack-security-6.3.2.jar:6.3.2]
at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:66) [elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.transport.TransportService$7.doRun(TransportService.java:664) [elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:725) [elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-6.3.2.jar:6.3.2]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_181]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_181]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_181]
Caused by: java.lang.NullPointerException
at org.elasticsearch.action.termvectors.TermVectorsWriter.setFields(TermVectorsWriter.java:82) ~[elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.action.termvectors.TermVectorsResponse.setFields(TermVectorsResponse.java:361) ~[elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.index.termvectors.TermVectorsService.getTermVectors(TermVectorsService.java:146) ~[elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.index.termvectors.TermVectorsService.getTermVectors(TermVectorsService.java:77) ~[elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.action.termvectors.TransportTermVectorsAction.shardOperation(TransportTermVectorsAction.java:89) ~[elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.action.termvectors.TransportTermVectorsAction.shardOperation(TransportTermVectorsAction.java:43) ~[elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.action.support.single.shard.TransportSingleShardAction$ShardTransportHandler.messageReceived(TransportSingleShardAction.java:287) ~[elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.action.support.single.shard.TransportSingleShardAction$ShardTransportHandler.messageReceived(TransportSingleShardAction.java:280) ~[elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.transport.TransportRequestHandler.messageReceived(TransportRequestHandler.java:30) ~[elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.xpack.security.transport.SecurityServerTransportInterceptor$ProfileSecuredRequestHandler$1.doRun(SecurityServerTransportInterceptor.java:259) ~[?:?]
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.xpack.security.transport.SecurityServerTransportInterceptor$ProfileSecuredRequestHandler.messageReceived(SecurityServerTransportInterceptor.java:317) ~[?:?]
at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:66) ~[elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.transport.TransportService$7.doRun(TransportService.java:664) ~[elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:725) ~[elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-6.3.2.jar:6.3.2]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_181]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_181]
at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_181]
[2018-08-06T16:04:28,008][WARN ][r.suppressed ] path: /my-index/my_doc/444/_termvectors, params: {index=my-index, id=444, type=my_doc}
org.elasticsearch.transport.RemoteTransportException: [T3sBXHE][127.0.0.1:9300][indices:data/read/tv[s]]
Caused by: org.elasticsearch.ElasticsearchException: failed to execute term vector request
at org.elasticsearch.index.termvectors.TermVectorsService.getTermVectors(TermVectorsService.java:150) ~[elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.index.termvectors.TermVectorsService.getTermVectors(TermVectorsService.java:77) ~[elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.action.termvectors.TransportTermVectorsAction.shardOperation(TransportTermVectorsAction.java:89) ~[elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.action.termvectors.TransportTermVectorsAction.shardOperation(TransportTermVectorsAction.java:43) ~[elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.action.support.single.shard.TransportSingleShardAction$ShardTransportHandler.messageReceived(TransportSingleShardAction.java:287) ~[elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.action.support.single.shard.TransportSingleShardAction$ShardTransportHandler.messageReceived(TransportSingleShardAction.java:280) ~[elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.transport.TransportRequestHandler.messageReceived(TransportRequestHandler.java:30) ~[elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.xpack.security.transport.SecurityServerTransportInterceptor$ProfileSecuredRequestHandler$1.doRun(SecurityServerTransportInterceptor.java:259) [x-pack-security-6.3.2.jar:6.3.2]
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.xpack.security.transport.SecurityServerTransportInterceptor$ProfileSecuredRequestHandler.messageReceived(SecurityServerTransportInterceptor.java:317) [x-pack-security-6.3.2.jar:6.3.2]
at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:66) [elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.transport.TransportService$7.doRun(TransportService.java:664) [elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:725) [elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-6.3.2.jar:6.3.2]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_181]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_181]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_181]
Caused by: java.lang.NullPointerException
at org.elasticsearch.action.termvectors.TermVectorsWriter.setFields(TermVectorsWriter.java:82) ~[elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.action.termvectors.TermVectorsResponse.setFields(TermVectorsResponse.java:361) ~[elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.index.termvectors.TermVectorsService.getTermVectors(TermVectorsService.java:146) ~[elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.index.termvectors.TermVectorsService.getTermVectors(TermVectorsService.java:77) ~[elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.action.termvectors.TransportTermVectorsAction.shardOperation(TransportTermVectorsAction.java:89) ~[elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.action.termvectors.TransportTermVectorsAction.shardOperation(TransportTermVectorsAction.java:43) ~[elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.action.support.single.shard.TransportSingleShardAction$ShardTransportHandler.messageReceived(TransportSingleShardAction.java:287) ~[elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.action.support.single.shard.TransportSingleShardAction$ShardTransportHandler.messageReceived(TransportSingleShardAction.java:280) ~[elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.transport.TransportRequestHandler.messageReceived(TransportRequestHandler.java:30) ~[elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.xpack.security.transport.SecurityServerTransportInterceptor$ProfileSecuredRequestHandler$1.doRun(SecurityServerTransportInterceptor.java:259) ~[?:?]
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.xpack.security.transport.SecurityServerTransportInterceptor$ProfileSecuredRequestHandler.messageReceived(SecurityServerTransportInterceptor.java:317) ~[?:?]
at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:66) ~[elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.transport.TransportService$7.doRun(TransportService.java:664) ~[elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:725) ~[elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-6.3.2.jar:6.3.2]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_181]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_181]
at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_181]

@colings86 colings86 added the :Search/Search Search-related issues that do not fall into other categories label Aug 7, 2018
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-search-aggs

@colings86 colings86 added the >bug label Aug 7, 2018
@colings86
Copy link
Contributor

@mayya-sharipova would you be able to have a look at this?

@kramarz
Copy link

kramarz commented Aug 24, 2018

I was running into a similar issue without "nested" in 6.3.2 But I wrote a script to reproduce it and after switching to 6.4.0 the problem is not there anymore 😆 . Then I also tested it with nested feature and it also seems to work. The script is here if anyone wants to check it : https://gist.github.com/kramarz/20844c1cb48fc745d806dff0c0016525

@rafaelglater
Copy link
Author

@kramarz I was also having trouble with non-nested fields (which I realized later, after opening this bug). Since my needs couldn't wait for this bug fix, I changed my code to request term vectors field by field separately and assigning "zero" in the cases where I got an error... This was time-consuming, by the way... Good to know it's fixed in version 6.4. I'll give a try. Thank you for letting me know!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
>bug :Search/Search Search-related issues that do not fall into other categories
Projects
None yet
Development

No branches or pull requests

5 participants