Phrase suggester collate_match field erased during result merging

[jbaiera]

When using a phrase suggester with a collate query and prune set to true, a suggested phrase may correspond to a successful collate search on only a couple of shards or a single shard. Since prune is enabled, the options from all shards are returned, even if failing the collate query.

When suggester options are merged at the end of a query, only the scores of the options are considered. The collate_match field values are ignored. This means that the suggester only reports that the collate_match is true for options returned from the first suggester result seen. This is mostly a problem in cases where suggested phrase is one that results in a rare hit.

It is only possible to see these false negatives in the result list when prune is enabled. Without prune enabled, the suggester will return the correct results, as the only suggestion options that are returned are those that pass the collate query.

https://github.com/elastic/elasticsearch/blob/master/server/src/main/java/org/elasticsearch/search/suggest/Suggest.java#L744

[elasticmachine]

Pinging @elastic/es-search-aggs

[colings86]

@jimczi could you take a look at this one?

[jbaiera]

Actually, I'm going to see if I can get together a fix for this. The fix itself is really simple, just |= the boolean. I'll see if I can get a PR for this up today.

[jimczi]

Thanks @jbaiera ! I agree this is a bug ;).

[jbaiera]

Fixed with #29514