Introduce the similarity as boost functionality to the Word2VecSynonyFilter #12433
Conversation
There was a problem hiding this comment.
Thanks @dantuzi!
This change looks good to me -- I just left a comment about back compat.
The naming conflict is "fun", and I think it's OK to rename the "old" BoostAttribute since it is @lucene.internal anyways.
The PR is marked as WIP still (why?), and also now has conflicts. @dantuzi do you want to refresh it? Thanks!
| @@ -62,14 +65,16 @@ public Word2VecSynonymFilter( | |||
| TokenStream input, | |||
| Word2VecSynonymProvider synonymProvider, | |||
| int maxSynonymsPerTerm, | |||
| float minAcceptedSimilarity) { | |||
| float minAcceptedSimilarity, | |||
| boolean similarityAsBoost) { | |||
There was a problem hiding this comment.
Should we add a back-compat ctor that doesn't take similarityAsBoost, and defaults it to false I guess?
|
Thanks @mikemccand for your feedback. |
Description
This is the follow-up of #12169
In the Word2VecSynonymFilter, when we extract the synonyms of a term, we have the cosine similarity between the vector associated with the original term and the vector associated with the synonym. The higher the cosine similarity is, the closer the meaning of the two terms.
In this PR we want to add the possibility of using the similarity value as a boost for synonym terms