adding fuzzy matching ability #10
Conversation
| var currentStart = 0 | ||
| words.foreach(word => { | ||
| val params = new ModifiableSolrParams() | ||
| params.add(CommonParams.Q, "tagname_str:" + word + "~" + fuzzinessValue) |
There was a problem hiding this comment.
"word" ought to be escaped. Use org.apache.solr.client.solrj.util.ClientUtils#escapeQueryChars
|
Hi @navd my apologies for not seeing this earlier. I just released v2 and was going through the project when I saw this PR. Solr v2 currently removes fuzzy matching because it is not in line with the streaming approach supported by SolrTextTagger, and based on results I got, not very good at what it does anyway. Solr v2 attempts to move the language analysis portion into Lucene. I am thinking of bringing back fuzzy search via my proposal described in issue 15. BTW, one problem with this code is that you are matching individual words against (potentially multi-word) entities in the dictionary, so there is scope for lots of false matches. Phrase chunking then matching entities with phrases are relatively low recall, so that is not ideal either. An idea might be to use phrases and stemmed matching. Also with the new Solr backend, we have access to the OpenNLP based Solr Analyzers, which might be a promising direction to look at. May even make more sense than my proposal on Issue 15. |
No description provided.