Investigate use of the BM25 algorithm to search image titles (original #288)

[obulat]

This issue has been migrated from the CC Search API repository

Author: kgodey
Date: Sat Apr 27 2019
Labels: ✨ goal: improvement,🏷 status: label work required,🙅 status: discontinued

The similarity algorithm used to search titles was switched from BM25 to boolean in cc-archive/cccatalog-api#281 to avoid ranking repeated words in titles higher.

We should investigate switching back to BM25 and set the k1 tuning value to a low value just for the title field.

See cc-archive/cccatalog-api#281 (review) and BM25 algorithm docs for more info.

---

Original Comments:

annatuma commented on Thu Jan 23 2020:

@aldenstpage I'm putting this in Q2 of the backlog, given that there are other search algorithm improvements scheduled for then. Please evaluate if this is a fit for community contributions and if so, label it accordingly.
source

[sarayourfriend]

This is effectively blocked on #3599, so that we can see what the actual effect of this kind of change would be.

It's also worth considering whether this is really more of a project than a one-off issue, but we'll wait and see what the blocking project gives us in the way of measurement tools and how much more beyond basic a/b testing rigging we'd have to do to actually test this kind of change.