Change tag search condition from `or` to `and` #6752

drew2a · 2022-02-02T06:10:17Z

This PR is related to #6708 and introduces a bit different logic for tag search than it was before.

As @kozlovsky suggested earlier, we should return 'intersection' of search results instead of 'union' in the case that more than one tag is specified in the search query.

This test that should clarify what I mean:

    @db_session
    async def test_get_infohashes(self):
        self.add_operation_set(
            self.db,
            {
                b'infohash1': [
                    Tag(name='tag1', count=SHOW_THRESHOLD),
                    Tag(name='tag2', count=SHOW_THRESHOLD)
                ],
                b'infohash2': [
                    Tag(name='tag1', count=SHOW_THRESHOLD)
                ],
                b'infohash3': [
                    Tag(name='tag2', count=SHOW_THRESHOLD)
                ]
            }
        )

        assert self.db.get_infohashes({'tag1', 'tag2'}) == [b'infohash1']

kozlovsky · 2022-02-02T10:15:05Z

src/tribler-core/tribler_core/components/tag/db/tag_db.py

-        return select(tt.torrent.infohash for tt in self.instance.TorrentTag
-                      if self._show_condition(tt) and tt.tag.name in tags).fetch()
+
+        # first, get all torrents that contains any tag from `tags`


issue: The result of a query that returns torrents with any tags may be several orders larger than the result of a query that returns torrents with all tags. Retrieving all these objects into memory may be very slow.

suggestion: It is better to select torrents with all tags directly in the SQL query.

SQL does not have a direct way to express the FORALL universal quantifier. To express the necessary condition, it is necessary to use a double negation: "select all torrents for which does not exist a tag from the list of required tags that is not associated with this torrent". It will be something like:

# torrents that contains all necessary tags query_results = select( torrent for torrent in self.instance.Torrent if not exists( tag for tag in self.instance.Tag if tag.name in tags and not exists( torrent_tag for torrent_tag in self.instance.TorrentTag if torrent_tag.torrent == torrent and torrent_tag.tag == tag and self._show_condition(torrent_tag) ) ) )

Queries with double not exists may be hard to read and understand, but they are the proper way to express the FORALL quantifier in SQL.