Weird growth of KnowledgeCommunity size to 172 peers

[synctext]

Yesterday and today I was able to reproduce the following weirdness on latest V8.0 on Ubuntu 20.04 LTS. Use number of peers in 1 trouble community.

After a restart of Tribler you can see it slowly climbing (slow .GIF of 70 seconds):

Very weird, good luck with this one!

[synctext]

After a night of running Tribler the yield is 291 peers 😨 :

[qstokkink]

The theoretical maximum number of peers that should appear in the KnowledgeCommunity under normal IPv8 operation is 216. So, 291 peers is definitely not normal.

So, either there is a bug in IPv8 or the KnowledgeCommunity is misbehaving.

[qstokkink]

First, I verified that IPv8's internal peer walking/dropping was correctly working. There were no errors.

Second, I checked that the KnowledgeCommunity was not receiving messages from unknown peers. I added a check to on_message() and on_request(). The former had no issues. However, the latter (on_request()) receives a lot of messages from unknown peers:

src/tribler/core/knowledge/community.py", line 107, in on_request
    assert peer in self.get_peers()
AssertionError

[qstokkink]

Long story short: the KnowledgeCommunity is too aggressive in sending request-response messages. This keeps the connections to other peers open. To counter this, we'll need a more aggressive peer dropping mechanism.

[synctext]

Can we delete? Given the hopeful on-device GenAI progress.

[qstokkink]

@synctext Probably best to wait until we have RSS feeds again. Otherwise we run the risk of going metadata-poor.