Limit max. nr. of PersistableNetworkPayload and ProtectedStorageEntries

[chimp1984]

To avoid that seed nodes get overloaded with requests for too many
PersistableNetworkPayload and ProtectedStorageEntry data we limit nr. of
entries to max 10000.

[ripcurlx]

utACK

[julianknutsen]

Is the UpdateDataRequest the only way to retrieve items > 10,000? What if the total set is larger than what the Initial & Update can handle? Is there a periodic on as well to sync over time?

[julianknutsen]

This leads to data structure inconsistency in the event that two consumers listen for the same Payload type and one consumer calls into this function to remove the data. It also breaks encapsulation having the consumers know about the internals of the P2PDataStore. This isn't perfect now, anyway, but moving towards a model where P2PDataStore is private to P2PService may be a good goal.

It seems like a cleaner way to do the same thing would be to have the P2PDataStore validate each stored entry prior to becoming "active" or signaling listeners. The ProtectedStorageEntry and/or payload could implement an interface that would do this check and remove the data before it was every available for consumers.

Is that what you were thinking with the "generic validation method"? Doing this would dovetail well with my existing refactoring and I can probably try it out and see how it looks. I think it would give a lot more flexibility for the future if there are certain Entrys or Payloads that we need to purge but made it past prior validation. Here is just one example I found while testing:

bisq/p2p/src/test/java/bisq/network/p2p/storage/payload/ProtectedMailboxStorageEntryTest.java

Lines 96 to 109 in 0ab156f

	// TESTCASE: validForAddOperation() should fail if Entry.receiversPubKey and Payload.ownerPubKey don't match
	// XXXBUGXXX: The current code doesn't validate this mismatch, but it would create an added payload that could never
	// be removed since the remove code requires Entry.receiversPubKey == Payload.ownerPubKey
	@Test
	public void isValidForAddOperation_EntryReceiverPayloadReceiverMismatch() throws NoSuchAlgorithmException, CryptoException {
	KeyPair senderKeys = TestUtils.generateKeyPair();
	KeyPair receiverKeys = TestUtils.generateKeyPair();
	MailboxStoragePayload mailboxStoragePayload = buildMailboxStoragePayload(senderKeys.getPublic(), receiverKeys.getPublic());
	ProtectedStorageEntry protectedStorageEntry = buildProtectedMailboxStorageEntry(mailboxStoragePayload, senderKeys, senderKeys.getPublic(), 1);
	// should be assertFalse
	Assert.assertTrue(protectedStorageEntry.isValidForAddOperation());
	}