fix: check SAF message inflight and check stored_at is in past

[sdbondi]

Description

• Keeps track of inflight SAF requests and only accepts responses for
  requests that are inflight
• Checks that stored_at is in the past
• Fixes Node should verify DHT message field "stored_at" #3412, Node should validate incoming SAF messages #3410

Motivation and Context

See #3412, #3410

How Has This Been Tested?

• New/existing unit/integration tests
• memorynet
• Manually

[SWvheerden]

LGTM,

This does seem to somewhat limit the poisoning that can happen to the LastSafMessageReceived variable, it can still happen. But we are still trusting in, information received and calculated by another peer.

[sdbondi]

Yeah if we look at the way it previously worked (before using a timestamp sent by a remote peer), there was no stored_at field but you essentially trusted that the peer gave you the latest messages, then marked the timestamp when the node shut down so that you would request messages from that time (while online you would receive propagated messages). There was still trust involved even without a timestamp. In the end, the protocol has to have the required rules to prevent abuse.

I think that message deliverability could be improved by only updating timestamp (based on offline timestamp or stored_at) only after the appropriate peers have been queried rather than just a single peer, this would also further limit "poisoning". I tend to think that this would be easier to implement with a comms RPC interface in a similar way to block sync et al.