Unit test for lost CRMP ack messages

[pan-apple]

Problem

Need a test where the ack to a received message is dropped. It's expected that message reliability protocol will kick in, and do retransmits until it gets an ack from the receiver (or the retransmit count is exhausted).

Change overview

This PR adds a test where 2 ack messages are dropped. It's expected that the 2nd retry (1 original + 2 retry) will cause the ack to be received by the sender.

However, this test uncovered an issue. The test is currently failing with the following logs.

[1622585846951] [0xe1277] CHIP: [IN] Secure message was encrypted: Msg ID 4
[1622585846951] [0xe1277] CHIP: [IN] Sending msg from 0x000000000001E306 to 0x0000000006A11E3D at utc time: 72271140 msec
[1622585846951] [0xe1277] CHIP: [IN] Sending secure msg on generic transport
[1622585846951] [0xe1277] CHIP: [IN] Secure transport received message destined to fabric 1, node 0x0000000006A11E3D. Key ID 2
[1622585846951] [0xe1277] CHIP: [EM] Received message of type 1 and protocolId 65536 on exchange 44255
[1622585846951] [0xe1277] CHIP: [EM] ec id: 44255, Delegate: 0xea022210
[1622585846951] [0xe1277] CHIP: [IN] Secure msg send status No Error
[1622585847019] [0xe1277] CHIP: [IN] Sending msg from 0x000000000001E306 to 0x0000000006A11E3D at utc time: 72271209 msec
[1622585847019] [0xe1277] CHIP: [IN] Sending secure msg on generic transport
[1622585847019] [0xe1277] CHIP: [IN] Message counter verify failed, err = 4047
[1622585847019] [0xe1277] CHIP: [EM] Error receiving message from UDP:127.0.0.1:11097: Error 4047 (0x00000FCF)
[1622585847019] [0xe1277] CHIP: [IN] Secure msg send status No Error
[1622585847019] [0xe1277] CHIP: [EM] Retransmit MsgId:00000004 Send Cnt 1
[1622585847085] [0xe1277] CHIP: [IN] Sending msg from 0x000000000001E306 to 0x0000000006A11E3D at utc time: 72271274 msec
[1622585847085] [0xe1277] CHIP: [IN] Sending secure msg on generic transport
[1622585847085] [0xe1277] CHIP: [IN] Message counter verify failed, err = 4047
[1622585847085] [0xe1277] CHIP: [EM] Error receiving message from UDP:127.0.0.1:11097: Error 4047 (0x00000FCF)
[1622585847085] [0xe1277] CHIP: [IN] Secure msg send status No Error
[1622585847085] [0xe1277] CHIP: [EM] Retransmit MsgId:00000004 Send Cnt 2
../src/messaging/tests/TestReliableMessageProtocol.cpp:559: assertion failed: "rm->TestGetCountRetransTable() == 0"

The issue seems to be related to message counter check.

            if (mSynced.mWindow.test(offset))
            {
                return CHIP_ERROR_INVALID_ARGUMENT; // duplicated, in window
            }

The code above return failure for the retransmitted message, as it considers it as a duplicate message.

Testing

CheckResendApplicationMessageWithLostAcks unit test is supposed to test this condition.

[bzbarsky-apple]

It's not clear to me why this is enough. ReliableMessageContext has three functions keeping track of acks:

1. HasPeerRequestedAck()
2. IsAckPending()
3. GetPendingPeerAckId()

For standalone acks, it looks like HasAckPending() is used. For piggybacked acks, HasPeerRequestedAck() is checked. I guess in this case there is no actual response so the piggybacking does not come in? But it's still not clear to me why HasPeerRequestedAck() even exists and why its use in ExchangeMessageDispatch::SendMessage is correct (instead of testing IsAckPending() there).

In particular, once HasPeerRequestedAck() starts testing true it never goes back to false afaict... That seems pretty dubious.

If none of that has to do with this PR, please just let me know and I'll file a followup.

[pan-apple]

I think a follow up will be useful for HasPeerRequestedAck. This test is relying on standalone ack.

My thinking is that, if it's a piggyback ack, the sender of ack will probably request an ack as well (for the message on which ack is being piggybacked). So, if the ack is lost, so is the message on which it was piggybacked. And, that might trigger a retransmit from the ack sender. The overall test case might be trickier, but still doable. Need more analysis if the missing piggybacked ack itself can cause this kind of a deadlock.

[bzbarsky-apple]

Filed #7339 on my comments above.

[woody-apple]

@andy31415 @Damian-Nordic @LuDuda ?

[todo]

- Enable test for lost CRMP ack messages

connectedhomeip/src/messaging/tests/TestReliableMessageProtocol.cpp

Lines 560 to 570 in 0adb96d

	// TODO - Enable test for lost CRMP ack messages
	// The following check is commented out because of https://github.com/project-chip/connectedhomeip/issues/7292
	// NL_TEST_ASSERT(inSuite, rm->TestGetCountRetransTable() == 0);
	NL_TEST_ASSERT(inSuite, mockReceiver.IsOnMessageReceivedCalled);
	mockReceiver.mTestSuite = nullptr;
	err = ctx.GetExchangeManager().UnregisterUnsolicitedMessageHandlerForType(Echo::MsgType::EchoRequest);
	NL_TEST_ASSERT(inSuite, err == CHIP_NO_ERROR);
	rm->ClearRetransTable(rc);

---

This comment was generated by todo based on a TODO comment in 0adb96d in #7291. cc @pan-apple.

[pan-apple]

rebased

[woody-apple]

@saurabhst @Damian-Nordic @andy31415?