Fix issue #13341 ARP entry can be out of sync between kernel and APPL_DB if multiple updates are received from RTNL

[stephenxs]

What I did

Fix issue sonic-net/sonic-buildimage#13341 the issue that ARP entry is out of sync between kernel and APPL_DB

In AppRestartAssist::insertToMap, in case an entry has been updated more than once with the same value but different from the stored one, keep the state as NEW.

Eg.
Assume the entry's value that is restored from the warm reboot is V0, the following events received.

1. The first update with value V1 is received and handled by the if (found != appTableCacheMap[tableName].end()) branch,
   • the state is set to NEW
   • value is updated to V1
2. The second update with the same value V1 is received and handled by this branch
   • Originally, the state was set to SAME, which is wrong because V1 is different from the stored value V0
   • The correct logic should be: set the state to SAME only if the state is not NEW

This is a very rare case because most of times, the entry won't be updated multiple times

Signed-off-by: Stephen Sun [email protected]

Why I did it

To fix the issue.

How I verified it

Mock test is added to cover the case.

Manually test. with the fix in, the log message indicates the new ARP entry has been applied to APPL_DB (and below)

Jan 12 13:06:18.800339 mtbc-sonic-01-2410 NOTICE swss#neighsyncd: :- isNeighRestoreDone: neighbor table restore to kernel is done
Jan 12 13:06:18.804322 mtbc-sonic-01-2410 INFO swss#supervisord: neighsyncd Listens to neigh messages...
Jan 12 13:06:18.806320 mtbc-sonic-01-2410 INFO swss#neighsyncd: :- insertToMap: Received message NEIGH_TABLE, key: Vlan1000:192.168.0.2, neigh:24:8a:07:ab:a1:12, family:IPv4, , delete = 0
Jan 12 13:06:18.806647 mtbc-sonic-01-2410 NOTICE swss#neighsyncd: :- insertToMap: NEIGH_TABLE, found key: Vlan1000:192.168.0.2, new value 
Jan 12 13:06:21.684996 mtbc-sonic-01-2410 INFO swss#neighsyncd: :- insertToMap: Received message NEIGH_TABLE, key: Vlan1000:192.168.0.2, neigh:24:8a:07:ab:a1:12, family:IPv4, , delete = 0
Jan 12 13:06:21.684996 mtbc-sonic-01-2410 NOTICE swss#neighsyncd: :- insertToMap: NEIGH_TABLE, found key: Vlan1000:192.168.0.2, it has been updated for the second time, keep state as NEW
Jan 12 13:06:21.684996 mtbc-sonic-01-2410 INFO swss#neighsyncd: :- insertToMap: Received message NEIGH_TABLE, key: Vlan1000:192.168.0.2, neigh:24:8a:07:ab:a1:12, family:IPv4, , delete = 0
Jan 12 13:06:21.684996 mtbc-sonic-01-2410 NOTICE swss#neighsyncd: :- insertToMap: NEIGH_TABLE, found key: Vlan1000:192.168.0.2, it has been updated for the second time, keep state as NEW
Jan 12 13:06:23.800836 mtbc-sonic-01-2410 INFO swss#neighsyncd: :- checkReconcileTimer: warmstart timer expired
Jan 12 13:06:23.800836 mtbc-sonic-01-2410 NOTICE swss#neighsyncd: :- reconcile: NEIGH_TABLE NEW, key: Vlan1000:192.168.0.2, neigh:24:8a:07:ab:a1:12, family:IPv4, cache-state:NEW, 
Jan 12 13:06:23.802468 mtbc-sonic-01-2410 NOTICE swss#neighsyncd: :- setWarmStartState: neighsyncd warm start state changed to reconciled

Details if related

[prsunny]

lgtm, request review from @yxieca

[yxieca]

@prsunny do we need this change in 202205?

[prsunny]

@prsunny do we need this change in 202205?

i dont think so