Revert revert 5131

[sudiksha27]

Goals (and why):
#5131 was reverted in #5181 as there were false positives. in #5131, it was (incorrectly) assumed the timestamps are strictly increasing with increasing sequence numbers. There is a possibility where consecutive seq numbers have the same timestamp. This is possible when a node gains leadership and does not know the value for latest sequence number - n . In this case, it forces agreement for nth sequence on the same value as that of (n-1)th or (n-2)th round (if value for n-1 is not known).
See PaxosTimestampBoundStore#getAgreedState for reference.

Implementation Description (bullets):
The invariant is now modified to - timestamps should not decrease with increasing sequence numbers.

Testing (What was existing testing like? What have you done to improve it?):
Added unit tests.
Also, I have validated the invariant on paxos logs from internal dev stack

Concerns (what feedback would you like?):
Corner cases I might have missed?

Where should we start reviewing?:
All of the code is same as in #5131 except the inequality check in LocalTimestampInvariantsVerifier#timestampInvariantsViolationLevel

Priority (whenever / two weeks / yesterday):
EOD tomorrow

[changelog-app]

Generate changelog in changelog/@unreleased

Type

• Feature
• Improvement
• Fix
• Break
• Deprecation
• Manual task
• Migration

Description

TimeLock now validates the expected invariant for its timestamp bounds i.e. the timestamp bound must not decrease with increasing sequence numbers.

Check the box to generate changelog(s)

• Generate changelog entry

[gmaretic]

Looks good!

[sudiksha27]

Merge #5183 before getting this in.

[svc-autorelease]

Released 0.283.1