[rel-v0.28] Full snapshot lease update retry on failure

[anveshreddy18]

What this PR does / why we need it:

cherry-pick of PR #711, #720, and #744 combined together

Which issue(s) this PR fixes:
Fixes #

Special notes for your reviewer:

Release note:

Introduce periodic updates to the Full Snapshot Lease, addressing delays in lease updates during failures

[renormalize]

Thanks for the PR @anveshreddy18.

Could you update the description to reflect that this PR is a cherry-pick of #711, #720, and #744?

I also feel that reviewing cherry-pick PRs would be easier if the cherry-picks aren't squashed before creating the PR.
That way, each commit can be verified to correspond to the diffs that a particular merged PR to master brought.

It would also maintain the same commit messages (when squashed and merged to this branch) that are present in the master branch, so it is easy to map changes in this branch and master.

[anveshreddy18]

@renormalize Out of these three PR's, the first one contains most of the functionality and since the other two PR's were just reverts and don't add anything, it was advised by @ishan16696 to not add those commits. But I agree it would make the reviewer's life easier in general, but in this case, since these are reverts, I didn't want to include them as commits. I hope this is fine.

[renormalize]

Verified to have the same diff as the three PRs mentioned. Thanks.

[ishan16696]

Out of these three PR's, the first one contains most of the functionality and since the other two PR's were just reverts and don't add anything, it was advised by @ishan16696 to not add those commits

Just 1 nit-pick, this PR can be splitted into 2 commits. 1 commit for cherry-pick and another commit for all reverts on top of that cherry-pick. May be I wasn't able to convey it properly. Anyway it look good.

[ishan16696]

LGTM!!

[ashwani2k]

Tested the scenario and it tries to renew the full snapshot lease every 1 min now.
We can merge this one.