fix(tests): Update timeout for Zebra sync tests

[oxarbitrage]

Motivation

Currently Zebra CI is failing due to multiple timeouts.

Close #4910

Solution

Increase the full sync timeout from 16 hours to 20 hours.
Increase the update sync timeout from 1-3 hours to 11 hours.

Review

Anyone can review, if the CI pass i think we should merge this fast as it is blocking all the other PRs.

Reviewer Checklist

• CI fully pass

Follow Up Work

Update the checkpoints.

[codecov]

Codecov Report

Merging #4918 (225f520) into main (52fa867) will decrease coverage by 0.05%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##             main    #4918      +/-   ##
==========================================
- Coverage   79.09%   79.03%   -0.06%     
==========================================
  Files         309      309              
  Lines       38781    38781              
==========================================
- Hits        30673    30651      -22     
- Misses       8108     8130      +22     

[teor2345]

I'm still seeing a timeout in the Zebra update to tip test, even with the updated checkpoints in PR #4919:

2022-08-18T23:00:06.180256Z INFO {net="Main"}: zebrad::components::sync::progress: estimated progress to chain tip sync_percent=99.754% current_height=Height(1773526) network_upgrade=Nu5 remaining_sync_blocks=4378 time_since_last_state_block=0s
Error:
0: stdout of command did not contain any matches for the given regex
Location:
/opt/zebrad/zebra-test/src/command.rs:779
Command:
"/opt/zebrad/target/release/zebrad" "-c" "/tmp/zebrad_testsBaar2c/zebrad.toml" "start"

Match Regex:
[
"finished initial sync to chain tip, using gossiped blocks .sync_percent.=.*100\.",
]

https://github.com/ZcashFoundation/zebra/runs/7918830518?check_suite_focus=true#step:6:143

[teor2345]

We also need to increase timeouts for the failing Zebra update to tip and lightwalletd full sync tests.

We probably need to update the timeouts for all the lightwalletd tests that sync Zebra's cached state. Some of them won't run until the lightwalletd full sync passes, but they'll have the same issue.

[teor2345]

Alfredo and I updated all the timeouts for the failing tests.

If CI passes, this will merge automatically.

[teor2345]

The regex for the 1740k job was wrong, it was expecting Zebra to sync to 1740k, but our cached state is now after 1740k.

This makes the update sync workflows run their whole Zebra sync in that job, then fail with a GitHub timeout.

Instead, we want to finish that job when we reach 1740k or later.

[teor2345]

I merged this manually, because:

• it succeeded the first time, but failed in mergify with an intermittent panic
• we have a plan for diagnosing and fixing the panic