Add a cache for GatewayBackend to HaGatewayManager

[choiwaiyiu]

Description

Currently, Trino Gateway stops routing queries when the database is unavailable.

This PR adds a cache of GatewayBackend to the HaGatewayManager, ensuring that queries can still be routed even when the database is unavailable.

The cache is read-only, so update operations to GatewayBackend will still fail if the database is unavailable.

Release notes

( x) This is not user-visible or is docs only, and no release notes are required.

[cla-bot]

Thank you for your pull request and welcome to our community. We could not parse the GitHub identity of the following contributors: Choi Wai Yiu.
This is most likely caused by a git client misconfiguration; please make sure to:

1. check if your git client is configured with an email to sign commits git config --list | grep email
2. If not, set it up using git config --global user.email [email protected]
3. Make sure that the git commit email is configured in your GitHub account settings, see https://github.com/settings/emails

[cla-bot]

Thank you for your pull request and welcome to our community. We could not parse the GitHub identity of the following contributors: Choi Wai Yiu.
This is most likely caused by a git client misconfiguration; please make sure to:

1. check if your git client is configured with an email to sign commits git config --list | grep email
2. If not, set it up using git config --global user.email [email protected]
3. Make sure that the git commit email is configured in your GitHub account settings, see https://github.com/settings/emails

[xkrogen]

I am supportive of adding caching at this layer, but it's important to note that this alone won't be sufficient for the GW to continue routing traffic when the DB is unavailable, since writes happen in-the-loop of new query submission, e.g. here:

trino-gateway/gateway-ha/src/main/java/io/trino/gateway/proxyserver/ProxyRequestHandler.java

Line 275 in 110cd7c

queryHistoryManager.submitQueryDetail(queryDetail);

We need a more holistic strategy around how to handle DB unavailability which may include treating some writes as best-effort and skipping them in case of DB unavailability.

cc @surajkn

[xkrogen]

We should use something like a Guava/Caffeine cache here that has more proper cache management including cache expiration times. Remember that the same database will be shared by multiple GW instances. This code seems to assume that it can keep the cache in sync by simply invalidating the cache when update operations are made, but updates may be made out-of-band by other GW instances.

[choiwaiyiu]

@xkrogen
Thank you for your review. @oneonestar and I are currently working on addressing the issues in this PR based on your feedback and advice :)

[choiwaiyiu]

@xkrogen
Thanks for the review, and sorry for the late reply!
I’ve worked on fixing the cache inconsistency issue by changing how we use the cache.
As for the implementation details, Supplier might not be strictly necessary, but I added it to help with performance.

[choiwaiyiu]

cc: @oneonestar