release-23.1: sql: add prepared_statements_cache_size setting

[michae2]

Backport 1/1 commits from #98917.

/cc @cockroachdb/release

Release justification: high priority business need for the functionality (fix for customer).

---

Add a new circular doubly-linked list of prepared statements to prepStmtNamespace which tracks the least-recently-used prepared statement. When new setting prepared_statements_cache_size is set, use this LRU list to automatically deallocate prepared statements.

Fixes: #97866

Epic: None

Release note (sql change): Add a new prepared_statements_cache_size setting which, when set to a non-zero number of bytes, causes the least-recently-used prepared statements to be automatically deallocated when prepared statement memory usage goes above the cache size. This setting can be used to avoid prepared statement leaks from long-lived connections which never DEALLOCATE prepared statements.

[blathers-crl]

Thanks for opening a backport.

Please check the backport criteria before merging:

• Patches should only be created for serious issues or test-only changes.
• Patches should not break backwards-compatibility.
• Patches should change as little code as possible.
• Patches should not change on-disk formats or node communication protocols.
• Patches should not add new functionality.
• Patches must not add, edit, or otherwise modify cluster versions; or add version gates.

If some of the basic criteria cannot be satisfied, ensure that the exceptional criteria are satisfied within.

• There is a high priority need for the functionality that cannot wait until the next release and is difficult to address in another way.
• The new functionality is additive-only and only runs for clusters which have specifically “opted in” to it (e.g. by a cluster setting).
• New code is protected by a conditional check that is trivial to verify and ensures that it only runs for opt-in clusters.
• The PM and TL on the team that owns the changed code have signed off that the change obeys the above rules.

Add a brief release justification to the body of your PR to justify this backport.

Some other things to consider:

• What did we do to ensure that a user that doesn’t know & care about this backport, has no idea that it happened?
• Will this work in a cluster of mixed patch versions? Did we test that?
• If a user upgrades a patch version, uses this feature, and then downgrades, what happens?

[cockroach-teamcity]

This change is 

[michae2]

(Hold off on reviewing just yet.)

[michae2]

I don't feel strongly about enabling this by default, but I would like to get it into 23.1, so I think this is RFAL.

[yuzefovich]

I don't have a strong opinion either about the default. Perhaps we'll wait to see whether we get another report where the unlimited memory usage of prepared statements is a problem, and if it happens, then we switch the default. At least now (assuming recent enough version) we should have decent observability in place to investigate this problem.

I do agree with you that we should get this into 23.1.0.

Also didn't review too carefully.

Reviewed 16 of 16 files at r1, all commit messages.
Reviewable status: complete! 0 of 0 LGTMs obtained (waiting on @mgartner)

[michae2]

I'm hoping to merge this to release-23.1 before #99663 (which hasn't yet been committed to master, but will be soon, I think).

[mgartner]

LGTM. I think leaving it off by default for v23.1 is good.

[rytaft]

Reviewable status: complete! 1 of 0 LGTMs obtained (waiting on @mgartner and @yuzefovich)