Adjust JDBC source fetch size dynamically to avoid OOME

[tuliren]

Summary

• Implement the solution discussed in Research OOM errors in JDBC source connectors #11883
  • The assumption is that all the database rows have similar size.
  • Firstly, set the fetch size to 10. Fetch the 10 rows, measure the mean serialized size of those rows, and use that size to estimate the best fetch size (N).
  • Secondly, set the fetch size to N. Sample the serialized size for every 100 to 1000 rows to adjust the fetch size.
• Make sure the JDBC drivers respect the fetch size, rather than taking it as a hint.

[tuliren]

Results

Testing Postgres database: one table, one text column, 10MB per row

Without dynamic fetch size adjustment	With dynamic fetch size adjustment
Immediate OOME when trying to fetching 1000 rows	Stable memory consumption

[tuliren]

Still need to publish the new connector versions.