Fix PostgreSQL metadata storage warning logs because of tablename casing issues

[Akshat-Jain]

Description

PostgreSQL stores table names in lowercase, unless explicitly enclosed in double quotes. Because of this, the Druid tables like druid_pendingSegments get created as druid_pendingsegments.

This causes commands like ALTER TABLE and CREATE INDEX to behave incorrectly, and we get warning logs in overlord service.

Overlord logs for ALTER TABLE for druid_pendingSegments table

2024-10-15T15:31:03,646 WARN [main] org.apache.druid.metadata.SQLMetadataConnector - Exception Altering table[druid_pendingSegments]
org.skife.jdbi.v2.exceptions.CallbackFailedException: org.skife.jdbi.v2.exceptions.UnableToExecuteStatementException: java.sql.BatchUpdateException: Batch entry 0 ALTER TABLE druid_pendingSegments ADD COLUMN upgraded_from_segment_id VARCHAR(255) was aborted: ERROR: column "upgraded_from_segment_id" of relation "druid_pendingsegments" already exists  Call getNextException to see other errors in the batch. [statement:"null", located:"null", rewritten:"null", arguments:null]
	at org.skife.jdbi.v2.DBI.withHandle(DBI.java:284) ~[jdbi-2.63.1.jar:2.63.1]
	at org.apache.druid.metadata.SQLMetadataConnector.lambda$retryWithHandle$0(SQLMetadataConnector.java:157) ~[classes/:?]
	at org.apache.druid.java.util.common.RetryUtils.retry(RetryUtils.java:129) ~[classes/:?]
	at org.apache.druid.java.util.common.RetryUtils.retry(RetryUtils.java:81) ~[classes/:?]
	at org.apache.druid.java.util.common.RetryUtils.retry(RetryUtils.java:163) ~[classes/:?]
	at org.apache.druid.java.util.common.RetryUtils.retry(RetryUtils.java:153) ~[classes/:?]
	at org.apache.druid.metadata.SQLMetadataConnector.retryWithHandle(SQLMetadataConnector.java:157) ~[classes/:?]
	at org.apache.druid.metadata.SQLMetadataConnector.retryWithHandle(SQLMetadataConnector.java:167) ~[classes/:?]
	at org.apache.druid.metadata.SQLMetadataConnector.alterTable(SQLMetadataConnector.java:246) [classes/:?]
	at org.apache.druid.metadata.SQLMetadataConnector.alterPendingSegmentsTable(SQLMetadataConnector.java:509) [classes/:?]
	at org.apache.druid.metadata.SQLMetadataConnector.createPendingSegmentsTable(SQLMetadataConnector.java:297) [classes/:?]
	at org.apache.druid.metadata.SQLMetadataConnector.createPendingSegmentsTable(SQLMetadataConnector.java:751) [classes/:?]
	at org.apache.druid.metadata.IndexerSQLMetadataStorageCoordinator.start(IndexerSQLMetadataStorageCoordinator.java:146) [classes/:?]
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?]
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
	at java.base/java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
	at org.apache.druid.java.util.common.lifecycle.Lifecycle$AnnotationBasedHandler.start(Lifecycle.java:446) [classes/:?]
	at org.apache.druid.java.util.common.lifecycle.Lifecycle.start(Lifecycle.java:341) [classes/:?]
	at org.apache.druid.guice.LifecycleModule$2.start(LifecycleModule.java:152) [classes/:?]
	at org.apache.druid.cli.GuiceRunnable.initLifecycle(GuiceRunnable.java:136) [classes/:?]
	at org.apache.druid.cli.GuiceRunnable.initLifecycle(GuiceRunnable.java:94) [classes/:?]
	at org.apache.druid.cli.ServerRunnable.run(ServerRunnable.java:70) [classes/:?]
	at org.apache.druid.cli.Main.main(Main.java:112) [classes/:?]
Caused by: org.skife.jdbi.v2.exceptions.UnableToExecuteStatementException: java.sql.BatchUpdateException: Batch entry 0 ALTER TABLE druid_pendingSegments ADD COLUMN upgraded_from_segment_id VARCHAR(255) was aborted: ERROR: column "upgraded_from_segment_id" of relation "druid_pendingsegments" already exists  Call getNextException to see other errors in the batch. [statement:"null", located:"null", rewritten:"null", arguments:null]
	at org.skife.jdbi.v2.Batch.execute(Batch.java:131) ~[jdbi-2.63.1.jar:2.63.1]
	at org.apache.druid.metadata.SQLMetadataConnector.lambda$alterTable$3(SQLMetadataConnector.java:253) ~[classes/:?]
	at org.skife.jdbi.v2.DBI.withHandle(DBI.java:281) ~[jdbi-2.63.1.jar:2.63.1]
	... 23 more

Overlord logs for CREATE INDEX for druid_pendingSegments table

2024-10-15T15:31:03,655 INFO [main] org.apache.druid.metadata.SQLMetadataConnector - Creating Index on Table [druid_pendingSegments], sql: [CREATE INDEX idx_druid_pendingSegments_datasource_task_allocator_id ON druid_pendingSegments(dataSource,task_allocator_id)] 
2024-10-15T15:31:03,657 ERROR [main] org.apache.druid.metadata.SQLMetadataConnector - Exception while creating index on table [druid_pendingSegments]
org.skife.jdbi.v2.exceptions.CallbackFailedException: org.skife.jdbi.v2.exceptions.UnableToExecuteStatementException: org.postgresql.util.PSQLException: ERROR: relation "idx_druid_pendingsegments_datasource_task_allocator_id" already exists [statement:"CREATE INDEX idx_druid_pendingSegments_datasource_task_allocator_id ON druid_pendingSegments(dataSource,task_allocator_id)", located:"CREATE INDEX idx_druid_pendingSegments_datasource_task_allocator_id ON druid_pendingSegments(dataSource,task_allocator_id)", rewritten:"CREATE INDEX idx_druid_pendingSegments_datasource_task_allocator_id ON druid_pendingSegments(dataSource,task_allocator_id)", arguments:{ positional:{}, named:{}, finder:[]}]
	at org.skife.jdbi.v2.DBI.withHandle(DBI.java:284) ~[jdbi-2.63.1.jar:2.63.1]
	at org.apache.druid.metadata.SQLMetadataConnector.lambda$retryWithHandle$0(SQLMetadataConnector.java:157) ~[classes/:?]
	at org.apache.druid.java.util.common.RetryUtils.retry(RetryUtils.java:129) ~[classes/:?]
	at org.apache.druid.java.util.common.RetryUtils.retry(RetryUtils.java:81) ~[classes/:?]
	at org.apache.druid.java.util.common.RetryUtils.retry(RetryUtils.java:163) ~[classes/:?]
	at org.apache.druid.java.util.common.RetryUtils.retry(RetryUtils.java:153) ~[classes/:?]
	at org.apache.druid.metadata.SQLMetadataConnector.retryWithHandle(SQLMetadataConnector.java:157) ~[classes/:?]
	at org.apache.druid.metadata.SQLMetadataConnector.retryWithHandle(SQLMetadataConnector.java:167) ~[classes/:?]
	at org.apache.druid.metadata.SQLMetadataConnector.createIndex(SQLMetadataConnector.java:1082) [classes/:?]
	at org.apache.druid.metadata.SQLMetadataConnector.alterPendingSegmentsTable(SQLMetadataConnector.java:513) [classes/:?]
	at org.apache.druid.metadata.SQLMetadataConnector.createPendingSegmentsTable(SQLMetadataConnector.java:297) [classes/:?]
	at org.apache.druid.metadata.SQLMetadataConnector.createPendingSegmentsTable(SQLMetadataConnector.java:751) [classes/:?]
	at org.apache.druid.metadata.IndexerSQLMetadataStorageCoordinator.start(IndexerSQLMetadataStorageCoordinator.java:146) [classes/:?]
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?]
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
	at java.base/java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
	at org.apache.druid.java.util.common.lifecycle.Lifecycle$AnnotationBasedHandler.start(Lifecycle.java:446) [classes/:?]
	at org.apache.druid.java.util.common.lifecycle.Lifecycle.start(Lifecycle.java:341) [classes/:?]
	at org.apache.druid.guice.LifecycleModule$2.start(LifecycleModule.java:152) [classes/:?]
	at org.apache.druid.cli.GuiceRunnable.initLifecycle(GuiceRunnable.java:136) [classes/:?]
	at org.apache.druid.cli.GuiceRunnable.initLifecycle(GuiceRunnable.java:94) [classes/:?]
	at org.apache.druid.cli.ServerRunnable.run(ServerRunnable.java:70) [classes/:?]
	at org.apache.druid.cli.Main.main(Main.java:112) [classes/:?]

This PR overrides the problematic methods in the PostgreSQL metadata storage connector to use lowercased table names.

---

This PR has:

• been self-reviewed.
• added documentation for new or modified features or behaviors.
• a release note entry in the PR description.
• added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
• added or updated version, license, or notice information in licenses.yaml
• added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
• added unit tests or modified existing tests to cover new code paths, ensuring the threshold for code coverage is met.
• added integration tests.
• been tested in a test Druid cluster.

[AmatyaAvadhanula]

Thanks @Akshat-Jain. LGTM

[kfaraz]

@Akshat-Jain , the CI is failing due to low code coverage.

[Akshat-Jain]

@kfaraz Yeah, is that critical or can they be ignored? Asking since it doesn't seem trivial to test those methods since actually testing them would ideally require a Postgres connection setup IIUC. Any suggestions?

[kfaraz]

@Akshat-Jain , yeah, you are right. It might be an overkill as the tests will not be straightforward.
I guess we can merge this PR for now since the change is simple enough.