Support predicate pushdown and deletion for non-identity partition columns in Iceberg

[phd3]

Currently, filters on non-identity partition columns are not utilized in Iceberg Connector predicate pushdown. Metadata deletion queries don't work on these:

CREATE TABLE customer_accounts (order_date DATE,account_number BIGINT) WITH (partitioning = ARRAY['month(order_date)'])

-- doesn't work
DELETE FROM customer_accounts WHERE date_trunc('month', order_date) = date_trunc('month', DATE '2018-06-01')

We can improve the SELECT queries for such cases by optimizing split scheduling. The constraint (with predicate) needs to be propagated to IcebergSplitManager through IcebergTableHandle and used to evaluate partition spec values inside FileScanTask to see if it matches the filter. However, it's currently not possible to tell the engine that such a filter is fully consumed. So supporting metadata delete in this case will require a bigger engine side change.

[findepi]

Requires #7608

[findepi]

We can improve the SELECT queries for such cases by optimizing split scheduling. The constraint (with predicate) needs to be propagated to IcebergSplitManager through IcebergTableHandle and used to evaluate partition spec values inside FileScanTask to see if it matches the filter.

Done in #9309 cc @homar

However, it's currently not possible to tell the engine that such a filter is fully consumed.

Besides this one, expression-based pushdown could help with subsequent dereference pushdown and also help with OPTIMIZE (#12362 cc @erichwang @losipiuk @alexjo2144 )

[kekwan]

Is predicate pushdown for identity partition columns enabled already?