Allow pyarrow.dataset.Expression in filters kwarg of deltalake.DeltaTable.to_pyarrow_table

[giacomorebecchi]

Description

I would like to have more flexibility in using the filters kwarg in the method to_pyarrow_table, in particular by providing directly a pyarrow.dataset.Expression.

Use Case
In my use case, this is due to the fact that I want to be able to filter out null values, particularly by using a version of this issue:

• [Python] Add support for "is" and "is not" to pyarrow.parquet.filters_to_expression apache/arrow#34504
  and, in order to be able to do this without using some exotic mocking, I would like to directly pass the Expression.

Related Issue(s)
Also, note that this PR might be an occasion to reduce code duplication by removing this function:

delta-rs/python/deltalake/table.py

Lines 333 to 350 in f041692

	def _filters_to_expression(filters: FilterType) -> Expression:
	"""
	Check if filters are well-formed and convert to an [pyarrow.dataset.Expression].
	"""
	if isinstance(filters[0][0], str):
	# We have encountered the situation where we have one nesting level too few:
	# We have [(,,), ..] instead of [[(,,), ..]]
	dnf = cast(FilterDNFType, [filters])
	else:
	dnf = cast(FilterDNFType, filters)
	dnf = _check_dnf(dnf, check_null_strings=False)
	disjunction_members = []
	for conjunction in dnf:
	conjunction_members = [
	_convert_single_predicate(col, op, val) for col, op, val in conjunction
	]
	disjunction_members.append(reduce(operator.and_, conjunction_members))
	return reduce(operator.or_, disjunction_members)

in favour of this:
https://github.com/apache/arrow/blob/69e8a78c018da88b60f9eb2b3b45703f81f3c93d/python/pyarrow/parquet/core.py#L135-L199