Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Kernel] Add ExpressionHandler.isSupported to check for expression support #2492

Merged
merged 3 commits into from
Mar 13, 2024

Conversation

vkorukanti
Copy link
Collaborator

@vkorukanti vkorukanti commented Jan 12, 2024

Description

Having the API to tell if an expression is supported on a given input schema and expected data type, can make the Kernel make better decisions in splitting the given query predicate to a guaranteed predicate and a best-effort predicate. The proposed API is:

    /**
     * Is the given expression evaluation supported on the data with given schema?
     *
     * @param inputSchema Schema of input data on which the expression is evaluated.
     * @param expression Expression to evaluate.
     * @param outputType Expected result data type.
     * @return true if supported and false otherwise.
     */
    boolean isSupported(StructType inputSchema, Expression expression, DataType outputType);

How was this patch tested?

Unittests

@@ -378,6 +378,7 @@ class GoldenTables extends QueryTest with SharedSparkSession {

val commitInfoFile = CommitInfo(
version = Some(0L),
None,
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is unrelated, but currently broken on the master. We don't compile when connectors when spark changes.

@vkorukanti vkorukanti merged commit c046547 into delta-io:master Mar 13, 2024
7 of 8 checks passed
@vkorukanti vkorukanti deleted the isSupportedExpr branch May 9, 2024 02:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants