[improvement](jdbc catalog) Disallow non-constant type conversion pushdown and implicit conversion pushdown #42102
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…hdown and implicit conversion pushdown
|
Thank you for your contribution to Apache Doris. Since 2024-03-18, the Document has been moved to doris-website. |
|
run buildall |
morningman
reviewed
Oct 20, 2024
fe/fe-core/src/main/java/org/apache/doris/datasource/jdbc/source/JdbcScanNode.java
Outdated
Show resolved
Hide resolved
fe/fe-core/src/main/java/org/apache/doris/qe/SessionVariable.java
Outdated
Show resolved
Hide resolved
github-actions
bot
added
the
approved
|
PR approved by at least one committer and no changes requested. |
|
PR approved by anyone and no changes requested. |
|
run buildall |
|
run buildall |
github-actions
bot
removed
the
approved
github-actions
bot
added
the
approved
|
PR approved by at least one committer and no changes requested. |
CalvinKirs
approved these changes
Oct 22, 2024
zy-kkk
added a commit
to zy-kkk/doris
that referenced
this pull request
Oct 22, 2024
…hdown and implicit conversion pushdown (apache#42102) Add a variable `enable_jdbc_cast_predicate_push_down`, the default value is false, which prohibits the pushdown of non-constant predicates with type conversion and all predicates with implicit conversion. This change can prevent the wrong predicates from being pushed down to the Jdbc data source, resulting in query data errors, because the predicates with cast were not correctly pushed down to the data source before. If you find that the data is read correctly and the performance is better before this change, you can manually set this variable to true ``` | Expression | Can Push Down | |-----------------------------------------------------|---------------| | column type equals const type | Yes | | column type equals cast const type | Yes | | cast column type equals const type | No | | cast column type equals cast const type | No | | column type not equals column type | No | | column type not equals cast const type | No | | cast column type not equals const type | No | | cast column type not equals cast const type | No | ```
morningman
pushed a commit
that referenced
this pull request
Oct 22, 2024
…n pushdown and implicit conversion pushdown (#42242) pick (#42102) Add a variable `enable_jdbc_cast_predicate_push_down`, the default value is false, which prohibits the pushdown of non-constant predicates with type conversion and all predicates with implicit conversion. This change can prevent the wrong predicates from being pushed down to the Jdbc data source, resulting in query data errors, because the predicates with cast were not correctly pushed down to the data source before. If you find that the data is read correctly and the performance is better before this change, you can manually set this variable to true ``` | Expression | Can Push Down | |-----------------------------------------------------|---------------| | column type equals const type | Yes | | column type equals cast const type | Yes | | cast column type equals const type | No | | cast column type equals cast const type | No | | column type not equals column type | No | | column type not equals cast const type | No | | cast column type not equals const type | No | | cast column type not equals cast const type | No | ```
16 tasks
zy-kkk
added a commit
that referenced
this pull request
Nov 5, 2024
Related PR: #42102 Because we previously disabled the pushdown of cast, a test case error occurred when checking the pushdown of the clickhosue function. Because the column type is largeint, a cast will be generated when using from_unixtime, causing the function to fail to be pushed down. So we changed the column type to bigint.
github-actions bot
pushed a commit
that referenced
this pull request
Nov 5, 2024
Related PR: #42102 Because we previously disabled the pushdown of cast, a test case error occurred when checking the pushdown of the clickhosue function. Because the column type is largeint, a cast will be generated when using from_unixtime, causing the function to fail to be pushed down. So we changed the column type to bigint.
github-actions bot
pushed a commit
that referenced
this pull request
Nov 5, 2024
Related PR: #42102 Because we previously disabled the pushdown of cast, a test case error occurred when checking the pushdown of the clickhosue function. Because the column type is largeint, a cast will be generated when using from_unixtime, causing the function to fail to be pushed down. So we changed the column type to bigint.
morningman
pushed a commit
to morningman/doris
that referenced
this pull request
Nov 6, 2024
…hdown and implicit conversion pushdown (apache#42102) Add a variable `enable_jdbc_cast_predicate_push_down`, the default value is false, which prohibits the pushdown of non-constant predicates with type conversion and all predicates with implicit conversion. This change can prevent the wrong predicates from being pushed down to the Jdbc data source, resulting in query data errors, because the predicates with cast were not correctly pushed down to the data source before. If you find that the data is read correctly and the performance is better before this change, you can manually set this variable to true ``` | Expression | Can Push Down | |-----------------------------------------------------|---------------| | column type equals const type | Yes | | column type equals cast const type | Yes | | cast column type equals const type | No | | cast column type equals cast const type | No | | column type not equals column type | No | | column type not equals cast const type | No | | cast column type not equals const type | No | | cast column type not equals cast const type | No | ```
morningman
pushed a commit
to morningman/doris
that referenced
this pull request
Nov 6, 2024
…3196) Related PR: apache#42102 Because we previously disabled the pushdown of cast, a test case error occurred when checking the pushdown of the clickhosue function. Because the column type is largeint, a cast will be generated when using from_unixtime, causing the function to fail to be pushed down. So we changed the column type to bigint.
morningman
added a commit
that referenced
this pull request
Nov 6, 2024
…hdown and implicit conversion pushdown #42102 (#43313) cherry pick from #42102 Co-authored-by: zy-kkk <[email protected]>
16 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Add a variable
enable_jdbc_cast_predicate_push_down, the default value is false, which prohibits the pushdown of non-constant predicates with type conversion and all predicates with implicit conversion. This change can prevent the wrong predicates from being pushed down to the Jdbc data source, resulting in query data errors, because the predicates with cast were not correctly pushed down to the data source before.If you find that the data is read correctly and the performance is better before this change, you can manually set this variable to true