Make Coral IR friendly to missing table alias prefix #449
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What changes are proposed in this pull request, and why are they necessary?
As Coral IR depends on the code of Calcite to generate
RelDataType
for each column, RelDataType of a column of struct type is always generated with StructKind.FULLY_QUALIFIED as shown in RelDataTypeFactoryImpl. ThereforeSqlValidator
from Calcite always requires a fully qualified name of a field from a column of struct type liketbl.structCol.field
. If the query defines a field without table alias prefix likestructCol.field
,SqlValidator
takesstructCol
as the table name and throw an exeption with the message oftable structCol not found
.In order to make Coral IR work without table alias prefix for a field from a column of struct type, the following changes are made:
CoraJavaTypeFactoryImpl
which extends the class ofJavaTypeFactoryImpl
(which is a subclass of RelDataTypeFactoryImpl) and it overrides the function ofcreateStructField
withStructKind.PEEK_FIELDS_NO_EXPAND
. Noted that the reason whyStructKind.PEEK_FIELDS_NO_EXPAND
is used not ``StructKind.PEEK_FIELDSor
StructKind.PEEK_FIELDS_DEFAULT` is the other two options would make the translation of `select *` not work as expected.RelDataTypeFactory
associated withRelOptCluster
is created in the code path ofFrameworks.withPrepare(FrameworkConfig config,BasePrepareAction<R> action)
which are all the code fromCalcite
which is called inRelBuilder.create()
, however we cannot change the code in Calcite to create an object ofCoralJavaTypeFactoryImpl
in this path. Therefore a new class ofHiveRelBuilder
is created to override the function ofRelBuilder.create()
where an object ofCoralJavaTypeFactoryImpl
and associatedRelOptCluster
is created to replace the object ofJavaTypeFactoryImpl
and associatedRelOptCluster
which are created in the code path of Calcite to create an object ofRelBuilder
. It is used byHiveToRelConverter
.How was this patch tested?
HiveToTrinoConverterTest.testSqlSelectAliasAppenderTransformerWithoutTableAliasPrefix
,CoralSparkTest.testStructColProjectionWithoutTableAliasPrefix
andCoralSparkTest.testStructColProjectionWithTableAliasPrefix