Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Encounter "com.linkedin.coral.hive.hive2rel.parsetree.parser.ParseException" while querying Hive View. #8789

Closed
Tracked by #5606
lxqfy opened this issue Aug 4, 2021 · 4 comments · Fixed by #8935
Closed
Tracked by #5606

Encounter "com.linkedin.coral.hive.hive2rel.parsetree.parser.ParseException" while querying Hive View. #8789

lxqfy opened this issue Aug 4, 2021 · 4 comments · Fixed by #8935
Labels
bug Something isn't working

Comments

@lxqfy
Copy link
Contributor

lxqfy commented Aug 4, 2021 •
edited
Loading

Hi,

I am trying to upgrade from prestosql-332 to trino-356. One of the Hive views that used to work with the old version.

SHOW CREATE VIEW in trino-356:

Query 20210804_130248_00442_8vahw failed: Failed to translate Hive view 'hive.test.hive_view1': com.linkedin.coral.hive.hive2rel.parsetree.parser.ParseException: line 1:0 cannot recognize input near '(' 'SELECT' '*'

SHOW CREATE VIEW in prestosql-332:

CREATE VIEW hive.test.hive_view1 AS
 (
    SELECT *
    FROM
      test.hive_table1
    WHERE ((((tz_type = 'local') AND (grass_region = 'XX')) AND (snap_date = DATE('2021-08-04'))) AND (grass_hour = 8))
 )

I am able to bypass the issue by set session hive.legacy_hive_view_translation=true;

Seem that it is due to the brackets in the view definition.
@hashhar
Copy link
Member

hashhar commented Aug 4, 2021

cc: @findepi @joshthoward Seems like Coral fails to translate DATE().

@findepi
Copy link
Member

findepi commented Aug 4, 2021

@lxqfy for completeness, what does SHOW CREATE VIEW in Hive return you?

@findepi findepi added the bug Something isn't working label Aug 4, 2021
@findepi findepi mentioned this issue Aug 4, 2021
Open
41 tasks
@lxqfy
Copy link
Contributor Author

lxqfy commented Aug 5, 2021

@lxqfy for completeness, what does SHOW CREATE VIEW in Hive return you?

# Storage Information
SerDe Library:          org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
InputFormat:            org.apache.hadoop.mapred.SequenceFileInputFormat
OutputFormat:           org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
Compressed:             No
Num Buckets:            -1
Bucket Columns:         []
Sort Columns:           []
Storage Desc Params:
        serialization.format    1

# View Information
View Original Text:     (

                                    SELECT *
                                    FROM test.hive_table1
                                    WHERE tz_type = 'local' AND grass_region = 'XX' AND snap_date = DATE('2021-08-04')
                                    AND grass_hour = 8

                                    )
View Expanded Text:     (

                                    SELECT *
                                    FROM test.hive_table1
                                    WHERE tz_type = 'local' AND grass_region = 'XX' AND snap_date = DATE('2021-08-04')
                                    AND grass_hour = 8

                                    )

@ebyhr
Copy link
Member

ebyhr commented Aug 18, 2021

Actually, there are two issues in a view definition. The first is DATE() function @hashhar already mentioned and the second is () that wrapping the entire SELECT statement.

※ The latest coral library 1.0.80 throws different message likes Unknown function name: date, but it still fails.

hive> create view test_view_date as select date('2021-01-01') as c1;

trino:default> select * from test_view_date;
Query 20210818_080020_00051_27wm3 failed: java.lang.NoSuchFieldError: HIVE_SUPPORT_SQL11_RESERVED_KEYWORDS
java.lang.RuntimeException: java.lang.NoSuchFieldError: HIVE_SUPPORT_SQL11_RESERVED_KEYWORDS
	at io.trino.dispatcher.LocalDispatchQueryFactory.lambda$createDispatchQuery$0(LocalDispatchQueryFactory.java:144)
	at io.trino.$gen.Trino_testversion____20210818_074312_3.call(Unknown Source)
	at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
	at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:69)
	at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.lang.NoSuchFieldError: HIVE_SUPPORT_SQL11_RESERVED_KEYWORDS
	at com.linkedin.coral.hive.hive2rel.parsetree.parser.HiveParser.useSQL11ReservedKeywordsForIdentifier(HiveParser.java:1000)
	at com.linkedin.coral.hive.hive2rel.parsetree.parser.HiveParser_IdentifiersParser.useSQL11ReservedKeywordsForIdentifier(HiveParser_IdentifiersParser.java:726)
	at com.linkedin.coral.hive.hive2rel.parsetree.parser.HiveParser_IdentifiersParser.identifier(HiveParser_IdentifiersParser.java:10107)
	at com.linkedin.coral.hive.hive2rel.parsetree.parser.HiveParser_IdentifiersParser.functionIdentifier(HiveParser_IdentifiersParser.java:10547)
	at com.linkedin.coral.hive.hive2rel.parsetree.parser.HiveParser_IdentifiersParser.synpred8_IdentifiersParser_fragment(HiveParser_IdentifiersParser.java:10978)
	at com.linkedin.coral.hive.hive2rel.parsetree.parser.HiveParser_IdentifiersParser.synpred8_IdentifiersParser(HiveParser_IdentifiersParser.java:11162)
	at com.linkedin.coral.hive.hive2rel.parsetree.parser.HiveParser_IdentifiersParser.functionName(HiveParser_IdentifiersParser.java:3378)
	at com.linkedin.coral.hive.hive2rel.parsetree.parser.HiveParser_IdentifiersParser.synpred11_IdentifiersParser_fragment(HiveParser_IdentifiersParser.java:11022)
	at com.linkedin.coral.hive.hive2rel.parsetree.parser.HiveParser_IdentifiersParser.synpred11_IdentifiersParser(HiveParser_IdentifiersParser.java:11260)
	at com.linkedin.coral.hive.hive2rel.parsetree.parser.HiveParser_IdentifiersParser$DFA32.specialStateTransition(HiveParser_IdentifiersParser.java)
	at org.antlr.runtime.DFA.predict(DFA.java:80)
	at com.linkedin.coral.hive.hive2rel.parsetree.parser.HiveParser_IdentifiersParser.atomExpression(HiveParser_IdentifiersParser.java:5394)
	at com.linkedin.coral.hive.hive2rel.parsetree.parser.HiveParser_IdentifiersParser.precedenceFieldExpression(HiveParser_IdentifiersParser.java:5600)
	at com.linkedin.coral.hive.hive2rel.parsetree.parser.HiveParser_IdentifiersParser.precedenceUnaryPrefixExpression(HiveParser_IdentifiersParser.java:5977)
	at com.linkedin.coral.hive.hive2rel.parsetree.parser.HiveParser_IdentifiersParser.precedenceUnarySuffixExpression(HiveParser_IdentifiersParser.java:6037)
	at com.linkedin.coral.hive.hive2rel.parsetree.parser.HiveParser_IdentifiersParser.precedenceBitwiseXorExpression(HiveParser_IdentifiersParser.java:6221)
	at com.linkedin.coral.hive.hive2rel.parsetree.parser.HiveParser_IdentifiersParser.precedenceStarExpression(HiveParser_IdentifiersParser.java:6381)
	at com.linkedin.coral.hive.hive2rel.parsetree.parser.HiveParser_IdentifiersParser.precedencePlusExpression(HiveParser_IdentifiersParser.java:6541)
	at com.linkedin.coral.hive.hive2rel.parsetree.parser.HiveParser_IdentifiersParser.precedenceAmpersandExpression(HiveParser_IdentifiersParser.java:6692)
	at com.linkedin.coral.hive.hive2rel.parsetree.parser.HiveParser_IdentifiersParser.precedenceBitwiseOrExpression(HiveParser_IdentifiersParser.java:6843)
	at com.linkedin.coral.hive.hive2rel.parsetree.parser.HiveParser_IdentifiersParser.precedenceEqualExpression(HiveParser_IdentifiersParser.java:7371)
	at com.linkedin.coral.hive.hive2rel.parsetree.parser.HiveParser_IdentifiersParser.precedenceNotExpression(HiveParser_IdentifiersParser.java:8380)
	at com.linkedin.coral.hive.hive2rel.parsetree.parser.HiveParser_IdentifiersParser.precedenceAndExpression(HiveParser_IdentifiersParser.java:8499)
	at com.linkedin.coral.hive.hive2rel.parsetree.parser.HiveParser_IdentifiersParser.precedenceOrExpression(HiveParser_IdentifiersParser.java:8656)
	at com.linkedin.coral.hive.hive2rel.parsetree.parser.HiveParser_IdentifiersParser.expression(HiveParser_IdentifiersParser.java:5322)
	at com.linkedin.coral.hive.hive2rel.parsetree.parser.HiveParser.expression(HiveParser.java:40409)
	at com.linkedin.coral.hive.hive2rel.parsetree.parser.HiveParser_SelectClauseParser.selectItem(HiveParser_SelectClauseParser.java:2414)
	at com.linkedin.coral.hive.hive2rel.parsetree.parser.HiveParser_SelectClauseParser.selectList(HiveParser_SelectClauseParser.java:1119)
	at com.linkedin.coral.hive.hive2rel.parsetree.parser.HiveParser_SelectClauseParser.selectClause(HiveParser_SelectClauseParser.java:874)
	at com.linkedin.coral.hive.hive2rel.parsetree.parser.HiveParser.selectClause(HiveParser.java:40397)
	at com.linkedin.coral.hive.hive2rel.parsetree.parser.HiveParser.selectStatement(HiveParser.java:36231)
	at com.linkedin.coral.hive.hive2rel.parsetree.parser.HiveParser.regularBody(HiveParser.java:36138)
	at com.linkedin.coral.hive.hive2rel.parsetree.parser.HiveParser.queryStatementExpressionBody(HiveParser.java:35170)
	at com.linkedin.coral.hive.hive2rel.parsetree.parser.HiveParser.queryStatementExpression(HiveParser.java:35047)
	at com.linkedin.coral.hive.hive2rel.parsetree.parser.HiveParser.execStatement(HiveParser.java:1519)
	at com.linkedin.coral.hive.hive2rel.parsetree.parser.HiveParser.statement(HiveParser.java:1079)
	at com.linkedin.coral.hive.hive2rel.parsetree.parser.CoralParseDriver.parse(CoralParseDriver.java:35)
	at com.linkedin.coral.hive.hive2rel.parsetree.ParseTreeBuilder.process(ParseTreeBuilder.java:165)
	at com.linkedin.coral.hive.hive2rel.parsetree.ParseTreeBuilder.processViewOrTable(ParseTreeBuilder.java:134)
	at com.linkedin.coral.hive.hive2rel.parsetree.ParseTreeBuilder.processView(ParseTreeBuilder.java:149)
	at com.linkedin.coral.hive.hive2rel.HiveToRelConverter.convertView(HiveToRelConverter.java:105)
	at io.trino.plugin.hive.ViewReaderUtil$HiveViewReader.decodeViewData(ViewReaderUtil.java:142)
	at io.trino.plugin.hive.HiveMetadata.lambda$getView$59(HiveMetadata.java:1996)
	at java.base/java.util.Optional.map(Optional.java:265)
	at io.trino.plugin.hive.HiveMetadata.getView(HiveMetadata.java:1990)
	at io.trino.plugin.base.classloader.ClassLoaderSafeConnectorMetadata.getView(ClassLoaderSafeConnectorMetadata.java:573)
	at io.trino.metadata.MetadataManager.getView(MetadataManager.java:1229)
	at io.trino.sql.analyzer.StatementAnalyzer$Visitor.visitTable(StatementAnalyzer.java:1324)
	at io.trino.sql.analyzer.StatementAnalyzer$Visitor.visitTable(StatementAnalyzer.java:361)
	at io.trino.sql.tree.Table.accept(Table.java:53)
	at io.trino.sql.tree.AstVisitor.process(AstVisitor.java:27)
	at io.trino.sql.analyzer.StatementAnalyzer$Visitor.process(StatementAnalyzer.java:378)
	at io.trino.sql.analyzer.StatementAnalyzer$Visitor.analyzeFrom(StatementAnalyzer.java:3184)
	at io.trino.sql.analyzer.StatementAnalyzer$Visitor.visitQuerySpecification(StatementAnalyzer.java:1956)
	at io.trino.sql.analyzer.StatementAnalyzer$Visitor.visitQuerySpecification(StatementAnalyzer.java:361)
	at io.trino.sql.tree.QuerySpecification.accept(QuerySpecification.java:155)
	at io.trino.sql.tree.AstVisitor.process(AstVisitor.java:27)
	at io.trino.sql.analyzer.StatementAnalyzer$Visitor.process(StatementAnalyzer.java:378)
	at io.trino.sql.analyzer.StatementAnalyzer$Visitor.process(StatementAnalyzer.java:388)
	at io.trino.sql.analyzer.StatementAnalyzer$Visitor.visitQuery(StatementAnalyzer.java:1168)
	at io.trino.sql.analyzer.StatementAnalyzer$Visitor.visitQuery(StatementAnalyzer.java:361)
	at io.trino.sql.tree.Query.accept(Query.java:107)
	at io.trino.sql.tree.AstVisitor.process(AstVisitor.java:27)
	at io.trino.sql.analyzer.StatementAnalyzer$Visitor.process(StatementAnalyzer.java:378)
	at io.trino.sql.analyzer.StatementAnalyzer.analyze(StatementAnalyzer.java:341)
	at io.trino.sql.analyzer.Analyzer.analyze(Analyzer.java:91)
	at io.trino.sql.analyzer.Analyzer.analyze(Analyzer.java:83)
	at io.trino.execution.SqlQueryExecution.analyze(SqlQueryExecution.java:269)
	at io.trino.execution.SqlQueryExecution.<init>(SqlQueryExecution.java:190)
	at io.trino.execution.SqlQueryExecution$SqlQueryExecutionFactory.createQueryExecution(SqlQueryExecution.java:806)
	at io.trino.dispatcher.LocalDispatchQueryFactory.lambda$createDispatchQuery$0(LocalDispatchQueryFactory.java:132)
	... 7 more
hive> create view test_view_parenthesis as (select 1 as c1);

trino:default> select * from test_view_parenthesis;
Query 20210818_075920_00050_27wm3 failed: Failed to translate Hive view 'default.test_view_parenthesis': com.linkedin.coral.hive.hive2rel.parsetree.parser.ParseException: line 1:0 cannot recognize input near '(' 'select' '1'

This was referenced Aug 20, 2021
Merged
Closed
Merged
@ebyhr ebyhr mentioned this issue Aug 26, 2021
Closed
10 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Upgrade Coral to version 1.0.89 ebyhr/trino
4 participants
@findepi @ebyhr @lxqfy @hashhar