Do case insensitive comparison between dereferenced fields and internal ORC field names

[willmostly]

Trino will currently return NULLs if table metadata contains a struct that uses caps, e.g. entryId struct<custId:string,acctGuid:string,requestDate:string> and the ORC internal field definition does not. Reading the code, I actually suspect that NULL would be returned even if the internal field name did match the casing in the table def'n, but I have not tested this scenario.

[findepi]

We should definitely have a test for this.

[willmostly]

FYI - this product test will fail without the code change introduced here.

[findepi]

I added a label to run all Hive product tests and also made small cleanup in a test and added an additional assertion.
@willmostly Let me know the build results.

[findepi]

build was almost green (expcet main and tests which seemed unrelated). i did restart all of it to be on the safe side

[phd3]

@willmostly thanks for making the change, lgtm. Also, could you please squash your commits.

[phd3]

since this is now getting a bit longer at two places, we could extract it out to simplify. (I'd leave it up to you if you'd like to tackle it separately.)

    projectionsByColumnName = projections.stream()
            .collect(Collectors.groupingBy(
                        HiveColumnHandle::getBaseColumnName,
                        mapping(OrcPageSourceFactory::getDereferencesAsList, toList())));


    private static List<String> getDereferencesAsList(HiveColumnHandle column)
    {
        return column.getHiveColumnProjectionInfo()
                .map(info -> info.getDereferenceNames().stream()
                        .map(dereference -> dereference.toLowerCase(ENGLISH))
                        .collect(toList()))
                .orElse(ImmutableList.<String>of());
    }

[phd3]

onTrino.getConnection() and query( use different connections, so seems like disabling the pushdown doesn't get applied really. Can you use onTrino().executeQuery? (Or something similar to AbstractTestHiveViews#setSessionProperty)

[findepi]

CI hit #7535

[findepi]

Merged as 8bac015, thanks!

[aakashnand]

Even though this has been fixed. @willmostly @findepi I found out that. I was getting NULL even for small case nested struct type of data until the Trino version 355. For example, I created an internal table in a hive using the following JSON with added NULL values and I was still getting NULL when I executed select data.timeonsite from small_case_int_hdfs;.

Data

{"data":{"visits":"1","hits":"8","pageviews":"4","timeonsite":"107","sessionqualitydim":"99"}}
{"data":{"visits":"1","hits":"19","pageviews":"10","timeonsite":"250","sessionqualitydim":"56"}}
{"data":{"visits":"1","hits":"10","pageviews":"5","timeonsite":"1439","sessionqualitydim":"23"}}
{"data":{"visits":"1","hits":"10","pageviews":"5","sessionqualitydim":"45"}}
{"data":{"visits":"1","hits":"10","pageviews":"5","sessionqualitydim":"45"}}
{"data":{"visits":"1","hits":"19","pageviews":"10","sessionqualitydim":"56"}}
{"data":{"visits":"1","hits":"19","pageviews":"10","timeonsite":"250"}}
{"data":{"visits":"1","hits":"19","pageviews":"10","timeonsite":"250"}}
{"data":{"visits":"1","hits":"19","pageviews":"10","timeonsite":"250"}}
{"data":{"visits":"1","hits":"19","pageviews":"10","timeonsite":"250"}}

Terminal Output

trino:test_schema> select data.timeonsite from small_case_int_hdfs;
 timeonsite 
------------
 NULL       
 NULL       
 NULL       
 NULL       
 NULL       
 NULL       
 NULL       
 NULL       
 NULL       
 NULL    
(10 rows)
Query 20210713_080046_00000_a9fyn, FINISHED, 1 node
Splits: 17 total, 17 done (100.00%)
3.26 [10 rows, 0B] [7 rows/s, 0B/s] 

[findepi]

@aakashnand please consider creating a new issue to avoid this being forgotten about

[aakashnand]

@findepi my above comment was just a note and the problem is solved after this PR. Should I still create a new issue and say that it was fixed by this PR? I just wanted to highlight that the problem is not only for camel cases but also for a small cases.