Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Timestamp type scan result is not correct #6831

Open
yma11 opened this issue Aug 14, 2024 · 1 comment
Open

Timestamp type scan result is not correct #6831

yma11 opened this issue Aug 14, 2024 · 1 comment
Labels
bug Something isn't working triage

Comments

@yma11
Copy link
Contributor

yma11 commented Aug 14, 2024

Backend

VL (Velox)

Bug description

Found by UT "Timestamp type" in VeloxOrcDataTypeValidationSuite.

2024-08-05T07:04:37.9907383Z - Timestamp type *** FAILED ***
2024-08-05T07:04:37.9907935Z   Results do not match for query:
2024-08-05T07:04:37.9911208Z   Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]]
2024-08-05T07:04:37.9915475Z   Timezone Env: 
2024-08-05T07:04:37.9915736Z   
2024-08-05T07:04:37.9915962Z   == Parsed Logical Plan ==
2024-08-05T07:04:37.9916335Z   'GlobalLimit 100
2024-08-05T07:04:37.9916617Z   +- 'LocalLimit 100
2024-08-05T07:04:37.9916924Z      +- 'Project ['timestamp]
2024-08-05T07:04:37.9917330Z         +- 'UnresolvedRelation [type1], [], false
2024-08-05T07:04:37.9917684Z   
2024-08-05T07:04:37.9917896Z   == Analyzed Logical Plan ==
2024-08-05T07:04:37.9918191Z   timestamp: timestamp
2024-08-05T07:04:37.9918455Z   GlobalLimit 100
2024-08-05T07:04:37.9918729Z   +- LocalLimit 100
2024-08-05T07:04:37.9919034Z      +- Project [timestamp#22489]
2024-08-05T07:04:37.9919382Z         +- SubqueryAlias type1
2024-08-05T07:04:37.9920450Z            +- View (`type1`, [byte#22478,short#22479,int#22480,long#22481L,float#22482,double#22483,decimal#22484,string#22485,binary#22486,bool#22487,date#22488,timestamp#22489,array#22490,struct#22491,map#22492])
2024-08-05T07:04:37.9922782Z               +- RelationV2[byte#22478, short#22479, int#22480, long#22481L, float#22482, double#22483, decimal#22484, string#22485, binary#22486, bool#22487, date#22488, timestamp#22489, array#22490, struct#22491, map#22492] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc
2024-08-05T07:04:37.9924240Z   
2024-08-05T07:04:37.9924461Z   == Optimized Logical Plan ==
2024-08-05T07:04:37.9924751Z   GlobalLimit 100
2024-08-05T07:04:37.9925016Z   +- LocalLimit 100
2024-08-05T07:04:37.9925902Z      +- RelationV2[timestamp#22489] orc file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scala-2.12/test-classes/data-type-validation-data/type1_orc
2024-08-05T07:04:37.9927209Z   
2024-08-05T07:04:37.9927563Z   == Physical Plan ==
2024-08-05T07:04:37.9928020Z   CollectLimit 100
2024-08-05T07:04:37.9928544Z   +- VeloxColumnarToRowExec
2024-08-05T07:04:37.9932170Z      +- ^(1285) BatchScanExecTransformer[timestamp#22489] OrcScan DataFilters: [], Format: orc, Location: InMemoryFileIndex(1 paths)[file:/__w/incubator-gluten/incubator-gluten/backends-velox/target/scal..., PartitionFilters: [], PushedAggregation: [], PushedFilters: [], PushedGroupBy: [], ReadSchema: struct<timestamp:timestamp>, PushedFilters: [], PushedAggregation: [], PushedGroupBy: [] RuntimeFilters: []
2024-08-05T07:04:37.9935679Z   
2024-08-05T07:04:37.9936024Z   == Results ==
2024-08-05T07:04:37.9936392Z   
2024-08-05T07:04:37.9936728Z   == Results ==
2024-08-05T07:04:37.9937353Z   !== Correct Answer - 1 ==   == Gluten Answer - 1 ==
2024-08-05T07:04:37.9937997Z    struct<>                   struct<>
2024-08-05T07:04:37.9938883Z   ![2013-04-08 13:10:15.0]    [2013-04-08 14:10:15.0] (GlutenQueryTest.scala:325)

Spark version

None

Spark configurations

No response

System information

Velox System Info v0.0.2
Commit: ac0a3ac
CMake Version: 3.28.3
System: Linux-5.4.0-156-generic
Arch: x86_64
CPU Name: Model name: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz
C++ Compiler: /usr/bin/c++
C++ Compiler Version: 9.4.0
C Compiler: /usr/bin/cc
C Compiler Version: 9.4.0
CMake Prefix Path: /usr/local;/usr;/;/usr/local/lib/python3.8/dist-packages/cmake/data;/usr/local;/usr/X11R6;/usr/pkg;/opt

\nThe results will be copied to your clipboard if xclip is installed.

Relevant logs

No response

@yma11 yma11 added bug Something isn't working triage labels Aug 14, 2024
@yma11
Copy link
Contributor Author

yma11 commented Aug 14, 2024

We may can double check this after Add session timezone to DWRF/ORC TimestampColumnWriter&TimestampColumnReader merged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working triage
Projects
None yet
Development

No branches or pull requests

2 participants
@yma11 and others