You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
Currently we have checks for time zones in a few places that disable timestamp support if the timezone is not what we want.
// Both of the Spark session time zone and JVM's default time zone should be UTC.
if (!TypeChecks.areTimestampsSupported(sessionZoneId)) {
willNotWorkOnGpu("Only UTC zone id is supported. "+
s"Actual session local zone id: $sessionZoneId")
}
valdefaultZoneId=ZoneId.systemDefault()
if (!TypeChecks.areTimestampsSupported(defaultZoneId)) {
willNotWorkOnGpu(s"Only UTC zone id is supported. Actual default zone id: $defaultZoneId")
}
}
In preparation to be able to support more time zones and doing it on a per operator basis we need to move these checks from the generic places that they are now, to individual expressions and operators.
This is going to be a lot of refactoring, but hopefully it will not be too difficult. The hard part will be testing this everywhere. Some operators that do not actually process the data, only send it along, will probably be allowed to work with timestamps.
We should pay special attention to Cast, because it does need a timezone, but only for a subset of the casts.
The text was updated successfully, but these errors were encountered:
Is your feature request related to a problem? Please describe.
Currently we have checks for time zones in a few places that disable timestamp support if the timezone is not what we want.
spark-rapids/sql-plugin/src/main/scala/com/nvidia/spark/rapids/GpuOverrides.scala
Lines 662 to 664 in 656e51a
spark-rapids/sql-plugin/src/main/scala/com/nvidia/spark/rapids/RapidsMeta.scala
Lines 374 to 385 in 656e51a
In preparation to be able to support more time zones and doing it on a per operator basis we need to move these checks from the generic places that they are now, to individual expressions and operators.
This is going to be a lot of refactoring, but hopefully it will not be too difficult. The hard part will be testing this everywhere. Some operators that do not actually process the data, only send it along, will probably be allowed to work with timestamps.
We should pay special attention to Cast, because it does need a timezone, but only for a subset of the casts.
The text was updated successfully, but these errors were encountered: