You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
The LEGACY timeParserPolicy setting affects how Spark parses timestamps from CSV files. We currently have no tests for parsing CSV files containing timestamps on the GPU with LEGACY mode enabled, so we don't know what our current behavior is.
Describe the solution you'd like
Add tests so that we understand the current behavior and then update our documentation to clarify our behavior and potentially file issues for follow on work to fix any bugs found.
Describe alternatives you've considered
None
Additional context
None
The text was updated successfully, but these errors were encountered:
CSV parsing uses a different code path to our CAST logic and does not have special handling for things like ansi mode or timeParserPolicy. Also, parsing from all non-string types is disabled by default and the documentation points out the reasons for this.
If we did want to support ansi/legacy in CSV parsing than we could consider a different approach where we just read strings from the csv file and then wrap in a projection that casts columns from string to another type.
Is your feature request related to a problem? Please describe.
The LEGACY timeParserPolicy setting affects how Spark parses timestamps from CSV files. We currently have no tests for parsing CSV files containing timestamps on the GPU with LEGACY mode enabled, so we don't know what our current behavior is.
Describe the solution you'd like
Add tests so that we understand the current behavior and then update our documentation to clarify our behavior and potentially file issues for follow on work to fix any bugs found.
Describe alternatives you've considered
None
Additional context
None
The text was updated successfully, but these errors were encountered: