-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Overwritten Format Configs by CreateExternalTable Options #9945
Comments
After the new way of setting the configs #9382, I believe we should eliminate them now. |
Is there any way to retain the |
That might be a good idea. Converting |
Given how long |
Sorry if my comments here were misinterpreted to mean that I am happy to help facilitate such a conversation if that would help |
Describe the bug
datafusion/datafusion/core/src/datasource/listing_table_factory.rs
Lines 67 to 73 in 4bd7c13
These lines of
ListingTableFactory.create()
overwrites the config options by cmd (CreateExternalTable options). Configs coming fromOPTIONS('format...
are discarded silently.To Reproduce
works as expected, but
does not, since
CREATE EXTERNAL TABLE
does not haveWITH HEADER ROW
, which overwritescsv.has_header
tofalse
.Expected behavior
The actual problem is that we can set the same settings by 2 different commands now, and one of them is silently chosen. Their default values are also different (CreateExternalTable's false, CsvOptions' true). We can set both of them as false initially. Then if one of them is true, then we expect a header. If
WITH HEADER ROW
exists and'format.has_header' 'false'
, we can give an error.Additional context
No response
The text was updated successfully, but these errors were encountered: