Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add serde support for CSV FileTypeWriterOptions #8641

Merged
merged 5 commits into from
Dec 28, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 18 additions & 0 deletions datafusion/proto/proto/datafusion.proto
Original file line number Diff line number Diff line change
Expand Up @@ -1207,6 +1207,7 @@ message FileTypeWriterOptions {
oneof FileType {
JsonWriterOptions json_options = 1;
ParquetWriterOptions parquet_options = 2;
CsvWriterOptions csv_options = 3;
}
}

Expand All @@ -1218,6 +1219,23 @@ message ParquetWriterOptions {
WriterProperties writer_properties = 1;
}

message CsvWriterOptions {
// Optional column delimiter. Defaults to `b','`
string delimiter = 1;
// Whether to write column names as file headers. Defaults to `true`
bool has_header = 2;
// Optional date format for date arrays
string date_format = 3;
// Optional datetime format for datetime arrays
string datetime_format = 4;
// Optional timestamp format for timestamp arrays
string timestamp_format = 5;
// Optional time format for time arrays
string time_format = 6;
// Optional value to represent null
string null_value = 7;
}

message WriterProperties {
uint64 data_page_size_limit = 1;
uint64 dictionary_page_size_limit = 2;
Expand Down
213 changes: 213 additions & 0 deletions datafusion/proto/src/generated/pbjson.rs

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

29 changes: 28 additions & 1 deletion datafusion/proto/src/generated/prost.rs

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading