Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

COPY TO fails on passing options for format through cli #9707

Closed
Tracked by #9682
tinfoil-knight opened this issue Mar 20, 2024 · 2 comments · Fixed by #9709
Closed
Tracked by #9682

COPY TO fails on passing options for format through cli #9707

tinfoil-knight opened this issue Mar 20, 2024 · 2 comments · Fixed by #9709
Assignees
Labels
bug Something isn't working regression Something that used to work no longer does

Comments

@tinfoil-knight
Copy link
Contributor

tinfoil-knight commented Mar 20, 2024

Describe the bug

COPY TO fails with Error: Invalid or Unsupported Configuration: Specify a format for TableOptions even when valid configuration options are provided for the format.

To Reproduce

Clone the main branch. In the datafusion-cli directory, run this command:

cargo run -- --command "COPY (VALUES (1, 'Foo'), (2, 'Bar')) TO 'test_files/scratch/copy/tab.parquet' STORED AS parquet OPTIONS ('format.compression' 'zstd(10)');"

Expected behavior

The command should run without error & the values should be copied to the provided destination.

Additional context

Running the same query in a SQL Logic Test (.slt) doesn't throw any error. See copy.slt which has tests with similar commands to the one described above.

The error originates here:
https://github.com/apache/arrow-datafusion/blob/ad8d552b9f150c3c066b0764e84f72b667a649ff/datafusion/common/src/config.rs#L1175-L1180

@tinfoil-knight tinfoil-knight added the bug Something isn't working label Mar 20, 2024
@tinfoil-knight
Copy link
Contributor Author

take

@alamb alamb added the regression Something that used to work no longer does label Mar 20, 2024
@alamb alamb mentioned this issue Mar 20, 2024
8 tasks
@alamb
Copy link
Contributor

alamb commented Mar 20, 2024

Thank you @tinfoil-knight

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working regression Something that used to work no longer does
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants