Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

File 'supplier.parquet' does not match the expected extension '.parquet' #8264

Open
andygrove opened this issue Nov 18, 2023 · 2 comments
Open
Labels
bug Something isn't working

Comments

@andygrove
Copy link
Member

Describe the bug

I am testing Ballista with DataFusion 33 and getting this confusing error:

File 'supplier.parquet' does not match the expected extension '.parquet'

I am pretty sure that supplier.parquet does end with .parquet.

Here is the code that produces this error message:

let file_name = path.prefix().filename().unwrap_or_default();
if !path.as_str().ends_with(&option_extension) && file_name.contains('.') {
    return exec_err!(
        "File '{file_name}' does not match the expected extension '{option_extension}'"
    );
}

The code checks to see if path.as_str() ends with the extension, but the error message shows file_name instead of path.as_str(), so this seems to be misleading. I think it would be better to also show the value of path.as_str() in the error message to make this easier to debug.

I also wonder whether it is intentional that we are checking path.as_str() rather than file_name?

To Reproduce

This is not easy to reproduce via Ballista.

Expected behavior

I expect the error message to give me all of the context necessary so that I can debug the issue (which may well be due to a Ballista bug in this case).

Additional context

No response

@andygrove andygrove added the bug Something isn't working label Nov 18, 2023
@andygrove
Copy link
Member Author

@Weijun-H fyi

@andygrove
Copy link
Member Author

If I change the error to show path.as_str() rather than file_name then I get this error, which makes it obvious what the issue is:

File 'file:///mnt/bigdata/tpch/sf10-parquet/supplier.parquet/' does not match the expected extension '.parquet'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant