Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Return error rather than panic when too many row groups are written #6629

Merged
merged 2 commits into from
Oct 28, 2024

Conversation

etseidl
Copy link
Contributor

@etseidl etseidl commented Oct 25, 2024

Which issue does this PR close?

Closes #6591.

Rationale for this change

Does not panic in the unlikely event that the row group ordinal is negative when writing bloom filters.

What changes are included in this PR?

Replace expect with map_err, also adds test for changes from #6378.

Are there any user-facing changes?

No, since the panic should no longer be possible to trigger.

@github-actions github-actions bot added the parquet Changes to the parquet crate label Oct 25, 2024
@etseidl
Copy link
Contributor Author

etseidl commented Oct 25, 2024

I wrote the test trying to track down #6591...if we don't want it then this can just be closed.

@tustvold tustvold merged commit 55fe847 into apache:master Oct 28, 2024
16 checks passed
@etseidl etseidl deleted the too_many_rg branch October 28, 2024 22:37
@alamb
Copy link
Contributor

alamb commented Oct 29, 2024

Thank you very much @tustvold and @etseidl

assert_eq!(i, 0x8000);
assert_eq!(
e.to_string(),
"Parquet error: Parquet does not support more than 32767 row groups per file (currently: 32768)"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💯

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
parquet Changes to the parquet crate
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Error Instead of Panic On Attempting to Write More Than 32769 Row Groups
3 participants