Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(rust): raise schema mismatch when decimal is not subset #2330

Merged
merged 9 commits into from
Mar 28, 2024

Conversation

ion-elgreco
Copy link
Collaborator

@ion-elgreco ion-elgreco commented Mar 24, 2024

Description

The can_cast_types is a bit to flexible with decimals, since we define the decimal type in the table metadata. So if the source data contains a decimal that is larger than the decimal in the table metadata, we should throw an error. Before we would let our writer just write because can_cast_types does not check precision and scale for casting so Decimal(30.1) could be written after the table was set to only allow precision=1, scale=1.

After the write, the json parser would not be able to read the table anymore since it throws a decimal overflow, and rightfully so. This fix will prevent tables to get in these invalid states.

@ion-elgreco ion-elgreco enabled auto-merge (squash) March 24, 2024 10:57
@github-actions github-actions bot added binding/python Issues for the Python package binding/rust Issues for the Rust crate labels Mar 24, 2024
@ion-elgreco ion-elgreco changed the title fix(rust): throw schema mismatch when decimal doesn't fit in the decimal type of table fix(rust): raise schema mismatch when decimal is not subset Mar 24, 2024
@ion-elgreco ion-elgreco merged commit bb46c8a into delta-io:main Mar 28, 2024
23 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
binding/python Issues for the Python package binding/rust Issues for the Rust crate
Projects
None yet
2 participants