Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

improve debuggability of json ser/de errors #1119

Merged
merged 1 commit into from
Feb 11, 2023
Merged

Conversation

houqp
Copy link
Member

@houqp houqp commented Feb 5, 2023

Description

JSON serde errors are very hard to debug because they are usually displayed with the following vague error messages:

Error: Invalid JSON in log record: EOF while parsing a value at line 1 column 1                                                                                                              
                                                                                                                                                                                             
Caused by:                                                                                                                                                                                   
    EOF while parsing a value at line 1 column 1  

This PR separates JSON ser/de errors by different scenarios as well as adding more context to each error whenever applicable.

@github-actions github-actions bot added binding/rust Issues for the Rust crate rust labels Feb 5, 2023
@@ -332,6 +332,25 @@ impl MetaData {
}
}

impl TryFrom<DeltaTableMetaData> for MetaData {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

drive by refactor, makes more sense to keep all MetaData impl in the same place.

@houqp
Copy link
Member Author

houqp commented Feb 5, 2023

Looks like the python azure test has flaky timeout errors.

@houqp houqp enabled auto-merge (squash) February 5, 2023 04:42
Copy link
Collaborator

@wjones127 wjones127 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great! Thank you.

@houqp houqp merged commit 02d5d88 into delta-io:main Feb 11, 2023
@houqp houqp deleted the qp_json_serde branch February 11, 2023 02:28
chitralverma pushed a commit to chitralverma/delta-rs that referenced this pull request Mar 17, 2023
# Description

JSON serde errors are very hard to debug because they are usually
displayed with the following vague error messages:

```
Error: Invalid JSON in log record: EOF while parsing a value at line 1 column 1                                                                                                              
                                                                                                                                                                                             
Caused by:                                                                                                                                                                                   
    EOF while parsing a value at line 1 column 1  
```

This PR separates JSON ser/de errors by different scenarios as well as
adding more context to each error whenever applicable.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
binding/rust Issues for the Rust crate rust
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants