Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Spammed by deletion vector warnings #278

Closed
matt035343 opened this issue Jun 8, 2023 · 4 comments
Closed

[BUG] Spammed by deletion vector warnings #278

matt035343 opened this issue Jun 8, 2023 · 4 comments
Assignees

Comments

@matt035343
Copy link
Member

matt035343 commented Jun 8, 2023

When using the load function for auto_replenishment.demand, I get spammed with messages like:

[2023-06-08T11:28:30Z WARN deltalake::action::parquet_read] Unexpected field name `deletionVector` for remove action: Row { fields: [("path", Str("part-00001-551118bc-002c-4cfe-bdb8-c37605ba6179-c000.snappy.parquet")), ("deletionTimestamp", Long(1686194252522)), ("dataChange", Bool(false)), ("extendedFileMetadata", Bool(true)), ("partitionValues", MapInternal(Map { entries: [] })), ("size", Long(38622590)), ("deletionVector", Null)] }

This is due to delta upgrade to 2.4

Related to delta-io/delta-rs#928, delta-io/delta-rs#1094

@matt035343 matt035343 added the code/bug Something isn't working label Jun 8, 2023
@george-zubrienko george-zubrienko removed the code/bug Something isn't working label Jun 8, 2023
@george-zubrienko
Copy link
Contributor

george-zubrienko commented Jun 8, 2023

Upstream bug - hope for the best.

Also, tables that do not do merges do not seem to be affected by this.

@george-zubrienko
Copy link
Contributor

@matt035343 if you re-create the table in question- do warnings go away?

@matt035343
Copy link
Member Author

@matt035343 if you re-create the table in question- do warnings go away?

They reappeared over the weekend.

The issue is confirmed for proteus 1.3.7

@george-zubrienko
Copy link
Contributor

Closing as current release is not affected by this

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants