You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
What happened:
The load_version does not work correctly when there exists the corresponding checkpoint of the user specified version. This is due to the condition is always evaluated as false.
What you expected to happen:
The load_version loads a specified version of a table particularly when there exists the corresponding checkpoint.
I don't believe it is, if you look at load_version, if the requested version is a checkpoint the state will get updated from the checkpoint, then when update_incremental is called it performs a check to see which version to load up to, however the table state version will equal the version requested and "fail" the check falling into getting the whole table state.
After loading the checkpoint if the table's version is equal to the requested update_incremental could not be called by load_version or itself could exit early.
Environment
Delta-rs version:
v.0.11.0
Binding:
Python
Environment:
Bug
What happened:
The load_version does not work correctly when there exists the corresponding checkpoint of the user specified version. This is due to the condition is always evaluated as
false
.What you expected to happen:
The load_version loads a specified version of a table particularly when there exists the corresponding checkpoint.
How to reproduce it:
Notebooks
More details:
$ tree _delta_log _delta_log ├── 00000000000000000000.checkpoint.parquet ├── 00000000000000000000.json ├── 00000000000000000001.json ├── 00000000000000000002.json ├── 00000000000000000003.json ├── 00000000000000000004.json ├── 00000000000000000005.json ├── 00000000000000000006.json ├── 00000000000000000007.json ├── 00000000000000000008.json ├── 00000000000000000009.json ├── 00000000000000000010.checkpoint.parquet ├── 00000000000000000010.json :
The text was updated successfully, but these errors were encountered: