You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
What happened:
Ingesting the delta table stored on the azure data lake, made by the deltalake pakage on azure databricks leads to databricks not picking up the delta tables properly; it loads up all of the parquet files instead of reading it as delta table.
What you expected to happen:
Only the latest version of the delta table is ingested
How to reproduce it:
Created a pandas dataframe, write to azure multiple times using overwrrite,
More details:
Creating a ticket with databricks as well to resolve the issue.
The text was updated successfully, but these errors were encountered:
Environment
Delta-rs version:
0.18.2
Binding:
Not sure
Environment:
Cloud provider:
Azure Data Lake Gen2
OS:
Windows 11
Other:
Bug
What happened:
Ingesting the delta table stored on the azure data lake, made by the deltalake pakage on azure databricks leads to databricks not picking up the delta tables properly; it loads up all of the parquet files instead of reading it as delta table.
What you expected to happen:
Only the latest version of the delta table is ingested
How to reproduce it:
Created a pandas dataframe, write to azure multiple times using overwrrite,
More details:
Creating a ticket with databricks as well to resolve the issue.
The text was updated successfully, but these errors were encountered: