Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: on append, overwrite, delete and z-ordering #1897

Merged
merged 4 commits into from
Nov 22, 2023

Conversation

MrPowers
Copy link
Collaborator

Description

Adds docs on how to append, overwrite, delete rows, and Z Order Delta tables.

Will add much more detailed pages in the future. Just getting the high-level skeleton of the docs developed.

Copy link

ACTION NEEDED

delta-rs follows the Conventional Commits specification for release automation.

The PR title and description are used as the merge commit message. Please update your PR title and description to match the specification.


```python
df = pd.DataFrame({"num": [8, 9], "letter": ["dd", "ee"]})
dl.writer.write_deltalake("tmp/some-table", df, mode="append")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we showcase examples where you import it like as dl.

I think we generally import directly, right?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ion-elgreco - I updated this. Also going to create an issue to discuss this in more detail. Thank you!

You can easily write a DataFrame to a Delta table.

```python
import deltalake as dl
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you forgot to remove this import as well

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated.


```python
import deltalake as dl
from deltalake.writer import write_deltalake
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We expose write_deltalake also on the deltalake module directly. Also in other docs we import directly from there, maybe do it here as well for consistency

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I get this error: cannot import name 'DeltaLake' from 'deltalake'

You sure write_deltalake is exposed on deltalake?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had a typo. You're right. Updated.


This section explains how to Z Order a Delta table.

Z Ordering colocates similar data in the same files, which allows for more better file skipping and faster queries.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

more or better, right? not both

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed, thanks

@ion-elgreco ion-elgreco changed the title docs on append, overwrite, delete and z ordering docs: on append, overwrite, delete and z-ordering Nov 22, 2023
@ion-elgreco ion-elgreco enabled auto-merge (squash) November 22, 2023 20:19
@ion-elgreco ion-elgreco merged commit 31767bf into delta-io:main Nov 22, 2023
25 of 26 checks passed
ion-elgreco pushed a commit to ion-elgreco/delta-rs that referenced this pull request Nov 25, 2023
# Description

Adds docs on how to append, overwrite, delete rows, and Z Order Delta
tables.

Will add much more detailed pages in the future. Just getting the
high-level skeleton of the docs developed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants