-
Notifications
You must be signed in to change notification settings - Fork 406
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
docs: update docs home page and add pandas integration #1905
Conversation
ACTION NEEDED delta-rs follows the Conventional Commits specification for release automation. The PR title and description are used as the merge commit message. Please update your PR title and description to match the specification. |
320df81
to
fa51d5f
Compare
|
||
Time travel is a powerful feature that pandas users cannot access with CSV or Parquet. | ||
|
||
## Schema enforcement |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we perhaps want to have a short section on how you can overwrite the schema and the table at the same time?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@ion-elgreco - I actually just tried to do write_deltalake("tmp/some-table", df, mode="overwrite")
thinking that would overwrite the scheme and table at the same time and it still gave a ValueError
surprisingly. Is there another syntax? Should I also create an issue to add schema evolution?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You need to also pass overwrite_schema = True
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@ion-elgreco - thanks for clarifying, added a section. Let me know how it looks!!
docs/index.md
Outdated
## Why implement the Delta Lake transaction log protocol in Rust and Scala? | ||
|
||
Delta Spark depends on Java and Spark, which is fine for many use cases, but not all Delta Lake users want to depend on these libraries. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
should we also mention that this allows using delta in Rust or other native projects? in many of these cases using a JVM is not an option.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I added a sentence to clarify this. Good idea!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice!!
Description
Updated the docs home page to include more information about the project.
Also added a pandas integration page. Lots more integration pages to come soon hopefully!