Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DOCS] Tutorial: FOTW data access #2384

Merged
merged 15 commits into from
Jul 15, 2024
Merged

Conversation

avriiil
Copy link
Contributor

@avriiil avriiil commented Jun 16, 2024

This adds the first FOTW tutorial on Data Access.

I haven't been able to figure out yet how to render the .ipynb as HTML. I've toyed around with index.rst in the new tutorials directory and the toctree but I must be doing something wrong. How does this work?

We should also decide where we want to store the toy data:

  • in the docs (probably not?)
  • just on Github and point folks there if they want to run the code
  • everything on S3 (not ideal because we want to show off local reads too)

TO DO
[x] decide where to store toy data
[x] render notebook as html
[x] store some simple delta data in daft-public-data bucket

For (1):

For (3):

  • delta table could be as simple as:
df = daft.from_pydict({
    "A": [1, 2, 3, 4],
    "B": [1.5, 2.5, 3.5, 4.5],
    "C": [True, True, False, False],
    "D": [None, None, None, None],
})

df.write_deltalake(...)

@github-actions github-actions bot added the documentation Improvements or additions to documentation label Jun 16, 2024
@avriiil avriiil marked this pull request as ready for review June 16, 2024 23:26
@avriiil
Copy link
Contributor Author

avriiil commented Jul 9, 2024

@jaychia -- this should be good to go now, lmk what you think!

Copy link
Contributor

@jaychia jaychia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is looking pretty good @avriiil!

Were you able to figure out the rendering issues? I also left some comments about the files in to-remove which I'm assuming we just forgot to-remove? 😛

@@ -0,0 +1,10 @@
Feature of the Week Tutorials
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this under to-remove? Are they still used?

@avriiil
Copy link
Contributor Author

avriiil commented Jul 11, 2024

Good catch @jaychia -- redundant files have been removed. Notebook renders well now:

image

@jaychia jaychia merged commit cdfe2b4 into Eventual-Inc:main Jul 15, 2024
41 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants