Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clarify DataFusion similarities and differences with duckdb, pola.rs and other similar systems #5498

Closed
Tracked by #3058
alamb opened this issue Mar 7, 2023 · 0 comments · Fixed by #5578
Closed
Tracked by #3058
Assignees
Labels
documentation Improvements or additions to documentation

Comments

@alamb
Copy link
Contributor

alamb commented Mar 7, 2023

Please comment if you have any thoughts on these ideas:

I think it would be good to update the text here: https://github.com/apache/arrow-datafusion/blob/main/README.md#comparisons-with-other-projects

In terms of competition / optics of DuckDB vs DataFusion (vs Pola.rs) -- I think the best approach is to define the areas each is best at rather than try to "compete" head to head. I would be quite happy to have comparable performance with DuckDB (not faster) and pola.rs

Some thoughts on the benefits of DataFusion where it has clear differentiation:

  1. Target audience is different (developers rather than end users / data scientists)
  2. Designed to be embedded (rather than designed to be a file based sql engine)
  3. Community / ASF (rather than being tightly controlled in Amsterdam)
  4. Rust implementation (all the cool kids want Rust, I hear!)
@alamb alamb changed the title Clarify DataFusion similarities and differences with duckdb, pola.rs and other similar systems [DISCUSS] Clarify DataFusion similarities and differences with duckdb, pola.rs and other similar systems Mar 7, 2023
@alamb alamb changed the title [DISCUSS] Clarify DataFusion similarities and differences with duckdb, pola.rs and other similar systems Clarify DataFusion similarities and differences with duckdb, pola.rs and other similar systems Mar 7, 2023
@alamb alamb self-assigned this Mar 7, 2023
@alamb alamb added the documentation Improvements or additions to documentation label Mar 7, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant