Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add 'Extensions List' page to the documentation #12420

Merged
merged 1 commit into from
Sep 12, 2024

Conversation

alamb
Copy link
Contributor

@alamb alamb commented Sep 10, 2024

Which issue does this PR close?

Related to #12357

Rationale for this change

Inspired by @phillipleblanc and @jonahgao comments on #12357 let's try and make it easier to find other features such as table providers rather than necessarily trying to add them to the core of DataFusion itself.

What changes are included in this PR?

  1. Add "Extensions List" page with a table of projects
  2. Move datafusion-contrib content to the new page

Are these changes tested?

CI doc tests. I also rendered it locally:

Screenshot 2024-09-10 at 10 38 20 AM Screenshot 2024-09-10 at 10 35 35 AM

Are there any user-facing changes?

@alamb alamb added the documentation Improvements or additions to documentation label Sep 10, 2024
we'll gladly set up a new repository for your extension.

| Name | Type | Description |
| ---------------------------- | ----------------- | --------------------------------------------------------------------------------- |
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These were the most seemingly active projects that I could find in https://github.com/datafusion-contrib -- there may more that we should list here

Copy link
Member

@jonahgao jonahgao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me👍

Copy link
Contributor

@phillipleblanc phillipleblanc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Excellent. I think this + encouraging people to start new extensions in datafusion-contrib + examples of integrating with extensions (like in datafusion-tui) will be really helpful.

I also wonder if we need some criteria for "graduating" an extension to be an official Apache DataFusion sub-project (i.e. similar to ballista/python bindings)? I realize that doesn't really solve the "maintainers being stretched thin" problem since it just means now more repos to monitor. Maybe the criteria is "if there are a subset of DataFusion committers that vote to maintain it longterm" ? Not important to solve now though imo.

Copy link
Contributor

@ozankabak ozankabak left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great, thank you

Copy link
Contributor

@comphead comphead left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm thanks folks

@comphead comphead merged commit 533fbc4 into apache:main Sep 12, 2024
5 checks passed
@alamb alamb deleted the alamb/extensions branch September 12, 2024 16:05
@alamb
Copy link
Contributor Author

alamb commented Sep 12, 2024

I also wonder if we need some criteria for "graduating" an extension to be an official Apache DataFusion sub-project (i.e. similar to ballista/python bindings)? I realize that doesn't really solve the "maintainers being stretched thin" problem since it just means now more repos to monitor.

Maybe the criteria is "if there are a subset of DataFusion committers that vote to maintain it longterm" ?

Yes for sure -- @phillipleblanc we are actually doing this in term apache/datafusion-sqlparser-rs#1294 -- the procedural method is that we do a binding community vote.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants