Elyra Incorporation

[lresende]

We propose to make Elyra an official Jupyter subproject.

[fperez]

First, thanks a lot for proposing this, and for the amazing work the entire Elyra team has done! It's a great contribution to the growing ecosystem of Jupyter tools for various use cases, with impressive functionality.

Now - what follows is just my personal take, not in any way an "official" stance, and I'd be happy to hear other voices. I also want to caveat this with the fact that I don't personally use Elyra, so this is a high-level perspective, and I'd be happy to be corrected on any misconceptions.

That said - my impression is that pulling Elyra in as a core Jupyter project would be a stretch: Jupyter isn't an ML-specific tool (while many people do ML with it), and Elyra is a fairly complex codebase aimed at this particular use case. It takes advantage wonderfully of various aspects of Jupyter, but for example maintaining pieces of it would probably be a tough stretch for any "regular" Jupyter developer who was not part of the Elyra team.

Having said that, as a project, we absolutely want to be a great foundation for projects and communities like Elyra, regardless of how specialized their use cases may be!

So I would ask instead:

• Are there aspects of Elyra that make sense to fold back into existing Jupyter components? If so, are there any barriers to that happening we could lower/remove, or ways in which Jupyter could help the Elyra team better achieve their goals?
• Are there pieces of the Jupyter APIs, protocols, etc. where modifications could be made to help Elyra? If so, those can be discussed (either as narrow PRs or larger scope JEPs if needed).
• Are there other ways, beyond code, where Jupyter could help the Elyra team? For example, regarding community building, visibility, integration, etc - maybe we can find other ways to signal to the broader community how projects like Elyra are part of this "extended ecosystem," and highly valued and important, without assuming a direct maintenance burden?

I think all of the above, from software to community work, are fair game to improve the experience and processes for Elyra users and developers. But for the Jupyter teams, who are already stretched pretty thin, I think it would be a strategically risky stretch to absorb this maintenance and development commitment.

Again - I hope this does not come across in any way as negative towards Elyra or the team: the project is awesome, and the team has been a wonderful citizen of the Jupyter community. But we should find ways to continue building on that track record without necessarily taking on potentially unsustainable responsibilities both towards Elyra and the rest of Jupyter.

[davidbrochart]

I just wanted to mention that there is kind of a similar situation with IPython and IPython Parallel: while the former is a general tool that can be used to do parallel computing, the latter does it by taking advantage of IPython's architecture (more precisely the kernel protocol, if I understand correctly). IPython Parallel started before Dask, which shares the same goal, so there might be historical reasons, but still, ipyparallel is part of the ipython project and Dask is not.

[westurner]

• https://numfocus.org/community/mission :

The mission of NumFOCUS is to promote open practices in research, data, and scientific computing by serving as a fiscal sponsor for open source projects and organizing community-driven educational programs.

NumFOCUS is a 501(c)(3) public charity in the United States.

• https://lfaidata.foundation/ :

The mission of LF AI & Data is to build and support an open artificial intelligence (AI) and data community, and drive open source innovation in the AI and data domains by enabling collaboration and the creation of new opportunities for all the members of the community

• Linux Foundation AI & Data landscape:
  https://landscapeapp.cncf.io/lfai/?grouping=category

• https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/customizing-your-repository/displaying-a-sponsor-button-in-your-repository

  *You can configure your sponsor button by editing a FUNDING.yml file in your repository's .github folder, on the default branch. You can configure the button to include sponsored developers in GitHub Sponsors, external funding platforms, or a custom funding URL"

• https://github.com/jupyter/jupyter/wiki#table-of-contents

  curated collection of Jupyter/IPython notebooks

• https://executablebooks.org/en/latest/gallery.html

  Gallery of Jupyter Books

[lresende]

Thank you. Elyra jas joined the Linux Foundation for open governance.