Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

7.0 Release Plan #6307

Closed
jtpio opened this issue Mar 11, 2022 · 48 comments
Closed

7.0 Release Plan #6307

jtpio opened this issue Mar 11, 2022 · 48 comments
Assignees
Milestone

Comments

@jtpio
Copy link
Member

jtpio commented Mar 11, 2022

This is an issue for 7.0 release planning. It will continue to be updated as we plan and develop Jupyter Notebook v7.

From the Notebook v7 Jupyter Enhancement Proposal:

The next major release of the Jupyter Notebook application, version 7, will be based on the JupyterLab codebase, but will provide an equivalent user experience to the current (version 6) application.

Notebook v7 Project Board

Issues and PRs related to Notebook v7 are tracked in this board: https://github.com/orgs/jupyterlab/projects/2/views/1

We also use the 7.0 milestone to add any issue and PR relevant to the 7.0 release: https://github.com/jupyter/notebook/milestone/45

Relevant Links

@jtpio jtpio added this to the 7.0 milestone Mar 11, 2022
@jtpio jtpio pinned this issue Mar 11, 2022
@jtpio jtpio moved this to In Progress Issues in Notebook v7 Mar 11, 2022
@gutow
Copy link
Contributor

gutow commented May 28, 2022

May 28, 2022 - interface testing from perspective of one of the users who would like to maintain parity with the old notebook interface.
First: this is looking really good from the user interface perspective. I am still trying to get set up to port extensions and will comment on that later/elsewhere, as appropriate.

  1. The interface recovers most of the simplicity of the classic notebook. It is not identical, but it is still relatively simple.
  2. It is odd to have it start up in collaborative mode. I am not sure that should be the default. Although, I am not sure it causes a problem.
  3. Automatic saving on by default seems a little strange to me. That said, fewer things are likely to be lost. I think problems depend on the rate of checkpointing. If somebody makes a mistake it may be necessary to role back quite a bit. With only intentional saves, things are not accidentally overwritten. However, most students are conditioned to expect automatic saving. It might be worth having a menu item for checkpoint rate and number of checkpoints to keep.
  4. "Close and Shutdown notebook" does not exit the browser tab. This is very confusing.
  5. I believe the notebook metadata specifies the kernel. If a kernel is specified the notebook should open with that kernel by default if the kernel is available.
  6. There is no way to access cell metadata for editing. It looks as if this is being addressed in Cell tools for notebook v7 #6419.
  7. It would also be nice to have the trusted/untrusted indicator back.

Things are looking good.

@jtpio
Copy link
Member Author

jtpio commented May 30, 2022

Many thanks @gutow for the feedback 👍

It is odd to have it start up in collaborative mode. I am not sure that should be the default. Although, I am not sure it causes a problem.

Probably you were trying using the Binder link in the README? It sets the collaborative flag by default so it's easier to test RTC on Binder:

"JupyterNotebookApp": { "collaborative": true, "expose_app_in_browser": true }

Automatic saving on by default seems a little strange to me.

This is also a side-effect of having RTC enabled, and should not be the case when starting without the collaborative flag.

You can try the latest v7 pre-release with the following gist on Binder which doesn't have RTC enabled: https://gist.github.com/jtpio/0f3c67a6a1767360df0e79b52d74a5fe

"Close and Shutdown notebook" does not exit the browser tab. This is very confusing.

Mind opening an issue? Or maybe this could be part of #6397.

It would also be nice to have the trusted/untrusted indicator back.

Yes, there was a previous attempt at adding it in the former RetroLab repo: jupyterlab/retrolab#279
Which could be ported over here.

@gutow
Copy link
Contributor

gutow commented Jun 4, 2022

Many thanks @gutow for the feedback +1

You're welcome. Sorry about the slow reply. I was traveling.

It is odd to have it start up in collaborative mode. I am not sure that should be the default. Although, I am not sure it causes a problem.

Probably you were trying using the Binder link in the README? It sets the collaborative flag by default so it's easier to test RTC on Binder:

Actually, I was working with a clone of the repository locally. I am hoping to get things to the stage where I can test porting some of my widgets to the JLab within this interface.

"Close and Shutdown notebook" does not exit the browser tab. This is very confusing.

Mind opening an issue? Or maybe this could be part of #6397.

I think this belongs as part of #6397 and will add to that.

It would also be nice to have the trusted/untrusted indicator back.

Yes, there was a previous attempt at adding it in the former RetroLab repo: jupyterlab/retrolab#279 Which could be ported over here.

Please port.

@frenzymadness
Copy link
Contributor

I'm interested in this from a different perspective. I maintain a lot of notebook/jupyter-related RPM packages in Fedora Linux. We had only Notebook packaged for a long time because Lab required too many dependencies and nobody was willing to invest time to package them. When I saw the effort to make Notebook 7 work on components from Lab, I started with packaging the long tree of dependencies and I'm almost done and Lab will soon be available in Fedora Linux as an RPM package.
Right now, we have Notebook version 6.4.12 and I'm thinking whether it makes sense to update it to 6.5.2. The problem is that Notebook 6.5.2 needs nbclassic package which we don't have yet and which is not needed for Notebook v 7. How long do you think it will take before the release of Notebook 7? Does it make sense to invest my time to packaging nbclassic and updating Notebook to 6.5.2 or better is to wait for Notebook 7?

@jtpio
Copy link
Member Author

jtpio commented Jan 20, 2023

Many thanks @frenzymadness for your work on packaging! And for raising the question here 👍

whether it makes sense to update it to 6.5.2

Normally the 6.5.x line is only for preparing the transition to 7. And potentially getting some critical security updates. There is more information in this blog post: https://blog.jupyter.org/jupyter-notebook-6-5-0-release-candidate-4c229c6dab55

Does it make sense to invest my time to packaging nbclassic

At first glance I would say the current list of dependencies for nbclassic should overlap quite a lot with the ones of the current Notebook 6 and JupyterLab 3. Looking at the list I am however wondering whether all of them are required.

How long do you think it will take before the release of Notebook 7?

The Notebook 7 release is conditioned by the JupyterLab 4 release, which is tracked in jupyterlab/jupyterlab#9647. In the last JupyterLab dev meeting, we discussed entering a feature freeze by the end of February 2023, so a beta would follow shortly after. Once a beta of JupyterLab 4 is available there will likely be a beta for Notebook 7 as well, and so on until the final release. Hopefully all of this will happen in the coming month although there is no clear date yet.

@frenzymadness
Copy link
Contributor

Thank you for the quick and useful answer. Fedora 38 (currently in development) has its beta freeze on the 21st of February so my current plan is to package nbclassic and update notebook to 6.5 before the beta freeze and update notebook to 7 in Fedora 39. Notebook is in Fedora for years so it should remain stable, and therefore small steps are better for it. I'd also like to add Lab (alpha/beta release) into Fedora 38/39 as soon as possible so users can test it and provide feedback.

@frenzymadness
Copy link
Contributor

The current status of Jupyter components in Fedora Linux is that Fedora 38 (to be released in April) has nbclassic and notebook 6.5. The development version 39 (release planned for October) just got lab 4 alpha34 and I'm preparing the update of notebook to 7 alpha 14. Adding an alpha release as a completely new package to Fedora is fine because nobody has any expectations and it cannot really break anything. The situation is quite different with the update of the notebook to the alpha release because notebook is in Fedora for many years. I can do that in the development version of Fedora and it will be good for the Jupyter project because it should bring some more attention to the changes happening now but I have to be sure that the notebook (and lab) reaches stable releases before Fedora 39 reaches beta freeze in August. Otherwise, people upgrading from 38 to 39 would go from stable 6.5 to unstable 7 and that won't be good.

Do you think it's doable? Do you have any final deadline?

@jasongrout
Copy link
Member

Thank you very much for updating these packages. We are really trying hard to release JupyterLab 4 before JupyterCon in May, and I think we are still on track to do so. (I'll let others speak to Notebook 7, though).

@jtpio
Copy link
Member Author

jtpio commented Mar 7, 2023

Notebook 7 will likely follow the release plan of JupyterLab 4.

With a first beta shortly (couple of days, or 1-2 weeks) after JupyterLab 4 beta. And a similar timeline for rc and final.

@Carreau
Copy link
Member

Carreau commented Mar 20, 2023

I just saw the "Read the migration plan to Notebook 7 ..." message. I thing there should be a way to "dissmiss for now" instead of "dismiss permanetly".

@jtpio
Copy link
Member Author

jtpio commented Mar 20, 2023

a way to "dissmiss for now"

You mean as a way to remind the user later?

cc @RRosio @echarles

@Carreau
Copy link
Member

Carreau commented Mar 20, 2023

Sure, even if it's remove until page refresh.
Say I'm teaching demoing a tutorial and I just want to reclaim the vertical space right now for the next hour.

@kiyoon
Copy link

kiyoon commented Mar 24, 2023

Hi, in my neovim plugin, I used Notebook's front-end APIs to interact with Notebook through neovim. For example, I used Jupyter.notebook. ... to set cell text and execute cells etc.

I don't see any way to use the front-end API's in Notebook 7 or Jupyter Lab. I prefer to just interact directly through the browser javascript, but is the only option writing a Jupyter Lab extension?

If you're interested, kiyoon/jupynium.nvim#74 is the summary of issues I have with supporting Notebook 7. Some of my points may be wrong.

Any guidance with migrating the plugin to support Notebook 7 will be much appreciated!

@gutow
Copy link
Contributor

gutow commented Mar 25, 2023

@kiyoon I agree that it would make it easier to write user friendly extensions to Jupyter Lab if the javascript APIs were exposed. As I understand it (I did not work on the development of Jupyter Lab), most of this is a side-effect of efforts to make the code more modular and thus easier to maintain. I believe the developers then chose not to expose the javascript APIs directly to provide what I would call "security by effort level". It is still possible to inject invisible javascript that does bad things, but the hope is the effort level (coding overhead) to do so is high enough that attackers will go after other lower coding effort targets. Note this is just my understanding/interpretation. I was not involved in any of the discussions or development.

IMHO I believe this protection by effort level will decrease over time as more examples of plugins that do various things are developed. At some point there will be plugins with the correct boilerplate so that nefarious code can easily be built by repurposing an already developed plugin, with minor changes to the code. This will also provide easier ways for people to extend Jupyter lab to meet their needs.

At this point I suggest you look at the code for existing plugins. If you have not already, I would start here: https://jupyterlab.readthedocs.io/en/stable/extension/extension_dev.html.

@kiyoon
Copy link

kiyoon commented Mar 25, 2023

@gutow Thanks for the input. Since my plugin is to interact front end mostly, it is much better to use a front end API than making an extension by design. It's a pity they decided to deprecate this. It seems like in the official website it is stated that the classic notebook will be supported for 2 years, so I'll probably use this. But I honestly would have preferred if Notebook 7 never existed, because it unnecessarily break things when there's Jupyter Lab exists as an alternative anyway. Or, it would have been nice to keep the classic notebook package to notebook, and introduce new notebook as a new name (like no nbclassic but have notebook7, labnotebook etc.). The URL shouldn't have changed to localhost:8888/nbclassic because now plugins need to check which notebook version is running after loading the page, and reconnect.

For example, if you only have nbclassic installed it will be the normal URL localhost:8888 but then if you have both notebook and nbclassic installed it will be localhost:8888/nbclassic. This kind of inconsistency shouldn't have existed in my opinion.

@jasongrout
Copy link
Member

For example, I used Jupyter.notebook. ... to set cell text and execute cells etc.

Do you have a list of what you specific APIs you need? Like

  • Set the cell text to a specific value
  • Execute a given cell
  • what else?....

It would be really great to develop more examples in https://github.com/jupyterlab/extension-examples showing how to easily do these things.

@jasongrout
Copy link
Member

I believe the developers then chose not to expose the javascript APIs directly to provide what I would call "security by effort level".

As a clarification: the extension system is more about making components modular, reusable, and easy to package/distribute/remix, with different components playing nicely with each other by providing and using each other's APIs. The Javascript APIs are mostly exposed (and of course as needs arise, more can be exposed), but the APIs are compartmentalized by component so those components can be remixed and reused independent of each other.

Security by effort level wasn't a design goal, largely because as you point out, it would just be an illusion anyway.

@kiyoon
Copy link

kiyoon commented Mar 25, 2023

Here is a list of things I used so far.

Jupyter.notebook.ncells()
Jupyter.notebook.cells_to_code([0])
Jupyter.notebook.cells_to_markdown([0])
Jupyter.notebook.get_cell(0).set_text(arguments[0])
Jupyter.notebook.get_cell(0).render()
Jupyter.notebook.get_cells()
Jupyter.notebook.insert_cell_below()
Jupyter.notebook.delete_cell(-1)
Jupyter.notebook.scroll_cell_percent(arguments[0], arguments[1], 0)
Jupyter.notebook.toJSON()    // this is to download notebook
Jupyter.notebook.scroll_manager.is_cell_visible(Jupyter.notebook.get_cell(arguments[0]))

Jupyter.notebook.kernel.name
Jupyter.kernelselector.kernelspecs
Jupyter.kernel_list.kernelspecs
Jupyter.kernelselector.set_kernel(arguments[0])
Jupyter.notebook.kernel.interrupt()
Jupyter.notebook.kernel.restart()
Jupyter.notebook.kernel.is_connected()
Jupyter.notebook.kernel.complete(arguments[0], arguments[1], completeCallback)
Jupyter.notebook.kernel.inspect(arguments[0], arguments[1], inspectCallback)

Jupyter.notebook.clear_output()
Jupyter.notebook.clear_cells_outputs(Jupyter.notebook.get_selected_cells_indices())
Jupyter.notebook.toggle_cells_outputs_scroll(Jupyter.notebook.get_selected_cells_indices())
Jupyter.notebook.execute_selected_cells()
Jupyter.notebook.save_checkpoint()
Jupyter.notebook.save_notebook()
Jupyter.notebook.scroll_manager.animation_speed = 0; Jupyter.notebook.scroll_manager.scroll_some(arguments[0])
Jupyter.notebook.rename(arguments[0])

Thanks for looking into this. My plugin basically enables vim to interact with Jupyter Notebook within vim through the front end APIs. So I need basically most of the controls, including kernel completion.

Also, my plugin doesn't install any notebook extensions. It just interacts with Notebook through Selenium browser javascript execution. Because the plugin is only about interacting with Notebook and not modifying any part of it, it works nicely if the front end APIs are exposed directly, through the browser.

@jasongrout
Copy link
Member

I made an extension that starts to explore equivalents to classic Notebook apis: https://github.com/jasongrout/nbactions/blob/main/src/index.ts

As for interacting with JupyterLab through Selenium (or these days, perhaps it is better to use one of the more modern packages for interacting with the browser?), perhaps there can be a connector extension that exposes an API to Selenium. Or maybe even better, there is a commands framework in JupyterLab for actions that is used for menu items, keyboard shortcuts, etc. Perhaps Selenium could have basically a single entry point to invoke a command in JupyterLab, which then would be able to do anything you can do in JupyterLab from a menu item or keyboard shortcut.

Our testing framework Galata can control JupyterLab in various ways too, so there is precedence for controlling JupyterLab from browser testing frameworks.

@kiyoon
Copy link

kiyoon commented Mar 25, 2023

@jasongrout Wow, thanks a lot for making this for me!

Regarding Selenium, I didn't know there are modern alternatives to Selenium. For a simple plugin like that Selenium met all requirements, but I may have been missing out on something. If I can guess the async functionality of my plugin can be improved for speed.
By the way I have zero experience in web development and javascript, which is why I found it hard to develop extensions.

So to enable this extension, I just pip install nbactions and it will expose the API's on Jupyter Lab and Notebook 7? I should check how it works soon.

Or maybe even better, there is a commands framework in JupyterLab for actions that is used for menu items, keyboard shortcuts, etc. Perhaps Selenium could have basically a single entry point to invoke a command in JupyterLab, which then would be able to do anything you can do in JupyterLab from a menu item or keyboard shortcut.

Any documentation on how to use the commands framework? If this solution doesn't require building an extension I'd maybe prefer this way. But I wouldn't want menus to pop up every time and the UI shouldn't glitch much. Keyboard shortcuts can also change and it's not going to be as smooth as the front end API.

Again I really appreciate the support!

@jtpio
Copy link
Member Author

jtpio commented May 23, 2023

Thanks @andrii-i. Yes doing a triage session sounds good. I won't be able to make it to the meeting this week but feel free to proceed. We can also do this asynchronously on the issues directly.

@andrii-i
Copy link
Contributor

During Notebook and JupyterLab meetings we triaged Notebook 7.0 milestone bugs and identified 4 release blockers. Please see details in the meeting notes: jupyter/notebook-team-compass#21 (comment)

@jtpio
Copy link
Member Author

jtpio commented May 25, 2023

Thanks @andrii-i!

@andrii-i
Copy link
Contributor

andrii-i commented Jun 7, 2023

During today's Notebook call we discussed release strategy for Notebook 7 and what should be done before the release:

  1. Regarding WCAG 2.1 accessibility issues listed in Accessibility Issues Needing Addressing for WCAG 2.1 compliance (As of Version 7.0.0a15) #6800, we should consider an issue as blocking if it prevents someone currently using Notebook 6 to migrate to Notebook 7. Either because there is a big regression, or because something is missing in Notebook 7.
  2. We should look at Notebook JEP 79 and what is promised for Notebook 7 release within it, in particular "Critical extensions" section, RISE: https://jupyter.org/enhancement-proposals/79-notebook-v7/notebook-v7.html
  3. There are still 3 open issues identified as "Release blockers": https://github.com/jupyter/notebook/issues?q=is%3Aissue+is%3Aopen+milestone%3A7.0+label%3A%22tag%3ARelease+Blocker%22. Looking for assignees for these issues

@andrii-i
Copy link
Contributor

andrii-i commented Jun 9, 2023

Critical extensions mentioned in JEP 79:
- nbgrader is work in progress: jupyter/nbgrader#1684
- RISE needs updating to JupyterLab 4: jupyterlab-contrib/rise#29
- jupytext is work in progress as a separate jupyterlab-jupytext extension: mwouts/jupyterlab-jupytext#6
- IPython parallel is done as a standalone package ipyparallel: https://ipyparallel.readthedocs.io/en/latest/#

@andrii-i
Copy link
Contributor

andrii-i commented Jun 9, 2023

There are 2 remaining blocking issues in the Notebook 7 milestone. I wanted to initiate a discussion about next steps and what specifically needs to be done to prepare for the release (other than resolving blocking issues).

From my perspective, we have achieved a decent level of consistency with JEP 79 and are ready for RC but we need an alignment as a team. If there are any issues that have not been sufficiently addressed and should be tackled before the release, please share.

@andrii-i
Copy link
Contributor

We've now resolved all the remaining blocking issues in the Notebook 7 milestone 🎉.

We're now preparing to release Notebook 7 Release Candidate 0 soon. If no critical issues arise after RC0, we plan to do a full release a week later.

@andrii-i
Copy link
Contributor

Update on Notebook 7 blogpost: jupyter/notebook-team-compass#24 (comment)

@fecet
Copy link

fecet commented Jun 17, 2023

In jupyter notebook v7, is there any way to list all kernels live in notebook? Say, I open 3 notebooks in jupyter notebook

../dir1/notebook1.ipynb
../dir1/notebook2.ipynb
../dir2/notebook3.ipynb

and I would like to know theire correspoding kernel connection files say

../dir1/notebook1.ipynb  python 3.8(env1)  kernel-0e32...json
../dir1/notebook2.ipynb  python 3.8(env1)  kernel-7b63...json
../dir2/notebook3.ipynb  python 3.10(env2)  kernel-53c3...json
...

like lkhphuc/jupyter-kernel.nvim#9.
I found it's diffuct to do so in notebook v6, is v7 provide any similar thing?
I'm a https://github.com/kiyoon/jupynium.nvim user and it seems that that need a lot effort to migrate to v7, as dicussion before in this issue. So I'm wondering if notebook v7 provide enough things for me to build a simple repl workflow.

@jtpio
Copy link
Member Author

jtpio commented Jun 19, 2023

FYI 7.0.0r0 is released 🎉

Please try it and report issues if you find any:

Thanks!

@gutow
Copy link
Contributor

gutow commented Jun 19, 2023

Non-blocking, but issue to consider:

  • Choosing File>Close and Shutdown Notebook closes and shuts down the notebook, but it does not close the tab/window as I would expect.

My preliminary testing hasn't shown any other UX issues or unexpected incompatibilities with existing modules I use. This is looking good.

Thanks for the work on this!

@jtpio
Copy link
Member Author

jtpio commented Jun 19, 2023

Thanks @gutow for testing and reporting 👍

Would you like to open an issue, so we can have a look during the RC period?

@andrii-i
Copy link
Contributor

andrii-i commented Jun 19, 2023

I found it's diffuct to do so in notebook v6, is v7 provide any similar thing?
I'm a https://github.com/kiyoon/jupynium.nvim user and it seems that that need a lot effort to migrate to v7, as dicussion before in this issue. So I'm wondering if notebook v7 provide enough things for me to build a simple repl workflow.

@fecet thank you for bringing this up. Would you like to test if this feature is available in v7.0.0rc0 (can be installed with pip install --pre notebook)? If feature is missing, creating new issue / enhancement request to track and consolidate work around it would be a good next step

@ericsnekbytes
Copy link
Collaborator

In the Notebook weekly meeting, we discussed some updates to release planning:

  • Wait until @jtpio is back to release
  • Target July 19th as a tentative new Notebook 7 release date (release party during the weekly!)
  • Plan to communicate (discourse, etc) the impending release in multiple places to community, important extension authors (RISE, NBGrader)
  • We have an opportunity to work on some accessbility issues in the interim
  • Can make additional RC releases as well in the interim
  • Put blog post up after pip and conda packages are out

@kiyoon
Copy link

kiyoon commented Jun 24, 2023

@andrii-i For jupynium.nvim, all I need is a front-end Javascript API so I can interact with the notebook through browser scripts, without writing extensions. If this is supported, jupynium.nvim can be easily rewritten.

I created the issue #6949

@jtpio
Copy link
Member Author

jtpio commented Jul 18, 2023

For reference a first nbgrader pre-release with support for Notebook 7 has just been released (thanks @brichet!): https://github.com/jupyter/nbgrader/releases/tag/v0.9.0a0

@jtpio
Copy link
Member Author

jtpio commented Jul 19, 2023

Notebook 7 is released 🎉

Please report issues if you find any 🙏

We will continue working on addressing issues, improving documentation and working with the community to ease the transition.

Thanks all!

@jtpio jtpio closed this as completed Jul 19, 2023
@github-project-automation github-project-automation bot moved this from In Progress Issues to Done in Notebook v7 Jul 19, 2023
@jtpio jtpio unpinned this issue Oct 3, 2023
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Jul 19, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests