New nbextensions installation API #879

ellisonbg · 2015-12-17T06:34:37Z

This is working draft exploring the feasibility of the first part of the work described in #878

So far this does:

Adds an nbextensions config traitlet to the main NotebookApp.
Adds logic to the templates and handlers to json serialize that list, put it into the template and get it into a location that main.js could access it.

Carreau · 2015-12-17T08:45:45Z

notebook/notebookapp.py

@@ -187,6 +187,7 @@ def init_settings(self, ipython_app, kernel_manager, contents_manager,
            },
            version_hash=version_hash,
            ignore_minified_js=ipython_app.ignore_minified_js,
+            nbextensions=ipython_app.nbextensions,


We need to be smarter if we want this to be updated while the server runs.

Again, I want to keep it simple. I think that allowing updating this while the server runs is out of scope. But, there is nothing to prevent someone else from just calling utils.load_extensions() in the frontend at any time to load an extension. We could even provide an IPython magic that does that, or another extension could be written to handle runtime loading.

minrk · 2015-12-18T18:14:51Z

notebook/notebookapp.py

@@ -645,6 +647,12 @@ def nbextensions_path(self):
            path.append(os.path.join(get_ipython_dir(), 'nbextensions'))
        return path

+    nbextensions_tree = Dict(config=True,
+        help="A list of nbextensions to enable for the tree/dashboard page.")


s/list/dict(key:bool)/

ellisonbg · 2015-12-19T00:06:26Z

OK, the current state of this code allows you to install the following nbextensions:

https://github.com/jdfreder/jupyter-tree-filter
https://github.com/jdfreder/jupyter-lightsaber
https://github.com/jdfreder/jupyter-lightsaber-yoda

Using the following sequence of commands (notebook doesn't have to be restarted):

pip install jupyter-tree-filter
jupyter nbextension install --python jupyter-tree-filter
jupyter nbextension enable --python jupyter-tree-filter

I am going to work on cleaning it all up, but review can start.

parente · 2015-12-19T04:08:39Z

My diff comment got clobbered in an update, so reposting as a comment to make sure it's not lost: there's a third view, the edit view, that has a config file and supports extensions, just like notebook and tree. I think that view should be supported in the new approach as well. (We're using it in jupyter-incubator/contentmanagement for example).

It means adding one more if/else case in a few spots.

damianavila · 2015-12-22T20:31:58Z

notebook/services/config/manager.py

    """Config Manager used for storing notebook frontend config"""
-


Ref for #893 discussion

captainsafia · 2016-01-01T03:21:46Z

Hopefully, I'm not late to any reviews but I just pulled this PR locally and tested out what I could. I noticed that that prior to rewrite --python is used to reference an extension to be installed from a Python package. I actually this is much more usable and friendly than the --py currently in place. Some code related comments are in-line as notes.

Edit: Also not sure if now would be the right time to start writing some documentation on this, if so, let me know I can get started and we can build documentation/code in parallel.

Edit 2: I have some questions/comments about logging but will wait until you are done with that part of the PR.

captainsafia · 2016-01-01T03:22:06Z

notebook/base/handlers.py

@@ -292,7 +292,7 @@ def check_origin(self, origin_to_satisfy_tornado=""):
    #---------------------------------------------------------------
    # template rendering
    #---------------------------------------------------------------
-
+        


OCD kicking in, but might want to remove this whitespace.

bollwyvl · 2016-02-01T17:31:08Z

notebook/nbextensions.py

+
+def _read_config_data(user=False, sys_prefix=False):
+    config_dir = _get_config_dir(user=user, sys_prefix=sys_prefix)
+    config_file = os.path.join(config_dir, 'jupyter_notebook_config.json')


it would be nice if jupyter_notebook_config.json was exposed as a constant someplace: but hopefully nobody should ever have to deal with this again!

ConfigManager can be used to do this reading/writing, without any open/json/ensure_dir calls:

config_man = BaseJSONConfigManager(config_dir) config = config_man.get('jupyter_notebook_config') # < update config > config_man.update('jupyter_notebook_config', new_config)

minrk · 2016-02-02T20:56:54Z

notebook/nbextensions.py

-            prefix=self.prefix,
-            destination=self.destination,
-            nbextensions_dir=self.nbextensions_dir,
+        if self.python:


these duplicate blocks can be removed by making the function a variable:

install = install_nbextension_python if self.python else install_nbextension

parente · 2016-02-09T22:00:49Z

@ellisonbg I worked on installing this branch and trying to migrate the https://github.com/jupyter-incubator/contentmanagement extension to using this to install/enable instead of its own custom CLI (https://github.com/jupyter-incubator/contentmanagement/blob/master/jupyter_cms/extensionapp.py).

Here's what I found:

jupyter nbextension install --py jupyter_cms works as expected as written and with various additional params (--symlink, --user)
jupyter nbextension enable --py jupyter_cms missed the editor section I specified in _jupyter_nbextension_path
As expected, there was no command provided in the new jupyter nbextension * --py implementation for enabling server-side extensions which jupyter_cms has
When I try to use the existing CLI that works in 4.1, the jupyter cms activate command raises the exception below. So overriding the CLI for install/activate is not backward compatible at the moment.

[ExtensionActivateApp] Activating jupyter_cms notebook server extensions
Traceback (most recent call last):
  File "/Users/parente/miniconda3/envs/notebook-4.2-nbexts/bin/jupyter-cms", line 11, in <module>
    sys.exit(main())
  File "/Users/parente/miniconda3/envs/notebook-4.2-nbexts/lib/python3.5/site-packages/jupyter_cms/extensionapp.py", line 226, in main
    ExtensionApp.launch_instance()
  File "/Users/parente/miniconda3/envs/notebook-4.2-nbexts/lib/python3.5/site-packages/traitlets/config/application.py", line 589, in launch_instance
    app.start()
  File "/Users/parente/miniconda3/envs/notebook-4.2-nbexts/lib/python3.5/site-packages/jupyter_cms/extensionapp.py", line 223, in start
    super(ExtensionApp, self).start()
  File "/Users/parente/miniconda3/envs/notebook-4.2-nbexts/lib/python3.5/site-packages/traitlets/config/application.py", line 276, in start
    return self.subapp.start()
  File "/Users/parente/miniconda3/envs/notebook-4.2-nbexts/lib/python3.5/site-packages/jupyter_cms/extensionapp.py", line 109, in start
    self.enable_server_extension('jupyter_cms')
  File "/Users/parente/miniconda3/envs/notebook-4.2-nbexts/lib/python3.5/site-packages/jupyter_cms/extensionapp.py", line 92, in enable_server_extension
    server_cm.update('jupyter_notebook_config', cfg)
  File "/Users/parente/projects/jupyter/ellisonbg/notebook/services/config/manager.py", line 33, in update
    self._write_config_manager.update(section_name, new_data)
AttributeError: 'ConfigManager' object has no attribute '_write_config_manager'

To double-check, I tried nbgrader's CLI. It, too, raises an error because the base class nbextension implementation has changed.

[ExtensionActivateApp] Activating create_assignment nbextension
Traceback (most recent call last):
  File "/Users/parente/miniconda3/envs/notebook-4.2-nbexts/bin/nbgrader", line 11, in <module>
    sys.exit(main())
  File "/Users/parente/miniconda3/envs/notebook-4.2-nbexts/lib/python3.5/site-packages/nbgrader/apps/nbgraderapp.py", line 232, in main
    NbGraderApp.launch_instance()
  File "/Users/parente/miniconda3/envs/notebook-4.2-nbexts/lib/python3.5/site-packages/jupyter_core/application.py", line 267, in launch_instance
    return super(JupyterApp, cls).launch_instance(argv=argv, **kwargs)
  File "/Users/parente/miniconda3/envs/notebook-4.2-nbexts/lib/python3.5/site-packages/traitlets/config/application.py", line 589, in launch_instance
    app.start()
  File "/Users/parente/miniconda3/envs/notebook-4.2-nbexts/lib/python3.5/site-packages/nbgrader/apps/nbgraderapp.py", line 225, in start
    super(NbGraderApp, self).start()
  File "/Users/parente/miniconda3/envs/notebook-4.2-nbexts/lib/python3.5/site-packages/jupyter_core/application.py", line 256, in start
    self.subapp.start()
  File "/Users/parente/miniconda3/envs/notebook-4.2-nbexts/lib/python3.5/site-packages/nbgrader/apps/extensionapp.py", line 207, in start
    super(ExtensionApp, self).start()
  File "/Users/parente/miniconda3/envs/notebook-4.2-nbexts/lib/python3.5/site-packages/traitlets/config/application.py", line 276, in start
    return self.subapp.start()
  File "/Users/parente/miniconda3/envs/notebook-4.2-nbexts/lib/python3.5/site-packages/nbgrader/apps/extensionapp.py", line 99, in start
    self.enable_nbextension("create_assignment/main")
AttributeError: 'ExtensionActivateApp' object has no attribute 'enable_nbextension'

Current takeaways:

If server-side extensions are not going to also be supported, then I still have to provide a custom CLI for activating them.
If there's no way for this change to go in without breaking the current nbextension CLI overrides used in nbgrader, jupyter-incubator/declarativewidgets, jupyter-incubator/dashboards, etc.,
then I'll have to do some version detection to support install/activate on both <4.2 and >4.1.

Small update: Here's the metadata I put into jupyter_cms to test this.

def _jupyter_nbextension_paths():
    return [{
        'section': 'notebook',
        'src': 'nbextension',
        'dest': 'jupyter_cms',
        'require': 'jupyter_cms/notebook/main'
    },
    {
        'section': 'tree',
        'src': 'nbextension',
        'dest': 'jupyter_cms',
        'require': 'jupyter_cms/dashboard/main'
    },
   # below does not get picked up
    {
        'section': 'edit',
        'src': 'nbextension',
        'dest': 'jupyter_cms',
        'require': 'jupyter_cms/editor/main'
    }]

damianavila · 2016-02-10T12:41:23Z

If server-side extensions are not going to also be supported, then I still have to provide a custom CLI for activating them.

This is really important... otherwise people working with nbextension/serverextension complex extensions should implement an additional CLI as @parente mentioned or alternative a custom configmanager to write config outside the user space (since the current config manager implementation on this PR only writes to the user space)

What about adding the server extension info at the python metadata level? I mean adding a field to load that info if it is provided... In that way we should only modify enable_nbextension_python to be able to write the server-related config in NotebookApp.server_extensions

I mean:

def _jupyter_nbextension_paths():
    return [{
        'section': 'notebook',
        'src': 'amd',
        'dest': 'jupyter-lightsaber',
        'require': 'jupyter-lightsaber/index',
        'server': ['my_server_ext1', 'my_server_ext2']
    }]

and then make enable_nbextension_python able to understand this server key and write to the NotebookApp.server_extensions space in the diff being created.

If there's no way for this change to go in without breaking the current nbextension CLI overrides used in nbgrader, jupyter-incubator/declarativewidgets, jupyter-incubator/dashboards, etc., then I'll have to do some version detection to support install/activate on both <4.2 and >4.1.

Not sure if it will be a nice way to not break this since the config manager implementation has changed a lot on this PR... version detection seems like a good alternative.

damianavila · 2016-02-10T13:21:36Z

Additionally, the CLI should offer some keyword to enable/disable the server part... I guess... or maybe an independent CLI for the server part...

minrk · 2016-02-10T13:34:28Z

I would probably treat server and client extensions separately

damianavila · 2016-02-10T14:23:17Z

notebook/nbextensions.py

@@ -299,118 +355,300 @@ def _config_file_name_default(self):
    def install_extensions(self):
        if len(self.extra_args)>1:


~~this is triggering an error when I tried to install using --py in a sys-prefix... I mean:~~

~~removing this condition work as expected... if we want to warn more than one nbextension we should detect that in another way...~~

Forget about this... it is working...

bollwyvl · 2016-03-24T13:52:03Z

Ok, with that PR in... are we looking for anything else? More doc? Glad to help!

logging is inappropriate here also only show config dir if there's config in it.

enable is user by default

add more docs coverage, help output So that the three cases are covered: - system-wide (default for install) - user (default for enable) - sys-prefix

minrk · 2016-03-24T14:27:59Z

@bollwyvl I think we're almost there. We can merge soon and make the beta. I opened ellisonbg#24 with a few things I found that were still missing.

willingc · 2016-03-24T14:34:28Z

@bollwyvl If you think that there are more docs needed, please dive in or ping me. Thanks :D

updates to nbextensions

minrk · 2016-03-25T08:32:15Z

I'm happy to merge this as-is, now.

minrk · 2016-03-25T08:33:30Z

I'm about to board my flight to California, but I'll merge by the end of Friday (CA time) unless someone wants to do anything else here.

willingc · 2016-03-25T08:34:23Z

Safe travels @minrk.

jdfreder · 2016-03-25T16:28:00Z

@minrk safe travels

jdfreder · 2016-03-26T14:04:02Z

🎉 🍰 Welcome back to cali @minrk (and @takluyver) !

https://www.youtube.com/watch?v=E_B0rOLvAOo

:)

takluyver · 2016-03-26T14:19:03Z

Thanks Jon! See you on Monday (or let me know if you're getting to the bay area sooner. https://youtu.be/5NTqZ347TKY

willingc · 2016-03-26T18:05:49Z

Welcome back @takluyver. See you on Monday.

Congrats to all on getting this merged.

SylvainCorlay · 2016-03-27T14:07:47Z

🍰 👍

bollwyvl · 2016-03-27T16:33:45Z

🍦 🍰 🍕 from DFW... See y'all soon!

On 09:07, Sun, Mar 27, 2016 Sylvain Corlay [email protected] wrote:

[image: 🍰] [image: 👍]

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#879 (comment)

damianavila · 2016-03-27T16:40:41Z

\o/
nice...
See you all in a little bit!! 😉

ellisonbg mentioned this pull request Dec 17, 2015

Improving 4.x nbextensions #878

Closed

Carreau reviewed Dec 17, 2015
View reviewed changes

Carreau added this to the 4.2 milestone Dec 17, 2015

minrk reviewed Dec 18, 2015
View reviewed changes

damianavila reviewed Dec 22, 2015
View reviewed changes

notebook/services/config/manager.py

"""Config Manager used for storing notebook frontend config"""

Copy link

Member

damianavila Dec 22, 2015

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ref for #893 discussion

Carreau mentioned this pull request Dec 23, 2015

Remove configurability of config_dir in nbconfig #893

Closed

ellisonbg mentioned this pull request Dec 30, 2015

restore hardcoded config_dir #905

Merged

captainsafia reviewed Jan 1, 2016
View reviewed changes

ellisonbg mentioned this pull request Jan 6, 2016

[WIP] allow nbconvert template loading from other libraries jupyter/nbconvert#141

Merged

bollwyvl reviewed Feb 1, 2016
View reviewed changes

minrk mentioned this pull request Feb 2, 2016

Implement delayed evaluation for the cell executions happening before a kernel is set up #1002

Merged

minrk reviewed Feb 2, 2016
View reviewed changes

damianavila reviewed Feb 10, 2016
View reviewed changes

parente mentioned this pull request Mar 23, 2016

Unrecognized JSON config file version, assuming version 1 jupyter-incubator/contentmanagement#28

Closed

minrk added 3 commits March 24, 2016 15:03

use print for nbextension list output

d0e0ddf

logging is inappropriate here also only show config dir if there's config in it.

Remove incorrect mention of --user in docs

1287c86

enable is user by default

add --system flag for enable/disable

6371f8a

add more docs coverage, help output So that the three cases are covered: - system-wide (default for install) - user (default for enable) - sys-prefix

Merge pull request #24 from minrk/nbextensions

53ce313

updates to nbextensions

minrk changed the title ~~Adding prototype nbextensions argument to NotebookApp~~ New nbextensions installation API Mar 25, 2016

Carreau mentioned this pull request Mar 25, 2016

Readme mention $(ipython locate) captainsafia/notebook-toc#3

Closed

minrk merged commit 41d6da2 into jupyter:master Mar 26, 2016

minrk added a commit that referenced this pull request Mar 30, 2016

Backport PR #879: New nbextensions installation API

8151f2f

damianavila mentioned this pull request Apr 9, 2016

nbextension does not load from system-wide config #331

Closed

haobibo mentioned this pull request Apr 10, 2016

Folder ReOrg: Consider making each backend extension (in extensions folders) as seperate packages? ipython-contrib/jupyter_contrib_nbextensions#421

Closed

This was referenced Apr 12, 2016

server extensions have no simple way to use templates #1339

Open

make package more pip-friendly ipython-contrib/jupyter_contrib_nbextensions#577

Closed

jhamrick mentioned this pull request Apr 21, 2016

Destination not used when installing nbextensions #1384

Closed

yuvipanda pushed a commit to yuvipanda/notebook that referenced this pull request Jul 26, 2016

Backport PR jupyter#879: New nbextensions installation API

66149aa

github-actions bot added the status:resolved-locked label Apr 17, 2021

github-actions bot locked as resolved and limited conversation to collaborators Apr 17, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

New nbextensions installation API #879

New nbextensions installation API #879

ellisonbg commented Dec 17, 2015

Carreau Dec 17, 2015

ellisonbg Dec 17, 2015

minrk Dec 18, 2015

ellisonbg commented Dec 19, 2015

parente commented Dec 19, 2015

damianavila Dec 22, 2015

captainsafia commented Jan 1, 2016

captainsafia Jan 1, 2016

bollwyvl Feb 1, 2016

minrk Feb 2, 2016

minrk Feb 2, 2016

jdfreder Feb 24, 2016

parente commented Feb 9, 2016

damianavila commented Feb 10, 2016

damianavila commented Feb 10, 2016

minrk commented Feb 10, 2016

damianavila Feb 10, 2016

damianavila Feb 10, 2016

damianavila Feb 10, 2016

bollwyvl commented Mar 24, 2016

minrk commented Mar 24, 2016

willingc commented Mar 24, 2016

minrk commented Mar 25, 2016

minrk commented Mar 25, 2016

willingc commented Mar 25, 2016

jdfreder commented Mar 25, 2016

jdfreder commented Mar 26, 2016

takluyver commented Mar 26, 2016 via email

willingc commented Mar 26, 2016

SylvainCorlay commented Mar 27, 2016

bollwyvl commented Mar 27, 2016

damianavila commented Mar 27, 2016

		"""Config Manager used for storing notebook frontend config"""

		@@ -299,118 +355,300 @@ def _config_file_name_default(self):
		def install_extensions(self):
		if len(self.extra_args)>1:

New nbextensions installation API #879

New nbextensions installation API #879

Conversation

ellisonbg commented Dec 17, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ellisonbg commented Dec 19, 2015

parente commented Dec 19, 2015

Choose a reason for hiding this comment

captainsafia commented Jan 1, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

parente commented Feb 9, 2016

damianavila commented Feb 10, 2016

damianavila commented Feb 10, 2016

minrk commented Feb 10, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bollwyvl commented Mar 24, 2016

minrk commented Mar 24, 2016

willingc commented Mar 24, 2016

minrk commented Mar 25, 2016

minrk commented Mar 25, 2016

willingc commented Mar 25, 2016

jdfreder commented Mar 25, 2016

jdfreder commented Mar 26, 2016

takluyver commented Mar 26, 2016 via email

willingc commented Mar 26, 2016

SylvainCorlay commented Mar 27, 2016

bollwyvl commented Mar 27, 2016

damianavila commented Mar 27, 2016