DS: Enable Jupyter widgets (aka ipywidgets)

[Babakjfard]

I use BeakerX widget in Jupyter a lot to explore my pandas data frames. It will be great if this feature can be added to vscode (and in general the support for Jupyter Widgets in vscode)

[Dessix]

This would be very helpful for one of my projects.

Do we know a rough outline of what needs completed in order to implement this?

[rchiodo]

I have a feeling we'll need something along these lines:
https://github.com/jupyter-widgets/ipywidgets/blob/master/docs/source/embedding.md

Beaker (at least plots) is using a custom jupyter widget mime type:
application/vnd.jupyter.widget-view+json

[rchiodo]

This is also likely part of the solution:
https://www.npmjs.com/package/@jupyter-widgets/controls

[rchiodo]

Yeah looking at the examples here:
https://github.com/jupyter-widgets/ipywidgets/blob/master/examples

We're going to have to refactor our jupyter communication to exist in our react control instead of the main extension.

[DonJayamanne]

Another solution is to try out the transform-widgets react package from nteract

@rchiodo /cc

[rchiodo]

See this issue for some more hints
jupyter-widgets/ipywidgets#2512

[DonJayamanne]

MVP

• Standard/builtin ipwidget (here's the list https://ipywidgets.readthedocs.io/en/latest/examples/Widget%20List.html)
• Interactive Widgets, with client side links (user interaction executes pure js code)
• Interactive Widgets, with kernel links (user interaction sends messages to kernels)
• Third-party widgets (list what works and what doesn't)
• Ensure widget state is maintained/updated even when cell output is hiddden/collapsed
• Moving cells around with output (widgets)
• Investigate why some third-party widgets don't work (reach out to ipywidgets/jupyterlabs folk)
• VS Code Remote (need to spike to find out what the issues are)

Long term goals

• Live share with interactive widgets
• Saving widgets in notebooks

Challenges/Unknowns

• Ensure widget state is updated/maintained even when not displayed (we have a spike for this)
  • Tehnically this should work out of the box, as backbone manages the state of these controls.
• Collapsing output cells (we'd need to show/hide ipywidgets, we have a spike for this)
• Saving state of widgets in notebooks (is this supported today in classic notebooks)
• Moving cells with ipywidget in output panels (should be possible, we'd need to re-render the widgets, we have a spike for this)
• Third-party widgets (some don't seem to work, no errors, nothing. Then again, these don't even work on other notebook implementation).
• VS Code Remote
• Live Share
  • Interactive widgets with js links will not work - i.e. moving a slider on host will not move slider on guest)
    • We could get this working by hooking into the state of each (backbone) control and broadcasting this and manually applying it. However, there's a complication of two users moving the same slider! Which one wins, gets even more complicated with links - impacting other controls
  • Interactive widgets with kernels links should technically work (we'll need to ensure all kernel messages are also broadcasted)

Solution

• The details found here will form the basis of the implementation https://ipywidgets.readthedocs.io/en/latest/embedding.html
  • Basically use ipywidget widget manager to do everything.
  • We create the manager, and give it the command chanel
  • Manager will render the widgets.
  • We can give the manager a map of request id to html elements. Will allow us to render the ipywidget in the right spot.
• Implement a communications channel on UI, so widgets can communicate directly with kernel (basically we'll need a proxy a connection to the actual one maintained within the extension process)
• Build and embed widgets as a separate JS project? (this project will only include the JS code for widgets, not the comms or the like). Basically it'll be similar to web3
• Integration with native editor (create a react control to render ipywidgets and just give it the requestid/commid). Attempting to give it the data, will be unnecessarily complicated, as we'll need to untangle the comms, etc. This is better handled by the ipywidget manager (based on the HTMLManager, see their docs)

Estimates (total 14.5)

• Clean up prototype (2)
• Support theming (0.5)
• Implement comms channel (2)
• Typing in widgets sets focus to monaco editor (bug) (1)
• Bundling jupyter ipywidgets (hopefully as a separate js project, as these require requirejs) (0)
• Integration with notebook (react integration) (0)
• Re-rendering widgets (show/hide/clear output cells or moving cells) (1)
• Add UI tests (2-3) (might not be possible without Spike - Serving notebook editor content via our own webserver #2825)
• Spike VS Code remote (1) (no idea whether it will even work, cannot estimate without knowing)
• Basic widgets (2)
• Third-party widgets (requirejs) (1/2)
• Ensure ensure widget state is updated/maintained even when not displayed (1/2)

** Updates to Estimates 7th Jan 2020 **

• Load 3rd party script files from jupyter server (if this fails, then fallback to download using internet connection). (2)
  • Currently 3rd party scripts are loaded from external resource (internet connection).
  • Downloading the local bits could be a preferred option (faster, no reliance on active internet connection, more accurate to what the user has installed).
• Add support for Output widget (requires special handling, seems jupyter notebooks and labs have special code to handle this particular widget) (2)
• Update old code base to latest codebase (updated React and other changes) (1)
  • The old prototype is based on the old code base

[stonebig]

test of today version on winpython:

• ipywidgets interactive doesn't seem to work
• latex rendering of latex expression doesn't seem to work

[rchiodo]

@stonebig not sure what you mean. We haven't submitted any code for ipywidgets support yet. So it's expected that ipywidgets don't work.

Can you go into more detail about latex expressions? Perhaps you're hitting this issue:
https://github.com/microsoft/vscode-python/issues/8673

[stonebig]

Yes probably same latex issue, with a $$ formula

[NellyWhads]

Hey folks, just wondering if this is still targeted for the February release?

[rchiodo]

No, sorry, it didn't make it in. Hopefully March.

[greazer]

Couldn't fit this in February release due to unexpected complications supporting Custom Editors. For March, it's still iffy.

[ablacklama]

Any updates on a planned release for this feature?

[rchiodo]

The current plan is our next release, pending some last minute issues we've found (like this one):
https://github.com/microsoft/vscode-python/issues/11239

[DonJayamanne]

This feature has been released hence closing this issue.

[idanpa]

Thanks, improved my work flow significantly!
Seems like this is still not supporting interactive matplotlib plots (see #3691)
Running this code in notebook:

%matplotlib notebook
import matplotlib.pyplot as plt
plt.plot([1,2,0])

yields:

ReferenceError: IPython is not defined
    at eval (eval at runCodeHere (vscode-resource://file///c%3A/Users/idank/.vscode/extensions/ms-python.python-2020.4.76186/out/datascience-ui/notebook/vendors~nteract_transforms.bundle.js:2:11583), <anonymous>:764:1)
    at runCodeHere (vscode-resource://file///c%3A/Users/idank/.vscode/extensions/ms-python.python-2020.4.76186/out/datascience-ui/notebook/vendors~nteract_transforms.bundle.js:2:11583)
    at t.value (vscode-resource://file///c%3A/Users/idank/.vscode/extensions/ms-python.python-2020.4.76186/out/datascience-ui/notebook/vendors~nteract_transforms.bundle.js:2:12015)
    at Io (vscode-resource://file///c%3A/Users/idank/.vscode/extensions/ms-python.python-2020.4.76186/out/datascience-ui/notebook/commons.initial.bundle.js:2:2210347)
    at Fo (vscode-resource://file///c%3A/Users/idank/.vscode/extensions/ms-python.python-2020.4.76186/out/datascience-ui/notebook/commons.initial.bundle.js:2:2213269)
    at vscode-resource://file///c%3A/Users/idank/.vscode/extensions/ms-python.python-2020.4.76186/out/datascience-ui/notebook/commons.initial.bundle.js:2:2226349
    at Object.t.unstable_runWithPriority (vscode-resource://file///c%3A/Users/idank/.vscode/extensions/ms-python.python-2020.4.76186/out/datascience-ui/notebook/commons.initial.bundle.js:2:25151)
    at Ms (vscode-resource://file///c%3A/Users/idank/.vscode/extensions/ms-python.python-2020.4.76186/out/datascience-ui/notebook/commons.initial.bundle.js:2:2226283)
    at Es (vscode-resource://file///c%3A/Users/idank/.vscode/extensions/ms-python.python-2020.4.76186/out/datascience-ui/notebook/commons.initial.bundle.js:2:2226059)
    at ws (vscode-resource://file///c%3A/Users/idank/.vscode/extensions/ms-python.python-2020.4.76186/out/datascience-ui/notebook/commons.initial.bundle.js:2:2225404)

[stonebig]

and like this ? (I guess 'widget' is what you shall use)

%matplotlib widget
import matplotlib.pyplot as plt
plt.plot([1,2,0])

[JustinStafford]

Thanks for the fast reply @rchiodo, appreciate it. I will give FigureWidget a go. This is what I get when I run the code in a Jupyter Notebook:

[JustinStafford]

Got it working. For anyone else struggling, this works in VSCode:

import plotly.graph_objects as go
import numpy as np
from ipywidgets import interact

fig = go.FigureWidget()
fig.add_scatter()

xs=np.linspace(0, 6, 100)

@interact(a=(1.0, 4.0, 0.01), b=(0, 10.0, 0.01), color=['red', 'green', 'blue'])
def update(a=3.6, b=4.3, color='blue'):
    with fig.batch_update():
        scatt = fig.data[0]
        scatt.x=xs
        scatt.y=np.sin(a*xs-b)
        scatt.line.color=color

fig

[rhaschke]

For me, ipywidgets don't work, even with the latest release 1.45. For example, the following simple code snippet:

import numpy as np
from ipywidgets import interact

@interact(a=(1.0, 4.0, 0.01), b=(0, 10.0, 0.01), color=['red', 'green', 'blue'])
def update(a=3.6, b=4.3, color='blue'):
    print(a, b, color)

doesn't show any widgets. The Jupyter log in VS code shows:
[IPKernelApp] WARNING | Widget Javascript not detected. It may not be installed or enabled properly.

However, running the very same code within the very same jupyter kernel, but in my web browser everything works just fine. As it is stated above that ipywidgets should work in VS code, I'm asking myself what I'm doing wrong.
I'm running Ubuntu 18.04 with standard python and VS code 1.45.0 installed from .deb.

[rchiodo]

@rhaschke What's your version of the python extension? That's the part that has the ipywidgets support.

[rhaschke]

The python extension states the version: 2020.5.78807

I just tried on another, freshly installed machine. There, VS code asked to install jupyter support, because it wasn't yet installed in the system. On this machine (Ubuntu 20.04), after installing all required packages via VS code in ~/.local, it works.
So my question now is: Which standard ubuntu packages should I install (system-wide) to get it working for all users on my main machine?

[rchiodo]

Sorry but I haven't the foggiest. If it works in jupyter it should work for us. We're using the same environment.

[rchiodo]

On the machine where it doesn't work, can you bring up the webview developer tools? (It's in the command palette).

The console there may shed some light on why we don't load the widgets.

Did you try other widgets too? Like this?

import ipywidgets as widgets
widgets.Text(value='Hello World!', disabled=True)

[jrmoulton]

If you have installed Jupyter in different environments try checking which Jupyter using !which jupyter in the notebook. I've noticed that if there are several versions of Jupyter on the system VSCode doesn't always use the one that is installed in that particular kernal.

[rhaschke]

Did you try other widgets too? Like this?

Same issue with widgets.Text. 😞 The webview developer tools' console shows the following:

Starting WebSocket: BOGUS_PVSC/api/kernels/b0b8b562-1208-4294-b56a-ab5555e5b076
commons.initial.bundle.js:2 Kernel: connected (b0b8b562-1208-4294-b56a-ab5555e5b076)
monaco.bundle.js:1 Could not create web worker(s). Falling back to loading web worker code in main thread, which might cause UI freezes. Please see https://github.com/Microsoft/monaco-editor#faq
we @ monaco.bundle.js:1
monaco.bundle.js:1 Failed to construct 'Worker': Script at 'vscode-resource://file///homes/rhaschke/tex/lehre/neuro1/notebooks/editor.worker.js' cannot be accessed from origin 'null'.
we @ monaco.bundle.js:1
2commons.initial.bundle.js:2 Widget data not avaialble to render an ipywidget
(anonymous) @ commons.initial.bundle.js:2

Edit: The first two warnings (from monaco.bundle.js) occur on the working machine as well.

!which jupyter yields: /usr/bin/jupyer, i.e. the correct system one.

[rchiodo]

This makes me feel like it might be jupyter not responding with the correct information:

Widget data not avaialble to render an ipywidget

What is the version of jupyter (and ipython) on both machines?

Edit:
From the command line you can print this out with jupyter --version

[rhaschke]

jupyter --version yields: 4.4.0 on the broken machine (Ubuntu 18.04 standard).
Edit: jupyter is reporting Adapting to protocol v5.1 in both cases, VS code and browser access.

On the (working) Ubuntu 20.04 machine, I have:

~/.local/bin/jupyter --version
jupyter core     : 4.6.3
jupyter-notebook : 6.0.3
qtconsole        : 4.7.4
ipython          : 7.13.0
ipykernel        : 5.2.1
jupyter client   : 6.1.3
jupyter lab      : not installed
nbconvert        : 5.6.1
ipywidgets       : 7.5.1
nbformat         : 5.0.6
traitlets        : 4.3.3

[rchiodo]

I would assume that's the problem then.

[rhaschke]

It's a bit disappointing that I cannot use the system-installed jupyter version, particularly because this works fine directly in the browser. I will try to install a newer jupyter version system-wide in /usr/local at least. Thanks for your help!

[rhaschke]

Using vscode-installed jupyter packages it works on the Ubuntu 18.04 machine as well.
On the Ubuntu 20.04 machine, removing vscode-installed jupyter packages, but installing the default system packages, it is broken there as well. juypter reports these versions:

jupyter --version
jupyter core     : 4.6.3
jupyter-notebook : 6.0.3
qtconsole        : not installed
ipython          : 7.13.0
ipykernel        : 5.2.0
jupyter client   : 6.1.2
jupyter lab      : not installed
nbconvert        : 5.6.1
ipywidgets       : 6.0.0
nbformat         : 5.0.4
traitlets        : 4.3.3

Except ipywidgets, all versions are identical up to patch version. So, indeed, looks like vscode is bound to a very recent ipywidgets version and cannot work with the system versions provided by default Ubuntu installs.

[rchiodo]

We should probably check that then. Put up a warning.

[jasongrout]

ipywidgets 7 came out August 2017, so 2.5 years ago. You'll definitely need ipywidgets at least 7.

[rhaschke]

Sorry, I just falsified my assumption: I installed ipywidgets 7.5.1 on top of the system one. This version is correctly reported by jupyter and vscode. Nevertheless, widgets are not working with this installation.

webview developer console reports:

Widget data not avaialble to render an ipywidget
(anonymous) @ commons.initial.bundle.js:2

[rhaschke]

ipywidgets 7 came out August 2017, so 2.5 years ago.

Ubuntu 20.04 (just released a few weeks ago) still ships ipywidgets 6.0.0. 😢

[rchiodo]

I'm not sure it's ipywidgets that's the culprit. It's more likely the kernel. Or some combination of the two.

But we'll have to figure out minimum versions for each.

[rhaschke]

Thanks for your help so far.

[nickcorona]

Progress bars don't work for me.

Result:
https://i.imgur.com/JhbieoW.png

Expected result:
https://i.imgur.com/iUCgTkE.png

[rchiodo]

@nickcorona can you open a separate issue? This issue is closed and is really about the general support for ipywidgets. If a specific widget doesn't work please open a new issue for that.

Thanks.

[LeeRippon]

Got it working. For anyone else struggling, this works in VSCode:

import plotly.graph_objects as go
import numpy as np
from ipywidgets import interact

fig = go.FigureWidget()
fig.add_scatter()

xs=np.linspace(0, 6, 100)

@interact(a=(1.0, 4.0, 0.01), b=(0, 10.0, 0.01), color=['red', 'green', 'blue'])
def update(a=3.6, b=4.3, color='blue'):
    with fig.batch_update():
        scatt = fig.data[0]
        scatt.x=xs
        scatt.y=np.sin(a*xs-b)
        scatt.line.color=color

fig

Does plot viewer work with it? I have my interactive plots working in my VScode Jupyter notebook but for some reason it won't let me open it into a separate screen (i.e., plot viewer). Thanks!

[rchiodo]

Sorry but widget based plots are not connected to our external viewer.

[JonesKenzie]

Hello, I can't use the widgets. I'm on a Windows 10, using Python 3.8.5 64-bits with the 2020.6.91350 Python Extension.

argon2-cffi==20.1.0
async-generator==1.10
attrs==20.3.0
backcall==0.2.0
bleach==3.3.0
cffi==1.14.5
colorama==0.4.4
cycler==0.10.0
decorator==5.0.0
defusedxml==0.7.1
entrypoints==0.3
ipykernel==5.5.3
ipython==7.22.0
ipython-genutils==0.2.0
ipywidgets==7.6.3
jedi==0.18.0
Jinja2==2.11.3
jsonschema==3.2.0
jupyter-client==6.1.12
jupyter-core==4.7.1
jupyterlab-pygments==0.1.2
jupyterlab-widgets==1.0.0
kiwisolver==1.3.1
MarkupSafe==1.1.1
matplotlib==3.4.1
mistune==0.8.4
nbclient==0.5.3
nbconvert==6.0.7
nbformat==5.1.3
nest-asyncio==1.5.1
notebook==6.3.0
numpy==1.20.2
packaging==20.9
pandas==1.2.3
pandocfilters==1.4.3
parso==0.8.2
pickleshare==0.7.5
Pillow==8.2.0
prometheus-client==0.9.0
prompt-toolkit==3.0.18
pycparser==2.20
Pygments==2.8.1
pyparsing==2.4.7
pyrsistent==0.17.3
python-dateutil==2.8.1
pytz==2021.1
pywin32==300
pywinpty==0.5.7
pyzmq==22.0.3
scipy==1.6.2
seaborn==0.11.1
Send2Trash==1.5.0
six==1.15.0
terminado==0.9.4
testpath==0.4.4
tornado==6.1
traitlets==5.0.5
wcwidth==0.2.5
webencodings==0.5.1
widgetsnbextension==3.5.1

jupyter core     : 4.7.1
jupyter-notebook : 6.3.0
qtconsole        : not installed
ipython          : 7.22.0
ipykernel        : 5.5.3
jupyter client   : 6.1.12
jupyter lab      : not installed
nbconvert        : 6.0.7
ipywidgets       : 7.6.3
nbformat         : 5.1.3
traitlets        : 5.0.5

[rchiodo]

@JonesKenzie can you enter a new issue? We'll need more logging to determine what the problem is.