Make widgets render using output areas in classic notebook.

[jasongrout]

This is work towards fixing #1272, and makes the rendering consistent between jupyterlab and the classic notebook.

Essentially, this gets rid of the special widget area above cells, and instead renders widgets using the widget display_data output messages.

This also finishes fixing #1237.

[gnestor]

@jasongrout Let me know if you have any questions about this stuff 👌

[jasongrout]

Thanks! I'm working through it right now. You may notice that I'm using a slightly different approach (I think it's a bit cleaner) to get the output area, etc.

[jasongrout]

(for example, I'm directly requiring the output area, and using the output area prototype to register the mime type, rather than searching for an instance among the cells.)

[gnestor]

@jasongrout Yep, this is cleaner. Before we unwebpackified the notebook, doing this didn't work. I will update the mimerender extensions to reflect this 👍

[jasongrout]

Yeah, there are other kludgy workarounds I can now delete because we don't have webpack too, like https://github.com/jupyter-widgets/ipywidgets/blob/master/widgetsnbextension/src/widget_output.js#L93

[jasongrout]

CC @SylvainCorlay, @maartenbreddels - it would be great if someone looked at/tested this out too, to see what I might have missed.

[maartenbreddels]

Any idea of the issue raised here (about Output not syncing is related to, or fixed by this) #1232

[jasongrout]

Any idea of the issue raised here (about Output not syncing is related to, or fixed by this) #1232

Oh, interesting. This might break #1232 - we'll probably need to do something like @gnestor does and hook into the clearing signals to remove views.

[jasongrout]

Note to self: @gnestor's clearing function is at https://github.com/jupyterlab/jupyterlab_geojson/blob/master/nbextension/src/renderer.js#L21, triggered at https://github.com/jupyterlab/jupyterlab_geojson/blob/master/nbextension/src/renderer.js#L71.

@gnestor - why did you also handle the output_added.OutputArea event?
https://github.com/jupyterlab/jupyterlab_geojson/blob/master/nbextension/src/renderer.js#L75

[jasongrout]

Do we have a test for the _view_count that would have failed here?

[maartenbreddels]

Do you mean a unittest? Or sth automated? Or you mean just an example to demonstrate it?

[maartenbreddels]

Btw, for ipyvolume, I've started doing testing in the notebook, as an integration test. Do you have experience or know someone who knows how to set up a full blown test with headless browser etc? Would be nice to automate integration tests.

[jasongrout]

CC @blink1073 - I think JLab probably currently has the nicest infrastructure for (Javascript) testing currently.

At various times, we've had headless browser testing for widgets using CasperJS. We don't have a good story for widgets right now.

[jasongrout]

This almost calls the .remove() when appropriate (though it's totally kludgy). Ideally, we should have the output area itself sending a signal just before it empties itself that we could latch on to.

The only cases I've seen where something messes up now is something like this:

from ipywidgets import *
out = Output()
x = IntSlider()
out

with out:
    display(x)
    display(x)

x._view_count

returns 1 when it should return 2, and then evaluating the last two cells again we get _view_count as the correct 4. However, clearing (out.clear_output) does not correctly decrement the view count because it's only listening for changes at the cell level, not the output area level.

[jasongrout]

Okay, the last commit takes care of clearing an output area. However, it doesn't address the problem of the first _view_count returning 1. Not sure what's happening there yet.

[jasongrout]

@maartenbreddels - okay, _view_count looks like it is updating correctly now. It required a few fixes to the recent syncing, so I made those PRs first, and then rebased this on the current master.

I think this is ready to be reviewed.

[jasongrout]

Feedback from @maartenbreddels:

1. output widgets don't seem to be incrementing _view_count.

2. Executing a blank cell to get rid of a widget doesn't seem to decrement _view_count

[maartenbreddels]

I don't have time to look into it now, but for the record, it seems like if a list of buffers is send, they are all the same.

[maartenbreddels]

Ok, sorry for the noise, that was a bug in ipyvolume.

[jasongrout]

@maartenbreddels - the output widget view counts should be fixed now, and if you run jupyter/notebook#2411, the view count should now be updated even if an empty cell is run.