Add e2e tests / be explicit about supported notebook environments

[manzt]

@keller-mark raises a good point in vitessce/vitessce-python#208 (comment).

Both anywidget and the Jupyter cookiecutter templates claim that widgets will be compatible with multiple notebook environments. However, neither the cookiecutters nor anywidget have automated end-to-end tests to ensure which check compatibility. Each test JS and Python code separately.

It will be a challenge, but I'd like to implemented automated e2e tests for anywidget in its "target" environments. First, we should be explicit about which environments to test. So far, I've identified:

• classic Jupyter Notebooks
• JupyterLab
• VS Code
• Google Colab
• nbconvert (HTML / reveal.js outputs)

An initial foreseen challenge is for the environments which rely on a CDN to load the widget code (VS Code, Google Colab, nbconvert). I'm not aware of how to "test" that widget code actually is loaded and runs in these prior to making a release to NPM because the code is loaded from a CDN like unpkg.com using the current version.

[manzt]

It would be awesome to have a caniuse for anywidget and these notebooks.

[keller-mark]

Just tested in Shiny for Python and the counter example seems to work out of the box!

https://github.com/keller-mark/anywidget-shiny-example/blob/main/app.py

[maartenbreddels]

I've added a pytest plugin in solara which can test with playwright against solara (fastest option, and no separate kernel process), but it can also test against Voila, Jupyter Notebook, Jupyter Lab and solara in a uniform way. https://github.com/widgetti/ipyreact/tree/master/tests/ui is an example of how that works.

It already saved me a few bad releases.

[maartenbreddels]

An update to this: We split off the package from solara and it is now called pytest-ipywidgets, which allows you to test with or without a browser (playwright).

It's documented on our solara website.

A basic example is:

import ipywidgets as widgets
import playwright.sync_api
from IPython.display import display

def test_widget_button_solara(solara_test, page_session: playwright.sync_api.Page):
    # The test code runs in the same process as solara-server (which runs in a separate thread)
    # Note: this test uses ipywidgets directly, not solara components.
    button = widgets.Button(description="Click Me!")

    def change_description(obj):
        button.description = "Tested event"

    button.on_click(change_description)
    display(button)
    page_session.locator("text=Click Me!").click()
    page_session.locator("text=Tested event").wait_for()