Support Perspective Backend

analysis/README.md

@@ -0,0 +1,99 @@
+# Perspective Analysis
+
+## The Problem
+
+Panel has its own [Panel Perspective Widget](https://panel.holoviz.org/reference/index.html).
+
+This widget wraps the `perspective-viewer` web component. Its does not import `perspective` or use other features of Perspective as described in the [Perspective Python User Guide](https://perspective.finos.org/docs/python/) or provided by the Jupyter Perspective widget.
+
+I.e. the Panel Perspective Widget does not scale in the same way as the Juyter Perspective Widget or as a *best practice* solution based on a Tornado or FastAPI backend.
+
+## How it works today
+
+See [app_as_is.py](app_is_is.py).
+
+```bash
+panel serve analysis/app_as_is.py --dev
+```
+
+You can check out the typescript implementation [here](https://github.com/holoviz/panel/blob/main/panel/models/perspective.ts).
+
+## How it could work in the future
+
+### Use Externally Hosted Tables
+
+We could ask the user to setup and deploy an externally hosted Perspective server and then in the Panel app the user can just do
+
+```python
+from panel.widgets import Perspective
+
+pn.extension("perspective")
+
+Perspective(
+        value="name-of-table",
+        websocket="wss://some-path/websocket",
+        height=500, sizing_mode="stretch_width"
+).servable()
+```
+
+This solution might perform and scale the best. But developing and managing the external server can also be very difficult for less technical users. And its not easily possible to dynamically update the perspective tables from the Panel apps - for example based on user input.
+
+---
+
+There is a POC of this concept in [app_external_server.py](app_external_server.py).
+
+You can start the Perspective server with
+
+```bash
+python analysis/app_external_server.py
+```
+
+and the Panel server with
+
+```bash
+panel serve analysis/app_external_server.py --dev
+```
+
+As a minimum lots of documentation and examples is needed. While developing this solution I found the Perspective documentation is not updated to work with the latest version of Perspective this would scare most users of. I had to navigate the internal implementation of Perspective to create a working example.
+
+### Use Internally Hosted Tables
+
+We could integrate the Perspective server with the Panel/ Bokeh/ Tornado server. This would make things a lot easier for datascience users at the cost of for example scaling. And it would enable datascience developers to dynamically update the tables managed by perspective for example based on user interactions.
+
+But we would probably have to have some if statements to figure out if Panel is running in Tornado, FastAPI, Pyodide or PY.CAFE environment and support that environment.
+
+---
+
+There is a POC of this concept in [app_internal_server.py](app_internal_server.py).
+
+You can start the server with
+
+```bash
+python analysis/app_internal_server.py
+```
+
+We can later continue work on this solution to enable users to start this with `panel serve ...` which is the normal way for users to start the panel server.
+
+We could maybe add a flag `--perspective` to signal to the panel server that it should add the perspective websocket end point and then enable the existing Perspective widget to take a table name as input. Or maybe it could be even simpler that the first time a named table is given as an argument to the Perspective widget, the websocket endpoint is started.
+
+### Use Existing Communication Channels
+
+Here we use the communication built into the [AnyWidget AFM Specification](https://anywidget.dev/en/afm/):
+
+```typescript
+/**
+ * Send a custom message to the backend
+ * @param content The content of the message
+ * @param callbacks Optional callbacks for the message
+ * @param buffers Optional binary buffers to send with the message
+ */
+send(content: any, callbacks?: any, buffers?: ArrayBuffer[] | ArrayBufferView[]): void;
+```
+
+instead of the `perspective.websocket` connection.
+
+This could simplify things further as we don't need to add an endpoint. If it performs and scales well would have to be tested.
+
+---
+
+POC coming up ...

analysis/app_as_is.py

@@ -0,0 +1,23 @@
+import random
+
+from datetime import datetime, timedelta
+
+import numpy as np
+import pandas as pd
+
+import panel as pn
+
+pn.extension('perspective')
+
+data = {
+    'int': [random.randint(-10, 10) for _ in range(9)],
+    'float': [random.uniform(-10, 10) for _ in range(9)],
+    'date': [(datetime.now() + timedelta(days=i)).date() for i in range(9)],
+    'datetime': [(datetime.now() + timedelta(hours=i)) for i in range(9)],
+    'category': ['Category A', 'Category B', 'Category C', 'Category A', 'Category B',
+             'Category C', 'Category A', 'Category B', 'Category C',],
+    'link': ['https://panel.holoviz.org/', 'https://discourse.holoviz.org/', 'https://github.com/holoviz/panel']*3,
+}
+df = pd.DataFrame(data)
+
+pn.pane.Perspective(df, width=1000).servable()

analysis/app_external_server.py

@@ -0,0 +1,60 @@
+def start_perspective_tornado_server():
+    import random
+
+    from datetime import datetime, timedelta
+
+    import pandas as pd
+    import perspective as psp
+    import tornado.ioloop
+    import tornado.web
+
+    from perspective.handlers.tornado import PerspectiveTornadoHandler
+
+    data = {
+        'int': [random.randint(-10, 10) for _ in range(9)],
+        'float': [random.uniform(-10, 10) for _ in range(9)],
+        'date': [(datetime.now() + timedelta(days=i)).date() for i in range(9)],
+        'datetime': [(datetime.now() + timedelta(hours=i)) for i in range(9)],
+        'category': ['Category A', 'Category B', 'Category C', 'Category A', 'Category B',
+                'Category C', 'Category A', 'Category B', 'Category C',],
+        'link': ['https://panel.holoviz.org/', 'https://discourse.holoviz.org/', 'https://github.com/holoviz/panel']*3,
+    }
+    df = pd.DataFrame(data)
+
+    server = psp.Server()
+    client = server.new_local_client()
+    client.table(df, name="data_source_one")
+    app = tornado.web.Application([
+        (r"/websocket", PerspectiveTornadoHandler, {
+            "perspective_server": server,
+        })
+    ])
+
+    app.listen(5005)
+    loop = tornado.ioloop.IOLoop.current()
+    loop.start()
+
+def start_panel_app():
+    import param
+
+    import panel as pn
+
+    class Perspective(pn.custom.AnyWidgetComponent):
+        value = param.String()
+        websocket = param.String()
+
+        _esm = "perspective_anywidget.js"
+        _stylesheets = ["https://cdn.jsdelivr.net/npm/@finos/perspective-viewer/dist/css/themes.css"]
+
+    Perspective(
+        value="data_source_one",
+        websocket="wss://mnr-jupyterhub.de-prod.dk/mt-ai/user/masma/vscode/proxy/5005/websocket",
+        height=500, sizing_mode="stretch_width"
+    ).servable()
+
+
+if __name__=="__main__":
+    start_perspective_tornado_server()
+
+elif __name__.startswith("bokeh"):
+    start_panel_app()

analysis/app_internal_server.py

@@ -0,0 +1,67 @@
+def add_perspective_to_panel_server(panel_server):
+    from perspective import GLOBAL_SERVER
+    from perspective.handlers.tornado import PerspectiveTornadoHandler
+
+    panel_server._tornado.add_handlers(
+        ".*",
+        [
+        (r"/websocket", PerspectiveTornadoHandler, {
+            "perspective_server": GLOBAL_SERVER,
+        })
+    ])
+
+
+def app():
+    import random
+
+    from datetime import datetime, timedelta
+
+    import pandas as pd
+    import param
+
+    from perspective import GLOBAL_CLIENT
+
+    import panel as pn
+
+    @pn.cache
+    def add_table():
+        data = {
+            'int': [random.randint(-10, 10) for _ in range(9)],
+            'float': [random.uniform(-10, 10) for _ in range(9)],
+            'date': [(datetime.now() + timedelta(days=i)).date() for i in range(9)],
+            'datetime': [(datetime.now() + timedelta(hours=i)) for i in range(9)],
+            'category': ['Category A', 'Category B', 'Category C', 'Category A', 'Category B',
+                    'Category C', 'Category A', 'Category B', 'Category C',],
+            'link': ['https://panel.holoviz.org/', 'https://discourse.holoviz.org/', 'https://github.com/holoviz/panel']*3,
+        }
+        df = pd.DataFrame(data)
+
+        GLOBAL_CLIENT.table(df, name="data_source_one")
+
+    add_table()
+
+    class Perspective(pn.custom.AnyWidgetComponent):
+        value = param.String()
+        websocket = param.String()
+
+        _esm = "perspective_anywidget.js"
+        _stylesheets = ["https://cdn.jsdelivr.net/npm/@finos/perspective-viewer/dist/css/themes.css"]
+
+    return Perspective(
+        value="data_source_one",
+        # The websocket argument could be removed or hidden from the user
+        websocket="wss://mnr-jupyterhub.de-prod.dk/mt-ai/user/masma/vscode/proxy/5006/websocket",
+        height=500, sizing_mode="stretch_width"
+    ).servable()
+
+
+if __name__ == '__main__':
+    import panel as pn
+    pn.extension()
+
+    server = pn.serve(app, start=False, port=5006)
+
+    add_perspective_to_panel_server(panel_server=server)
+
+    server.start()
+    server.io_loop.start()

analysis/perspective_anywidget.js

@@ -0,0 +1,18 @@
+import "https://cdn.jsdelivr.net/npm/@finos/perspective-viewer/dist/cdn/perspective-viewer.js"
+import "https://cdn.jsdelivr.net/npm/@finos/perspective-viewer-datagrid/dist/cdn/perspective-viewer-datagrid.js"
+import "https://cdn.jsdelivr.net/npm/@finos/perspective-viewer-d3fc/dist/cdn/perspective-viewer-d3fc.js"
+import perspective from "https://cdn.jsdelivr.net/npm/@finos/perspective/dist/cdn/perspective.js";
+
+async function render({ model, el }) {
+    const elem = document.createElement('perspective-viewer');
+    elem.style.height = '100%';
+    elem.style.width = '100%';
+    el.appendChild(elem);
+
+    const websocket = await perspective.websocket(
+        model.get("websocket")
+    );
+    const server_table = await websocket.open_table(model.get("value"));
+    await elem.load(server_table);
+}
+export default { render };

panel/.eslintrc.js

@@ -9,7 +9,7 @@ module.exports = {
   },
   "plugins": ["@typescript-eslint", "@stylistic/eslint-plugin"],
   "extends": [],
-  "ignorePatterns": ["*/dist", "*/theme/**/*.js", ".eslintrc.js", "*/_templates/*.js", "*/template/**/*.js", "examples/*", "scripts/*", "doc/_static/*.js"],
+  "ignorePatterns": ["*/dist", "*/theme/**/*.js", ".eslintrc.js", "*/_templates/*.js", "*/template/**/*.js", "examples/*", "scripts/*", "doc/_static/*.js", "analysis/perspective_anywidget.js"],
   "rules": {
     "@typescript-eslint/ban-types": ["error", {
       "types": {