Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Switch to a lab-based app for the Voila frontend #846

Merged
merged 9 commits into from
Oct 27, 2022

Conversation

jtpio
Copy link
Member

@jtpio jtpio commented Feb 26, 2021

Switch to a lab-based app for the Voila frontend.

Binder

Closes #4.
Closes #238

Might fix #904

This replaces the current requirejs + nbextensions approach by a JupyterLab plugin based application.

Current Status

Basic rendering of widgets, still a couple of issues to fix:

image

TODO

Next steps

@jtpio
Copy link
Member Author

jtpio commented Feb 26, 2021

Support for third-party widgets built as a prebuilt extension for JupyterLab 3.0:

image

Comment on lines +81 to +79
shared: {
...data.dependencies
}
Copy link
Member Author

@jtpio jtpio Feb 26, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This might be revisited at some point, based on the discussions in jupyterlab/jupyterlab#9829

@jtpio
Copy link
Member Author

jtpio commented Mar 1, 2021

Testing with prebuilt mimerender extensions for JupyterLab 3.0, for example fasta:

image

@jtpio jtpio force-pushed the lab-based branch 2 times, most recently from e96e5be to 65b7185 Compare March 5, 2021 15:05
@bollwyvl
Copy link
Contributor

bollwyvl commented Mar 7, 2021

It's looking really promising 🎉 ! Some thoughts on things in the description:

  • Remove dependency on jupyterlab_server if it is not strictly needed

It seems like the localization, themes, and the (hopefully) upcoming licenses, would be worth the cost of the dependency. I also imagine workspaces being a fairly killer feature at some point, perhaps with a locked down DockPanel (e.g. hidden tabs if there aren't any, fixed aspect ratios). Settings would be... complicated.

  • Load JupyterLab extensions from the CDN? Requires upstream feature for dynamic extensions?

Whatever happens, a public CDN requirement (or having to run one's own, unless shipped as part of this tool, or otherwise easily installed and served from the same host) is going to be a hard pass for a lot of deployment scenarios...

@jtpio
Copy link
Member Author

jtpio commented Mar 9, 2021

Thanks!

It seems like the localization, themes, and the (hopefully) upcoming licenses, would be worth the cost of the dependency. I also imagine workspaces being a fairly killer feature at some point, perhaps with a locked down DockPanel (e.g. hidden tabs if there aren't any, fixed aspect ratios). Settings would be... complicated.

Probably it would make sense to depend on it, since it would make the Voila frontend a lot more like JupyterLab and easier to reuse extensions too. We just need to make sure it doesn't open the door to some misuses (by adding more endpoints).

For the settings the lab ThemeManager depends on it for now:

https://github.com/jupyterlab/jupyterlab/blob/03d1edc02feb46bf7b7755d5694d0be5f7457a7d/packages/apputils/src/thememanager.ts#L488-L491

It might be possible to make it optional, so the @jupyterlab/apputils-extension:themes plugin could be reused as is.
Apart from the themes, settings might not be needed and might not be useful for Voila dashboards anyway.

@bollwyvl
Copy link
Contributor

Re: endpoints: the addition of the handler (and adding the info to PageConfig, etc) is gated by whether the URL is set:
https://github.com/jupyterlab/jupyterlab_server/blob/master/jupyterlab_server/handlers.py#L177

This could probably be made even more explicit and regular... it would likely be an even better approach to break up that function into something with many smaller methods that can be overridden by the implementation. Made an issue!

@bollwyvl
Copy link
Contributor

Also, I generally try to respect the draft status, and really appreciate you sharing... but if there's anything I can do to help, please let me know!

@jtpio
Copy link
Member Author

jtpio commented Mar 17, 2021

Made an issue!

Thanks for opening this!

Also, I generally try to respect the draft status, and really appreciate you sharing... but if there's anything I can do to help, please let me know!

Yes the idea of opening the PR early is to get a clearer idea of potential issues, and also send upstream fixes early if needed. For now it would be great to try to go through the list of todo items in the top post, and reach parity with the existing frontend (with enable_labextensions=True but not necessarily loading from a CDN for now)

@jtpio
Copy link
Member Author

jtpio commented Mar 18, 2021

This could probably be made even more explicit and regular... it would likely be an even better approach to break up that function into something with many smaller methods that can be overridden by the implementation

Another thing we might want to revive soon is the switch to an ExtensionApp: #592

Which might help streamline the setup and reuse bits of other ExtensionApps. Although in that case the Voila ExtensionApp would be configured with load_other_extensions = True

packages/voila/src/manager.ts Outdated Show resolved Hide resolved
voila/handler.py Outdated
}

mathjax_config = self.settings.get("mathjax_config", "TeX-AMS_HTML-full,Safe")
# TODO Remove CDN usage.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to cast "as any" until this is merged and released: jupyter-widgets/ipywidgets#3625
.binder/postBuild Outdated Show resolved Hide resolved
@martinRenou
Copy link
Member

update galata references

@github-actions
Copy link
Contributor

Benchmark report

The execution time (in milliseconds) are grouped by test file, test type and browser.
For each case, the following values are computed: min <- [1st quartile - median - 3rd quartile] -> max.

Results table
Test file voila-tree-classic.ipynb voila-tree-light.ipynb voila-tree-dark.ipynb voila-tree-miami.ipynb basics.ipynb bqplot.ipynb dashboard.ipynb gridspecLayout.ipynb interactive.ipynb ipympl.ipynb ipyvolume.ipynb multiple_widgets.ipynb query-strings.ipynb reveal.ipynb
Render
chromium
actual 420 <- [514 - 671 - 984] -> 1912 67 <- [71 - 79 - 97] -> 137 64 <- [66 - 72 - 85] -> 113 63 <- [64 - 72 - 84] -> 117 2814 <- [2929 - 3043 - 3138] -> 4628 2803 <- [2823 - 2834 - 2883] -> 3140 3022 <- [3050 - 3064 - 3131] -> 3357 3148 <- [3198 - 3206 - 3398] -> 3502 2348 <- [2441 - 2486 - 2619] -> 2786 3856 <- [3961 - 4169 - 4608] -> 5657 8144 <- [8169 - 8172 - 8202] -> 8225 6565 <- [6575 - 6613 - 6635] -> 6959 1925 <- [2038 - 2160 - 2232] -> 2491 3097 <- [3105 - 3196 - 3308] -> 3424
expected 3379 <- [3442 - 3517 - 3701] -> 3876 2976 <- [3227 - 3321 - 3421] -> 3604 3608 <- [3623 - 3709 - 3793] -> 3825 4453 <- [4453 - 4523 - 4661] -> 4748 2559 <- [2655 - 2656 - 2660] -> 2674 3982 <- [4079 - 4213 - 4356] -> 4743 12183 <- [18509 - 19553 - 20811] -> 21515 15319 <- [15660 - 15796 - 15912] -> 16056 1517 <- [1920 - 1997 - 2103] -> 2113

❗ Test metadata have changed
--- /dev/fd/63	2022-10-27 10:12:24.936589904 +0000
+++ /dev/fd/62	2022-10-27 10:12:24.940589906 +0000
@@ -4,37 +4,37 @@
     "BENCHMARK_REFERENCE": "actual"
   },
   "browsers": {
-    "chromium": "97.0.4666.0"
+    "chromium": "94.0.4595.0"
   },
   "systemInformation": {
     "cpu": {
-      "brand": "Xeon® Platinum 8370C",
+      "brand": "Xeon® E5-2673 v3",
       "cache": {
-        "l1d": 98304,
+        "l1d": 65536,
         "l1i": 65536,
-        "l2": 2097152,
-        "l3": 50331648
+        "l2": 524288,
+        "l3": 31457280
       },
       "cores": 2,
       "family": "6",
-      "flags": "fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology cpuid pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch invpcid_single pti fsgsbase bmi1 hle avx2 smep bmi2 erms invpcid rtm avx512f avx512dq rdseed adx smap clflushopt avx512cd avx512bw avx512vl xsaveopt xsavec xsaves md_clear",
+      "flags": "fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology cpuid pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm abm invpcid_single pti fsgsbase bmi1 avx2 smep bmi2 erms invpcid xsaveopt md_clear",
       "governor": "",
       "manufacturer": "Intel®",
-      "model": "106",
+      "model": "63",
       "physicalCores": 2,
       "processors": 1,
       "revision": "",
       "socket": "",
-      "speed": 2.8,
+      "speed": 2.4,
       "speedMax": null,
       "speedMin": null,
-      "stepping": "6",
+      "stepping": "2",
       "vendor": "GenuineIntel",
       "virtualization": false,
       "voltage": ""
     },
     "mem": {
-      "total": 7281311744
+      "total": 7291699200
     },
     "osInfo": {
       "arch": "x64",
@@ -42,11 +42,11 @@
       "codename": "Focal Fossa",
       "codepage": "UTF-8",
       "distro": "Ubuntu",
-      "kernel": "5.15.0-1022-azure",
+      "kernel": "5.8.0-1040-azure",
       "logofile": "ubuntu",
       "platform": "linux",
-      "release": "20.04.5 LTS",
-      "serial": "9720c16c9bca4d94b48929079ff4f0c7",
+      "release": "20.04.3 LTS",
+      "serial": "cfc067bfcb844f35865e279a1b0e66c5",
       "servicepack": "",
       "uefi": false
     }

@github-actions
Copy link
Contributor

Benchmark report

The execution time (in milliseconds) are grouped by test file, test type and browser.
For each case, the following values are computed: min <- [1st quartile - median - 3rd quartile] -> max.

Results table
Test file voila-tree-classic.ipynb voila-tree-light.ipynb voila-tree-dark.ipynb voila-tree-miami.ipynb basics.ipynb bqplot.ipynb dashboard.ipynb gridspecLayout.ipynb interactive.ipynb ipympl.ipynb ipyvolume.ipynb multiple_widgets.ipynb query-strings.ipynb reveal.ipynb
Render
chromium
actual 133 <- [151 - 185 - 254] -> 453 70 <- [75 - 84 - 105] -> 159 66 <- [69 - 75 - 87] -> 115 65 <- [67 - 73 - 82] -> 96 2941 <- [3014 - 3167 - 3254] -> 4744 2738 <- [2829 - 2848 - 2911] -> 3174 3028 <- [3041 - 3046 - 3095] -> 3177 3223 <- [3260 - 3266 - 3379] -> 3590 2408 <- [2425 - 2540 - 2668] -> 2826 4055 <- [4198 - 4498 - 4854] -> 6218 8065 <- [8815 - 8889 - 9080] -> 9396 6904 <- [6947 - 7076 - 7088] -> 7277 1942 <- [1969 - 2057 - 2292] -> 2333 6745 <- [7677 - 9250 - 12447] -> 21550
expected 3379 <- [3442 - 3517 - 3701] -> 3876 2976 <- [3227 - 3321 - 3421] -> 3604 3608 <- [3623 - 3709 - 3793] -> 3825 4453 <- [4453 - 4523 - 4661] -> 4748 2559 <- [2655 - 2656 - 2660] -> 2674 3982 <- [4079 - 4213 - 4356] -> 4743 12183 <- [18509 - 19553 - 20811] -> 21515 15319 <- [15660 - 15796 - 15912] -> 16056 1517 <- [1920 - 1997 - 2103] -> 2113

❗ Test metadata have changed
--- /dev/fd/63	2022-10-27 10:19:51.538672454 +0000
+++ /dev/fd/62	2022-10-27 10:19:51.538672454 +0000
@@ -4,37 +4,37 @@
     "BENCHMARK_REFERENCE": "actual"
   },
   "browsers": {
-    "chromium": "97.0.4666.0"
+    "chromium": "94.0.4595.0"
   },
   "systemInformation": {
     "cpu": {
-      "brand": "Xeon® Platinum 8272CL",
+      "brand": "Xeon® E5-2673 v3",
       "cache": {
         "l1d": 65536,
         "l1i": 65536,
-        "l2": 2097152,
-        "l3": 36700160
+        "l2": 524288,
+        "l3": 31457280
       },
       "cores": 2,
       "family": "6",
-      "flags": "fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology cpuid pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch invpcid_single pti fsgsbase bmi1 hle avx2 smep bmi2 erms invpcid rtm avx512f avx512dq rdseed adx smap clflushopt avx512cd avx512bw avx512vl xsaveopt xsavec xsaves md_clear",
+      "flags": "fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology cpuid pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm abm invpcid_single pti fsgsbase bmi1 avx2 smep bmi2 erms invpcid xsaveopt md_clear",
       "governor": "",
       "manufacturer": "Intel®",
-      "model": "85",
+      "model": "63",
       "physicalCores": 2,
       "processors": 1,
       "revision": "",
       "socket": "",
-      "speed": 2.6,
+      "speed": 2.4,
       "speedMax": null,
       "speedMin": null,
-      "stepping": "7",
+      "stepping": "2",
       "vendor": "GenuineIntel",
       "virtualization": false,
       "voltage": ""
     },
     "mem": {
-      "total": 7281307648
+      "total": 7291699200
     },
     "osInfo": {
       "arch": "x64",
@@ -42,11 +42,11 @@
       "codename": "Focal Fossa",
       "codepage": "UTF-8",
       "distro": "Ubuntu",
-      "kernel": "5.15.0-1022-azure",
+      "kernel": "5.8.0-1040-azure",
       "logofile": "ubuntu",
       "platform": "linux",
-      "release": "20.04.5 LTS",
-      "serial": "9720c16c9bca4d94b48929079ff4f0c7",
+      "release": "20.04.3 LTS",
+      "serial": "cfc067bfcb844f35865e279a1b0e66c5",
       "servicepack": "",
       "uefi": false
     }

@martinRenou
Copy link
Member

I removed the ui-test with ipyvolume. We could probably reduce the number of ui-tests to the bare minimum. Checking that one custom widget extension works is already enough to check that Voila works properly with custom widgets, testing more can only lead to unreliable tests.

@github-actions
Copy link
Contributor

Benchmark report

The execution time (in milliseconds) are grouped by test file, test type and browser.
For each case, the following values are computed: min <- [1st quartile - median - 3rd quartile] -> max.

Results table
Test file voila-tree-classic.ipynb voila-tree-light.ipynb voila-tree-dark.ipynb voila-tree-miami.ipynb basics.ipynb bqplot.ipynb dashboard.ipynb gridspecLayout.ipynb interactive.ipynb ipympl.ipynb multiple_widgets.ipynb query-strings.ipynb reveal.ipynb
Render
chromium
actual 100 <- [111 - 128 - 162] -> 271 77 <- [81 - 88 - 101] -> 137 74 <- [80 - 89 - 106] -> 157 71 <- [76 - 86 - 106] -> 141 2948 <- [3013 - 3073 - 3142] -> 3936 2815 <- [2865 - 2866 - 2902] -> 2908 2912 <- [2941 - 3076 - 3084] -> 3275 3156 <- [3183 - 3258 - 3333] -> 3554 2232 <- [2310 - 2398 - 2537] -> 2718 3870 <- [3959 - 4101 - 4429] -> 5257 7252 <- [7295 - 7297 - 7314] -> 7373 1959 <- [1984 - 2037 - 2056] -> 2068 2862 <- [2911 - 2955 - 2966] -> 3000
expected 3379 <- [3442 - 3517 - 3701] -> 3876 2976 <- [3227 - 3321 - 3421] -> 3604 3608 <- [3623 - 3709 - 3793] -> 3825 4453 <- [4453 - 4523 - 4661] -> 4748 2559 <- [2655 - 2656 - 2660] -> 2674 3982 <- [4079 - 4213 - 4356] -> 4743 12183 <- [18509 - 19553 - 20811] -> 21515 15319 <- [15660 - 15796 - 15912] -> 16056 1517 <- [1920 - 1997 - 2103] -> 2113

❗ Test metadata have changed
--- /dev/fd/63	2022-10-27 13:44:55.095045738 +0000
+++ /dev/fd/62	2022-10-27 13:44:55.095045738 +0000
@@ -4,37 +4,37 @@
     "BENCHMARK_REFERENCE": "actual"
   },
   "browsers": {
-    "chromium": "97.0.4666.0"
+    "chromium": "94.0.4595.0"
   },
   "systemInformation": {
     "cpu": {
-      "brand": "Xeon® Platinum 8272CL",
+      "brand": "Xeon® E5-2673 v3",
       "cache": {
         "l1d": 65536,
         "l1i": 65536,
-        "l2": 2097152,
-        "l3": 36700160
+        "l2": 524288,
+        "l3": 31457280
       },
       "cores": 2,
       "family": "6",
-      "flags": "fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology cpuid pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch invpcid_single pti fsgsbase bmi1 hle avx2 smep bmi2 erms invpcid rtm avx512f avx512dq rdseed adx smap clflushopt avx512cd avx512bw avx512vl xsaveopt xsavec xsaves md_clear",
+      "flags": "fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology cpuid pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm abm invpcid_single pti fsgsbase bmi1 avx2 smep bmi2 erms invpcid xsaveopt md_clear",
       "governor": "",
       "manufacturer": "Intel®",
-      "model": "85",
+      "model": "63",
       "physicalCores": 2,
       "processors": 1,
       "revision": "",
       "socket": "",
-      "speed": 2.6,
+      "speed": 2.4,
       "speedMax": null,
       "speedMin": null,
-      "stepping": "7",
+      "stepping": "2",
       "vendor": "GenuineIntel",
       "virtualization": false,
       "voltage": ""
     },
     "mem": {
-      "total": 7281307648
+      "total": 7291699200
     },
     "osInfo": {
       "arch": "x64",
@@ -42,11 +42,11 @@
       "codename": "Focal Fossa",
       "codepage": "UTF-8",
       "distro": "Ubuntu",
-      "kernel": "5.15.0-1022-azure",
+      "kernel": "5.8.0-1040-azure",
       "logofile": "ubuntu",
       "platform": "linux",
-      "release": "20.04.5 LTS",
-      "serial": "9720c16c9bca4d94b48929079ff4f0c7",
+      "release": "20.04.3 LTS",
+      "serial": "cfc067bfcb844f35865e279a1b0e66c5",
       "servicepack": "",
       "uefi": false
     }

@jtpio
Copy link
Member Author

jtpio commented Oct 27, 2022

I removed the ui-test with ipyvolume.

That one has indeed been very unstable on that branch so fine to remove it 👍 Otherwise we could remove the reference snapshot and do a DOM test instead.

Copy link
Member

@martinRenou martinRenou left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit: :shipit: :shipit:

@trungleduc
Copy link
Member

Thanks to all for working on this awesome PR!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api-change enhancement New feature or request new
Projects
None yet
Development

Successfully merging this pull request may close these issues.

display(Markdown(f'![]({relpth})') - works in Jupyter Lab (3.0.14) but not Voila (0.2.9)
5 participants