One canvas per app

[markusbkk]

I thought I saw this somewhere before the huge change in documentation but is there a way to launch every application into its own canvas?

My use case is a DOM based web desktop where I'd like to be able to launch a Wayland application by spawning a "window"/surface that contains a canvas with only one Wayland application maximized to the borders of said DOM-based "window".

[Zubnix]

Short answer: It is possible, but currently not implemented like that for reasons.

Long answer: In the beginning Greenfield was implemented like that. Each app would have it's own canvas, but this had some drawbacks:

• Each canvas would have it's own WebGL context to paint the app pixels. Unfortunately a browser only has a limited set of WebGL contexts, so you have a limited amount of apps you can show (also you don't know what this limit is, the oldest webgl context would just stop working).
• When a canvas is resized, the entire content is cleared. This means that when you resize a window, the window would flash white because the new content is busy painting. To fix this, a second offscreen canvas can be used to paint and be used as a 'double' buffer that is swapped once done painting. The drawback of this approach is also that you now have even less applications (divided by 2 now) that you can show.
• For some reason this approach is a bit slower than using the single canvas approach.

So functionally is perfectly doable if you don't mind the technical constraints. I admit it would be very nice to have it closely integrate with the DOM.

[markusbkk]

Short answer: It is possible, but currently not implemented like that for reasons.

Long answer: In the beginning Greenfield was implemented like that. Each app would have it's own canvas, but this had some drawbacks:

• Each canvas would have it's own WebGL context to paint the app pixels. Unfortunately a browser only has a limited set of WebGL contexts, so you have a limited amount of apps you can show (also you don't know what this limit is, the oldest webgl context would just stop working).
• When a canvas is resized, the entire content is cleared. This means that when you resize a window, the window would flash white because the new content is busy painting. To fix this, a second offscreen canvas can be used to paint and be used as a 'double' buffer that is swapped once done painting. The drawback of this approach is also that you now have even less applications (divided by 2 now) that you can show.
• For some reason this approach is a bit slower than using the single canvas approach.

So functionally is perfectly doable if you don't mind the technical constraints. I admit it would be very nice to have it closely integrate with the DOM.

Awesome! Thanks for the short history on this feature.
Guess I'll look into a proof-of-concept/dig into the older code.

Would be a real boon to have it at least run inside my own desktop environment. Even with all the drawbacks.

[KuennekeS]

For what it's worth. The software I'm working with was originally using Plotly for some graph rendering and we needed to figure an alternative because we hit the same WebGL context issue. I've been passively watching Plotly to see if they ever solved it. They recently took the approach of using virtual-webgl to share a few contexts amongst all plots they have.

I don't know if you've seen that yet, but our use case would eventually need to tackle this. We are going towards a single Wayland application per compositor.

[Zubnix]

A possible solution for Greenfield is to do all WebGL calls in a single context. WebGL is used to do YUV+A->RGBA color conversion, which is stateless between renders and fast, so it could be shared for all apps. This can be done using an offscreen canvas that transfers the render result to the application window canvas that has a ImageBitmapRenderingContext. This way we only need a single WebGL context for all apps. The drawback is that all apps now share a single WebGL context to do color conversion, transferring the result although using zero-copy semantics, might also have some overhead. Overall I do still expect this to be fast enough to be more than acceptable.

[Zubnix]

@markusbkk @KuennekeS I'm curious how you would use this from a pure functional perspective (so I know how to potentially support this). Do you aim to integrate applications in an existing website layout? Should these applications know from each others existence for e.g. copy-paste? etc.

[markusbkk]

@markusbkk @KuennekeS I'm curious how you would use this from a pure functional perspective (so I know how to potentially support this). Do you aim to integrate applications in an existing website layout? Should these applications know from each others existence for e.g. copy-paste? etc.

In my specific case it would be part of an HTML5 based desktop environment.

Copy/Paste support is definitely something I'm interested in. Although if that somehow doesn't work by default, I'd gladly engineer my own Portals-like solution for it. Might be better anyways, from a sandboxing perspective.

[KuennekeS]

From my end. We are building a web analysis dashboard where a user can select objects to perform assessment on. The datasets are so large that they outweigh the resources available on a client machine, but we still need low latency interaction with plotted data. Which Greenfield looks perfect for.

So, Greenfield's usage will be dynamic spin up/down of backend applications that are at the moment only showing the visual display of data, no controls associated. Copy pasting between compositors is not expected. But we may hit a scenario where up to 64 visual displays are needed at once, and based on the analyst workflow we can't group plots/displays together yet.

Which this feels like an extreme edge case.

[Zubnix]

working on this, hope to have something working in the coming weeks. Each toplevel window of an app will be forced in fullscreen mode and assigned it's own dedicated workspace (html canvas). You'll then be free to manage these canvas workspaces in your own web environment.

[markusbkk]

working on this, hope to have something working in the coming weeks. Each toplevel window of an app will be forced in fullscreen mode and assigned it's own dedicated workspace (html canvas). You'll then be free to manage these canvas workspaces in your own web environment.

Curious. What will that mean for applications with separate windows for things like toolbars?

[Zubnix]

Curious. What will that mean for applications with separate windows for things like toolbars?

If the toolbar is created by the application as a 'top level window' without a parent, it will need it's own canvas.

I'm currently implementing a system where you must give a canvas creation callback that will fire when the compositor needs a canvas. You must then create the new canvas yourself and return it in the callback. I'm looking at providing canvas creation parameters as an argument to the callback, so you have more context about the canvas that needs to be created (what application is it for, what size does the app desire etc.)

[Zubnix]

I'll try to give this some more attention soon. (I've been mostly busy investigating a Linux WASM port and getting a WASM build of GTK4 to work fyi).