Wasm support #759

elrnv · 2020-03-26T10:31:55Z

This is a continuation of #30 and includes all the changes made there + support for scrolling and keyboard input.

As mentioned in #30, there is an issue with std::time, which was solved with the instant crate, but I believe there are other options available as well.

A demo of the working widgets is available.

The limitations in this PR AFAICT are:

Loading images
Clipboard implementation
File dialog handling
Menu (this may require designing a custom menu widget)

This PR also depends on a pull request in piet #153, which makes returning the piet context easier.

Edit: Fixes #128

luleyleo · 2020-03-26T11:36:01Z

I've been playing around with the demo (in Firefox and Chromium) and everything seems to work, one exception being zoom, which seems to break things quite a bit.

When loading example with zoom 100%:

Zooming in makes content smaller
Zooming out makes content larger
Zooming in any way breaks cursor position

When loading example with zoom > 100%:

Example displays at normal (100%) size

When loading example with zoom < 100%:

White bar at the right (in the backround)
Druid can draw above it
Leaves artifacts behind when druid stops drawing over the white bar (flex example)

In both cases != 100%

Zooming in any direction / resizing the window makes it jump to the chosen zoom level (for example when loading at 200% an pressing - sets zoom to 170% and then it looks like when loaded at 100% and then zoomed to 170%

elrnv · 2020-03-27T00:40:29Z

Thanks @Finnerale! I believe I have resolve the issues you found with zooming. This relies on #158 being merged as well.

It seems the zooming issue was in part because the context wasn't scaled properly during the resize callback.

elrnv · 2020-03-27T01:09:01Z

I organized a small repo for building and hosting these examples on the web at https://github.com/elrnv/druid-wasm-examples/, hopefully it can serve as a reference on how one would go about building and deploying these examples to the web. It certainly took me some time to figure it out.

ijsnow · 2020-03-28T19:00:34Z

Woah! Amazing timing! I just discovered druid and was hoping for wasm support and here it is.

It appears another limitation of the current state is that it doesn't capture the browser's native functionality of navigating back when you press "backspace" when a text box is focused

elrnv · 2020-03-28T22:09:49Z

Oh thanks for finding that @ijsnow, in my setup, the browser doesn't respond to backspace so I never noticed this. I was able to reproduce this on another machine, and indeed it makes the text widgets basically unusable. I think this would be an unacceptable limitation, so I patched it by disabling the default backspace functionality altogether.

Perhaps this is too conservative and the default backspace behaviour should be enabled when the text widget is not focused, but for this PR I think this is acceptable since most likely the kinds of web applications that would require a GUI like druid would opt for a custom backspace behaviour anyway.

The druid-wasm-examples should be updated with the fix now. It would be great if you can verify if this issue is resolved there on your end :)

elrnv · 2020-03-30T22:47:56Z

@cmyr, @raphlinus: Is there anything else that needs to be done to get this through, or is it just not a priority right now?

cmyr · 2020-03-30T23:46:21Z

No, I'd like to get this merged while it's warm, just haven't built up the courage to give it a closer look. will do that soon!

cmyr

Okay I have some notes but this looks like a pretty good place to start, I'm happy to merge this as a checkpoint soon.

druid/src/contexts.rs

druid-shell/Cargo.toml

druid-shell/src/platform/web/window.rs

elrnv · 2020-04-04T00:00:06Z

All review comments should be addressed in 3813e3c. Thanks for the feedback!
I should mention that this PR depends on linebender/piet#153 and a few other piet PRs that have already been merged but aren't in the release yet.

elrnv · 2020-04-08T17:57:25Z

The last PR linebender/piet#153 was merged, so this PR is just blocked on having another piet release.

cmyr · 2020-04-12T15:23:01Z

@elrnv the piet changes have made it to druid.

A basic wasm backend is added as a continuation of linebender#30. Since std::time isn't available on wasm, an additional dependency is added to abstract std::time::Instant to work with wasm.

This commit ensures that all examples run with minimal changes. More specifically we don't need to remove the .use_simple_logger() call on all Apps.

The resize callback should be registered to the window, since resizing the window doesn't call the resize callback for canvases even if those do get resized.

The scroll example now works as expected on hi and low dpi screens.

This makes passing it easier to pass owned strings when creating a new KeyEvent, which is necessary when interfacing with WASM.

This should be the default behavior to make text widgets usable. At a later iteration it may be better to only disable the browser from going back when a widget is selected that is expecting keyboard input where backspace is intended for something else. I expect this functionality would require some way of accessing the web_sys event from within a druid callback.

The newly introduced lifetime parameter must be specified exmplicitly.

wasm targets cannot be run in the normal way.

The tests cannot be run in the normal way anyways. Leaving this for a future PR to flush out.

elrnv · 2020-04-12T22:12:18Z

@cmyr Thanks! The PR is ready now.

luleyleo

That's a lot of work here, thanks 🎉 !
I've left some comments and requests tho.

The biggest open issues seem to me testing and clippy (which behaves different for wasm?)
But those can be handled in future PRs.

druid-shell/src/platform/web/application.rs

druid-shell/src/platform/web/dialog.rs

druid-shell/src/platform/web/window.rs

luleyleo · 2020-04-13T10:05:47Z

druid-shell/src/platform/web/window.rs

+            state
+                .window
+                .set_timeout_with_callback_and_timeout_and_arguments_0(
+                    Closure::once_into_js(f).as_ref().unchecked_ref(),
+                    interval,
+                )
+                .expect("Failed to call setTimeout with a callback");


I haven't worked with wasm yet, but in other places with Closure there is allways a call to forget

let closure = Closure::wrap(Box::new(f) as Box<dyn FnMut(_)>); window_state .window .add_event_listener_with_callback(event_type, closure.as_ref().unchecked_ref()) .unwrap(); closure.forget();

Not sure if this would need one as well.

From what I understand this is true for Closures created with wrap or once, but once_into_js is special in that it does the forgetting for us. Quoting the wasm_bindgen docs for once_into_js:

Unlike Closure::once, this does not return a Closure that can be dropped before the function is invoked to deallocate the closure.

druid-shell/src/platform/web/window.rs

druid/examples/wasm/.gitignore

druid/examples/wasm/Cargo.toml

luleyleo · 2020-04-13T10:54:03Z

druid/examples/wasm/src/examples.rs

+//! This module is intentionally left blank.
+//! The contents are automatically generated in the "build.rs" script.


I would prefer if this file would not exist at all before building the wasm examples.
Because this file was added to git, the .gitignore entry for it will have no effect.

Ok I readjusted this mechanism to avoid any changes to any committed files in e072555. Would this work?

luleyleo · 2020-04-13T11:05:03Z

druid/src/lib.rs

+#[cfg(not(target_arch = "wasm32"))]
 #[cfg(test)]
 mod tests;


Is there a benefit for making the tests work on wasm?
On a quick glance this seems to test nothing platform specific @cmyr?
If it does tho, there should be an issue opened for it.

I am not sure about this one. I left it out for the time being because it requires changes in piet to get this to compile, (I submitted a PR to add the missing types to piet in linebender/piet#167) and I'm not really familiar with the testing code. I suspect it can be used for testing although it needs to be run in a special way for wasm and not through cargo test, but I'm not sure. Is this appropriate for a separate PR? For reference there is some code testing the wasm backend in piet.

Sure, this can be done in a later PR, if it makes sens to do so.
Which is what I hoped @cmyr would enlighten me about.

yea, happy for this to be followup.

luleyleo · 2020-04-13T11:11:19Z

druid/examples/game_of_life.rs

+#![allow(clippy::unreadable_literal)]
+


Wow, that's odd. Does not happen without --target wasm32-unknown-unknown 😒
I could not find out why clippy does that tho

I could be wrong but my impression was that the examples were actually not checked by clippy before, but since the druid-wasm-examples is now in the workspace, they all get checked with cargo clippy --all -- -D warnings with or without --target wasm32-unknown-unknown. I remember having to make some other tweaks to the examples to satisfy clippy.

This commit exposes the error of the missing implementation for file dialogs in the web backend using log::warn. This is a temporary solution until the `open_file_sync` and `save_as_sync` functions propagate the error downstream using Result instead of Option. Co-Authored-By: Leopold Luley <[email protected]>

Co-Authored-By: Leopold Luley <[email protected]>

The types in that module have since made their way into druid itself.

This is most likely not needed.

This commit attempts a different approach at including the automatically generated examples from the parent directory. This method satisfies both rustfmt and cargo test as before, but it also doesn't modify any files in the source tree, keeping the diff clean after a build.

elrnv · 2020-04-13T20:19:59Z

Thanks for the feedback! I tried to address all of the comments as best as I can. Passing the baton back now :)

luleyleo

Code wise I'm happy now 😄

I installed wasm-pack and things worked out well!
Playing with the examples I've discovered two issues:

The switch example has an unfortunate name for JS
The identity example color cycling is limited to red
probably instant::Instant being less accurate
As this does not really matter for the example I think this can be ignored

I personally only care about the first point

This commit fixes the generated html file for the switch example to load "switch_demo" instead of "switch" for the switch example. The word switch conflicts with JavaScript's switch statement, which is the reason for this awkwardness.

elrnv · 2020-04-14T00:37:02Z

Regarding the first bullet, for the time being the switch example is renamed to switch_demo when building the WASM interface (the actual issue should be fixed in 3a528fa). Perhaps in another PR we could rename the example itself to switch_demo or something like that, which will clean up a bit of code from the WASM build.rs and lib.rs files.

I did a little digging about the second bullet to find the [MDN article on performance.now()]
(https://developer.mozilla.org/en-US/docs/Web/API/Performance/now) which mentions that browsers reduce the precision of time for privacy reasons.

luleyleo

I've nothing more to add to this 👍
Would be happy to merge it.

cmyr

I've done a scan and everything seems reasonable; I'm happy to at least get this in as a checkpoint!

elrnv mentioned this pull request Mar 26, 2020

Remove refs from WebRenderContext constructor API linebender/piet#153

Merged

elrnv mentioned this pull request Mar 27, 2020

Switch to offset_{width,height} when clearing web context linebender/piet#158

Merged

cmyr added the S-needs-review waits for review label Apr 1, 2020

cmyr reviewed Apr 3, 2020

View reviewed changes

druid/src/contexts.rs Outdated Show resolved Hide resolved

druid-shell/Cargo.toml Outdated Show resolved Hide resolved

druid-shell/src/platform/web/window.rs Outdated Show resolved Hide resolved

druid-shell/src/platform/web/window.rs Outdated Show resolved Hide resolved

cmyr added S-waiting-on-author waits for changes from the submitter and removed S-needs-review waits for review labels Apr 3, 2020

elrnv force-pushed the wasm-support branch from 4d1f67b to 3813e3c Compare April 3, 2020 23:53

futurepaul mentioned this pull request Apr 4, 2020

Simple wasm example #801

Merged

cmyr mentioned this pull request Apr 9, 2020

Update to piet v0.0.12 #824

Merged

elrnv force-pushed the wasm-support branch from 4e4ec4c to d15bcd4 Compare April 10, 2020 21:18

elrnv added 10 commits April 12, 2020 10:25

Add wasm backend + fix std::time dep

19c186f

A basic wasm backend is added as a continuation of linebender#30. Since std::time isn't available on wasm, an additional dependency is added to abstract std::time::Instant to work with wasm.

Disable simple logger on wasm

30d66ba

This commit ensures that all examples run with minimal changes. More specifically we don't need to remove the .use_simple_logger() call on all Apps.

Fix resizing in wasm backends

cfb3627

The resize callback should be registered to the window, since resizing the window doesn't call the resize callback for canvases even if those do get resized.

Added scrolling support + fixed hidpi handling

fbce712

The scroll example now works as expected on hi and low dpi screens.

Use explicit lifetime in StrOrChar

999ef45

This makes passing it easier to pass owned strings when creating a new KeyEvent, which is necessary when interfacing with WASM.

Added keyboard support on wasm

8ee237f

Fix browser zooming issue for web backend

659e136

Fix a typo causing the web build to break

4658a17

Format with cargo fmt

439b319

elrnv added 2 commits April 12, 2020 10:28

Fix x11 keycodes StrOrChar conversion

ea81ffe

The newly introduced lifetime parameter must be specified exmplicitly.

Rustfmt on recent changes to examples

78b4631

elrnv force-pushed the wasm-support branch from 4d619d6 to 78b4631 Compare April 12, 2020 17:31

elrnv added 5 commits April 12, 2020 11:06

Fix clippy warnings -- small structs passed by ref

3cd4144

Fix wasm build for multiwin example

1dbb4e9

Add --no-run to cargo test for wasm targets

2102f82

wasm targets cannot be run in the normal way.

Do not build the unit test module for wasm32

5021017

The tests cannot be run in the normal way anyways. Leaving this for a future PR to flush out.

Install necessary deps in wasm CI for macos/ubuntu

a143692

elrnv mentioned this pull request Apr 12, 2020

Wasm bitmap target linebender/piet#167

Merged

luleyleo requested changes Apr 13, 2020

View reviewed changes

elrnv and others added 7 commits April 13, 2020 11:10

Remove windows specific comment from web/window.rs

0df38af

Co-Authored-By: Leopold Luley <[email protected]>

Ignore automatically generated html files in wasm example

5f64d4b

Co-Authored-By: Leopold Luley <[email protected]>

Remove unused dialog module from web backend

ee88c7f

The types in that module have since made their way into druid itself.

Remove TODO for adding to app state in web backend

ea79c84

This is most likely not needed.

Update Cargo.lock

d0bc9df

luleyleo requested changes Apr 13, 2020

View reviewed changes

Rename switch example js entry point for wasm build

3a528fa

This commit fixes the generated html file for the switch example to load "switch_demo" instead of "switch" for the switch example. The word switch conflicts with JavaScript's switch statement, which is the reason for this awkwardness.

luleyleo approved these changes Apr 14, 2020

View reviewed changes

luleyleo added S-needs-review waits for review and removed S-waiting-on-author waits for changes from the submitter labels Apr 14, 2020

cmyr approved these changes Apr 15, 2020

View reviewed changes

luleyleo merged commit 37b9646 into linebender:master Apr 15, 2020

xStrom mentioned this pull request Apr 15, 2020

Proof of concept druid running on web #30

Closed

elrnv deleted the wasm-support branch April 15, 2020 17:37

xStrom removed the S-needs-review waits for review label May 15, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Wasm support #759

Wasm support #759

elrnv commented Mar 26, 2020 •

edited

Loading

luleyleo commented Mar 26, 2020

elrnv commented Mar 27, 2020

elrnv commented Mar 27, 2020

ijsnow commented Mar 28, 2020

elrnv commented Mar 28, 2020

elrnv commented Mar 30, 2020

cmyr commented Mar 30, 2020

cmyr left a comment

elrnv commented Apr 4, 2020 •

edited

Loading

elrnv commented Apr 8, 2020

cmyr commented Apr 12, 2020

elrnv commented Apr 12, 2020

luleyleo left a comment

luleyleo Apr 13, 2020

elrnv Apr 13, 2020

luleyleo Apr 13, 2020

elrnv Apr 13, 2020

luleyleo Apr 13, 2020

elrnv Apr 13, 2020

luleyleo Apr 13, 2020

cmyr Apr 15, 2020

luleyleo Apr 13, 2020

elrnv Apr 13, 2020

elrnv commented Apr 13, 2020

luleyleo left a comment

elrnv commented Apr 14, 2020

luleyleo left a comment

cmyr left a comment

		//! This module is intentionally left blank.
		//! The contents are automatically generated in the "build.rs" script.

		#![allow(clippy::unreadable_literal)]

Wasm support #759

Wasm support #759

Conversation

elrnv commented Mar 26, 2020 • edited Loading

luleyleo commented Mar 26, 2020

elrnv commented Mar 27, 2020

elrnv commented Mar 27, 2020

ijsnow commented Mar 28, 2020

elrnv commented Mar 28, 2020

elrnv commented Mar 30, 2020

cmyr commented Mar 30, 2020

cmyr left a comment

Choose a reason for hiding this comment

elrnv commented Apr 4, 2020 • edited Loading

elrnv commented Apr 8, 2020

cmyr commented Apr 12, 2020

elrnv commented Apr 12, 2020

luleyleo left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

elrnv commented Apr 13, 2020

luleyleo left a comment

Choose a reason for hiding this comment

elrnv commented Apr 14, 2020

luleyleo left a comment

Choose a reason for hiding this comment

cmyr left a comment

Choose a reason for hiding this comment

elrnv commented Mar 26, 2020 •

edited

Loading

elrnv commented Apr 4, 2020 •

edited

Loading