Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for wasm-bindgen. #240

Merged
merged 19 commits into from
Oct 31, 2019
Merged
Show file tree
Hide file tree
Changes from 18 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .bazelci/presubmit.yml
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,10 @@ tasks:
- "-@examples//hello_lib:hello_lib_doc_test"
- "-//tools/runfiles:runfiles_doc_test"
- "-@examples//ffi/rust_calling_c/simple/..."
# See https://github.com/bazelbuild/bazel/issues/9987
- "-@examples//ffi/rust_calling_c:matrix_dylib_test"
# rust-lld isn't available on RBE
- "-@examples//hello_world_wasm:hello_world_wasm_test"
examples:
name: Examples
platform: ubuntu1804
Expand Down
1 change: 1 addition & 0 deletions AUTHORS
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
# The email address is not required for organizations.

Google Inc.
Spotify AB
Damien Martin-Guillerez <[email protected]>
David Chen <[email protected]>
Florian Weikert <[email protected]>
Expand Down
1 change: 1 addition & 0 deletions CONTRIBUTORS
Original file line number Diff line number Diff line change
Expand Up @@ -19,3 +19,4 @@ Kristina Chodorow <[email protected]>
Philipp Wollermann <[email protected]>
Ulf Adams <[email protected]>
Justine Alexandra Roberts Tunney <[email protected]>
John Edmonds <[email protected]>
9 changes: 9 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,15 @@ This repository provides rules for building [Rust][rust] projects with [Bazel](h
</ul>
</div>

#### WebAssembly

To build a `rust_binary` for wasm32-unknown-unknown add the `--platforms=//rust/platform:wasm` flag.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a great start, but I think that the requirement to set target platform is quite limiting.

WebAssembly is used outside of browsers / JavaScript host environments, and Wasm modules can be loaded by other build targets in the same Bazel workspace, and those build targets must be built using "native" platforms and not the //rust:platform:wasm pseudo-platform.

For example, imagine such BUILD file:

cc_library(
    name = "runtime",
    srcs = "runtime.cc",
    data = [
        ":wasm",
    ],
)

rust_library(
    name = "wasm",
    srcs = ["wasm.rs"],
    edition = "2018",
    crate_type = "cdylib",
    rustc_flags = ["--target=wasm32-unknown-unknown"],
)

where Wasm module compiled using rust_library must be loaded by the runtime written in C/C++.

The ability to override --target is something that I have hacked in my local tree, but ideally we would have something like rust_wasm_library that uses --crate-type=cdylib --target=wasm32-unknown-unknown to compile Wasm modules.

Basically, we need the ability to indicate Wasm target on a per build target basis, and not globally.

What do you think?

Copy link
Contributor Author

@johnedmonds johnedmonds Oct 14, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@PiotrSikora Thanks for the comment! I think I had a similar problem where I wanted to include the compiled wasm (compiled as wasm32-unknown-unknown) with include_bytes and serve it from the binary (compiled as a native Rust binary). We allow it in this PR by using Bazel transitions. See https://github.com/bazelbuild/rules_rust/pull/240/files#diff-9a42a2c125ed40c511198be1665d0faeR346. This allows us to build the entire build with the default target platform (e.g. x86 + Linux) but temporarily transition to //rust:platform:wasm when building the wasm code. We also handle transitioning to the host platform when building proc-macro crates.

The easiest way is to do this in practice is to make a rust_wasm_bindgen target which will force the transition for the dependent rust_library (and transitively for all its dependencies). So I think what you want to do is this:

cc_library(
    name = "runtime",
    srcs = "runtime.cc",
    data = [
        ":wasm_bindgen_bg.wasm",
    ],
)

rust_wasm_bindgen(
    name = "wasm_bindgen",
    wasm_file = ":wasm"
)

rust_library(
    name = "wasm",
    srcs = ["wasm.rs"],
    edition = "2018",
)

With this you should be able to build :runtime as a native library and :wasm will automatically get built as //rust:platform:wasm.

I have a note about this here https://github.com/bazelbuild/rules_rust/pull/240/files/7305a12fd276b0c21432d00d4e2ddd8b15115e13#diff-04c6e90faac2675aa89e2176d2eec7d8R31. As I read my note again I realize that perhaps it's a bit unclear since "transition" is a Bazel-specific word. Do you think it would be clearer if I referenced it in this paragraph? Or did I misunderstand your usecase?

Basically in this paragraph I was trying to explain how you could get the .wasm file if that's the only thing you wanted.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, the transitions are great, I didn't realize that you could do it this way.

I think that rust_wasm_bindgen is close to what I want, but after reading that paragraph, I simply ignored it, because I don't want anything related to wasm-bindgen, I want pure Rust code that's compiled to wasm32-unknown-unknown.

As such, I'm suggesting that we would add rust_wasm_library that uses the transitions that you've already added for rust_wasm_bindgen, i.e.

cc_library(
    name = "runtime",
    srcs = "runtime.cc",
    data = [
        ":wasm",
    ],
)

rust_wasm_library(
    name = "wasm",
    deps = ":library"
)

rust_library(
    name = "library",
    srcs = ["library.rs"],
    edition = "2018",
)

Alternatively, since you're already doing it for proc-macro, I think that we could use crate_type = "wasm32" and simply transition to wasm32-unknown-unknown in the rust_library, i.e.

cc_library(
    name = "runtime",
    srcs = "runtime.cc",
    data = [
        ":library",
    ],
)

rust_library(
    name = "library",
    srcs = ["library.rs"],
    crate_type = "wasm32",
    edition = "2018",
)

What do you think?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Friendly ping before this gets merged.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you think we should do it in a different PR? There are several ways to do this like:

  1. The method you proposed of introducing a rule to force a transition
  2. Adding a setting to the rust_library to enable the transition
  3. Change Bazel to make it easier to do transitions on demand.
  4. Existing workaround via rust_wasm_bindgen.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor

@PiotrSikora PiotrSikora Oct 30, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm definitely against (4), since this feature has nothing to do with wasm-bindgen.

(3) would still require action from the person building the target (vs person writing BUILD file), so it's not much different than requiring extra --platforms=wasm32 flag, IMHO.

I'm fine with either (1) or (2), with a slight preference for (1), since then you could easily add a target that extends existing rust_library to be also compiled as Wasm module (i.e. you could have both native and Wasm build targets built at the same time) by simply adding few lines with rust_wasm_library (see the example above). You could achieve the same with (2), but then you'd need to duplicate the complete build target, so it would be more error prone, I think.

Also, there is already a precedence for rust_xxx_library with rust_proto_library and rust_grpc_library.

It's fine if you want to add it in a separate PR (i.e. I don't want to block this PR from getting merged), but could you rename it to "Add support for wasm-bindgen", since it's not a complete WebAssembly support?

What do you think?


bazel build @examples//hello_world_wasm --platforms=//rust/platform:wasm

`rust_wasm_bindgen` will automatically transition to the wasm platform and can be used when
building wasm code for the host target.

### Protobuf
<div class="toc">
<ul>
Expand Down
12 changes: 11 additions & 1 deletion WORKSPACE
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,9 @@ rust_proto_repositories()
load("@io_bazel_rules_rust//bindgen:repositories.bzl", "rust_bindgen_repositories")
rust_bindgen_repositories()

load("@io_bazel_rules_rust//wasm_bindgen:repositories.bzl", "rust_wasm_bindgen_repositories")
rust_wasm_bindgen_repositories()

# Stardoc and its dependencies
http_archive(
name = "io_bazel_skydoc",
Expand All @@ -54,9 +57,16 @@ skydoc_repositories()
load("@io_bazel_rules_sass//:package.bzl", "rules_sass_dependencies")
rules_sass_dependencies()

load("@build_bazel_rules_nodejs//:defs.bzl", "node_repositories")
load("@build_bazel_rules_nodejs//:defs.bzl", "node_repositories", "npm_install")
node_repositories()

# Dependencies for the @examples//hello_world_wasm example.
npm_install(
johnedmonds marked this conversation as resolved.
Show resolved Hide resolved
name = "npm",
package_json = "//:package.json",
package_lock_json = "//:package-lock.json",
)

load("@io_bazel_rules_sass//:defs.bzl", "sass_repositories")
sass_repositories()
# --- end stardoc
Expand Down
21 changes: 21 additions & 0 deletions examples/WORKSPACE
Original file line number Diff line number Diff line change
Expand Up @@ -37,10 +37,31 @@ http_archive(
],
)

http_archive(
name = "build_bazel_rules_nodejs",
url = "https://github.com/bazelbuild/rules_nodejs/archive/0.16.2.zip",
strip_prefix = "rules_nodejs-0.16.2",
sha256 = "9b72bb0aea72d7cbcfc82a01b1e25bf3d85f791e790ddec16c65e2d906382ee0"
)

load("@build_bazel_rules_nodejs//:defs.bzl", "node_repositories", "npm_install")
node_repositories()

# Dependencies for the @examples//hello_world_wasm example.
npm_install(
name = "npm",
package_json = "@io_bazel_rules_rust//:package.json",
package_lock_json = "@io_bazel_rules_rust//:package-lock.json",
)

load("@io_bazel_rules_rust//bindgen:repositories.bzl", "rust_bindgen_repositories")

rust_bindgen_repositories()

load("@io_bazel_rules_rust//wasm_bindgen:repositories.bzl", "rust_wasm_bindgen_repositories")

rust_wasm_bindgen_repositories()

load("@io_bazel_rules_rust//:workspace.bzl", "bazel_version")

bazel_version(name = "bazel_version")
39 changes: 39 additions & 0 deletions examples/hello_world_wasm/BUILD
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
package(default_visibility = ["//visibility:public"])

load(
"@io_bazel_rules_rust//rust:rust.bzl",
"rust_binary",
)

load(
"@io_bazel_rules_rust//wasm_bindgen:wasm_bindgen.bzl",
"rust_wasm_bindgen",
)

load(
"@build_bazel_rules_nodejs//:defs.bzl",
"jasmine_node_test",
)

rust_binary(
name = "hello_world_wasm",
srcs = ["main.rs"],
edition = "2018",
deps = ["@io_bazel_rules_rust//wasm_bindgen/raze:wasm_bindgen"],
)

rust_wasm_bindgen(
name = "hello_world_wasm_bindgen",
wasm_file = ":hello_world_wasm"
)

jasmine_node_test(
name = "hello_world_wasm_test",
srcs = [
"hello_world_wasm_test.js",
],
data = [
":hello_world_wasm_bindgen_bg.wasm",
"@npm//jasmine",
],
)
10 changes: 10 additions & 0 deletions examples/hello_world_wasm/hello_world_wasm_test.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
const fs = require('fs');
const path = require('path');

describe('Calling WebAssembly code', () => {
it('Should run WebAssembly code', async () => {
const buf = fs.readFileSync(path.join(__dirname, 'hello_world_wasm_bindgen_bg.wasm'));
const res = await WebAssembly.instantiate(buf);
expect(res.instance.exports.hello_world(2)).toEqual(4);
})
});
10 changes: 10 additions & 0 deletions examples/hello_world_wasm/main.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
use wasm_bindgen::prelude::*;

#[wasm_bindgen]
pub fn hello_world(i: i32) -> i32 {
i * 2
}

fn main() {
println!("Hello {}", hello_world(2));
}
110 changes: 110 additions & 0 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

7 changes: 7 additions & 0 deletions package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
{
"private": true,
"devDependencies": {
"jasmine": "3.4.0"
},
"scripts": {}
}
8 changes: 8 additions & 0 deletions rust/platform/BUILD
Original file line number Diff line number Diff line change
Expand Up @@ -3,3 +3,11 @@ package(default_visibility = ["//visibility:public"])
load(":platform.bzl", "declare_config_settings")

declare_config_settings()

package_group(
name = "function_transition_whitelist",
packages = [
"//...",
],
)

12 changes: 12 additions & 0 deletions rust/platform/platform.bzl
Original file line number Diff line number Diff line change
Expand Up @@ -79,3 +79,15 @@ def declare_config_settings():
name = triple,
constraint_values = triple_to_constraint_set(triple),
)

native.constraint_value(
name = "wasm32",
constraint_setting = "@platforms//cpu"
)

native.platform(
name = "wasm",
constraint_values = [
"@io_bazel_rules_rust//rust/platform:wasm32",
]
)
6 changes: 6 additions & 0 deletions rust/platform/triple_mappings.bzl
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ _SYSTEM_TO_BINARY_EXT = {
"darwin": "",
"windows": ".exe",
"emscripten": ".js",
"unknown": "",
}

_SYSTEM_TO_STATICLIB_EXT = {
Expand All @@ -52,6 +53,7 @@ _SYSTEM_TO_STATICLIB_EXT = {
# TODO(acmcarther): To be verified
"windows": ".lib",
"emscripten": ".js",
"unknown": "",
}

_SYSTEM_TO_DYLIB_EXT = {
Expand All @@ -61,6 +63,7 @@ _SYSTEM_TO_DYLIB_EXT = {
# TODO(acmcarther): To be verified
"windows": ".dll",
"emscripten": ".js",
"unknown": ".wasm",
acmcarther marked this conversation as resolved.
Show resolved Hide resolved
}

def cpu_arch_to_constraints(cpu_arch):
Expand Down Expand Up @@ -120,6 +123,9 @@ def triple_to_constraint_set(triple):
if len(component_parts) == 4:
abi = component_parts[3]

if cpu_arch == "wasm32":
return ["@io_bazel_rules_rust//rust/platform:wasm32"]

constraint_set = []
constraint_set += cpu_arch_to_constraints(cpu_arch)
constraint_set += vendor_to_constraints(vendor)
Expand Down
13 changes: 13 additions & 0 deletions rust/private/dummy_cc_toolchain/BUILD
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
load(":dummy_cc_toolchain.bzl", "dummy_cc_toolchain")
dummy_cc_toolchain(name = "dummy_cc_wasm32")

# When compiling Rust code for wasm32, we avoid linking to cpp code so we introduce a dummy cc
# toolchain since we know we'll never look it up.
# TODO([email protected]): Need to support linking C code to rust code when compiling for wasm32.
toolchain(
name = "dummy_cc_wasm32_toolchain",
toolchain = ":dummy_cc_wasm32",
toolchain_type = "@bazel_tools//tools/cpp:toolchain_type",
target_compatible_with = ["//rust/platform:wasm32"]
)

8 changes: 8 additions & 0 deletions rust/private/dummy_cc_toolchain/dummy_cc_toolchain.bzl
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
def _dummy_cc_toolchain_impl(ctx):
return [platform_common.ToolchainInfo()]

dummy_cc_toolchain = rule(
implementation = _dummy_cc_toolchain_impl,
attrs = {},
)

Loading