Parcel 2 ready for production?

[talkingtab]

Parcel is just great. However, I can't use parcel-bundler because I use react-bootstrap. Parcel 2 works just fine with react-bootstrap. My sense is to use Parcel 2 despite it being alpha for my production release. Either it will work or some testing will show it has problems. It would be nice to get some guidance of issues, things to look for, concerns etc.

Please don't force me back to other build systems. My build size is just great, its easy to use etc.

[DeMoorJasper]

You are free to use it in production at your own risk, the main remaining issues are stability, caching and potential breaking changes. Also documentation is pretty much non existent for Parcel 2 which is also something that might be annoying sometimes... But we'll be working on that soonish

So as long as you're not writing any custom plugins and keep in mind that you might have to remove cache on a weird bug you should be fine. These issues should get resolved before a stable release.

[dandv]

You might want to pin your version to a stable one rather than @next. I've just installed [email protected] and I get "404 Page no found" when serving a simple HTML file with no links to other sources.

[talkingtab]

@dandv Good thought, thanks much for the idea.

@DeMoorJasper - I'm having a hard time finding the doc for how to import an svg file, in parcel-bundler this worked

import menusvg from "./images/menu.svg" 

Would appreciate ideas. Other than that (and pinning a version) 2.0 seems good. Thanks!

[DeMoorJasper]

We have named pipelines in Parcel 2 this removed the implicit url importing you now have to use import icon from url:./images/menu.svg;

[cdbattags]

Looks like a few of the "loaders" (unsure if that's the right terminology) are not working on v2+? I just tried with a Rust file and got an error?

[D1plo1d]

I ran into the same issue as @cdbattags . I see #3365 is tracking Rust/Parcel 2 but there hasn't been an update in a bit. Is the Rust integration still planned for Parcel 2?

[DeMoorJasper]

@D1plo1d we're still planning to support rust but we don't have any Rust users/experts in the Parcel core team currently so it's probably best if someone external with better knowledge of rust would contribute the rust support like what happened in Parcel 1.

At some point we will probably port the old Parcel 1 rust transformer to Parcel 2 if no-one does. We're also mainly focused on getting the current version stable and ready for a beta and stable version.

[mysterycommand]

Hey, hi, I'm the author of parcel-plugin-wasm-pack (on npm here). As I mentioned in #3365 (comment) I'd be interested in contributing to Rust/wasm support in Parcel 2 (either as a plugin or as part of the official project), but I'm a bit confused as to where/how to start.

There's a RustAsset that's registered to intercept rs files, and emit wasm type assets … shouldn't this be made to work only on Cargo.toml files? Is there a way to build a wasm asset from a Rust file without a Cargo.toml?

• packages/core/parcel-bundler/src/assets/RustAsset.js

These integration tests all mention wasm somewhere in them, but where they do they're all .skipped. Which ones should I focus on first?

• packages/core/integration-tests/test/javascript.js
• packages/core/integration-tests/test/rust.js
• packages/core/integration-tests/test/wasm.js

There appear to be several wasm-loader.jss:

• packages/runtimes/js/src/loaders/browser/wasm-loader.js
• packages/runtimes/js/src/loaders/node/wasm-loader.js
• packages/runtimes/js/src/loaders/worker/wasm-loader.js
• packages/core/parcel-bundler/src/builtins/loaders/browser/wasm-loader.js
• packages/core/parcel-bundler/src/builtins/loaders/node/wasm-loader.js

Anyway, I guess this is just my way of getting the lay of the land here. Looking in packages/core/integration-tests/test/integration I see several sort of "project fixtures" … the rust (with no Cargo.toml) one should, I think, be considered invalid? The rest fall into two categories:

1. I already have a .wasm asset (generated elsewhere) and I want to import it
2. I have a .rs asset, and I want to import it (generating a .wasm asset as part of the build process)

I think both these cases don't really fit with how wasm-pack wants to treat crates being turned into wasm modules, so I guess I'm going to propose a third case (and maybe even suggest deprecating the first two? … the behavior seems surprising or unclear to me, though I'd be happy to learn about what use cases these approaches support).

3. I have a crate (e.g. a Cargo.toml and one or more .rs files … default crate entries are lib.rs or main.rs but a custom entry can be specified int the Cargo.toml I believe) and I want to import it (again, generate the .wasm asset as part of the build)

Okay, so the base cases I think would look like these:

1. A single crate mixed into the Parcel "entry" with intermingled source files:

   ├── src
   │   ├── entry.{html|js}
   │   └── lib.rs
   ├── Cargo.toml
   ├── package.json
   └── yarn.lock

2. Multiple crates in their own cargo workspace that live alongside the Parcel "entry":

   ├── crates
   │   ├── hello
   │   │   ├── src
   │   │   │   └── lib.rs
   │   │   └── Cargo.toml
   │   └── world
   │       ├── src
   │       │   └── lib.rs
   │       └── Cargo.toml
   ├── src
   │   └── entry.{html|js}
   ├── Cargo.toml
   ├── package.json
   └── yarn.lock

I might be biased here, these are pretty much the examples I set up in my Parcel 1 plugin. The new implementation would be different in that it wouldn't let an entry import lib.rs it'd have to import the Cargo.toml … so in the "single crate" case:

diff --git a/examples/single/src/index.js b/examples/single/src/index.js
index b055b85..fd4a856 100644
--- a/examples/single/src/index.js
+++ b/examples/single/src/index.js
@@ -1,2 +1,2 @@
-import { run } from './lib.rs';
+import { run } from '../Cargo.toml';
 run();

… and the multiple:

diff --git a/examples/multiple/src/index.js b/examples/multiple/src/index.js
index d1cc5fe..c31baea 100644
--- a/examples/multiple/src/index.js
+++ b/examples/multiple/src/index.js
@@ -1,5 +1,5 @@
-import { hello } from '../crates/hello/src/lib.rs';
-import { world } from '../crates/world/src/lib.rs';
+import { hello } from '../crates/hello/Cargo.toml';
+import { world } from '../crates/world/Cargo.toml';

 hello();
 world();

I think this'd actually work with my plugin as is, but importing lib.rs directly would not/should not be supported in the new version … I think.

---

Okay, sorry this is getting so long, but if you're still with me, I think I'm getting closer to an idea of the solution. Wasm-pack will generate some files using it's bundler target (assuming a project generated with wasm-pack new abc and then wasm-pack build):

├── README.md
├── abc.d.ts
├── abc.js
├── abc_bg.d.ts
├── abc_bg.js
├── abc_bg.wasm
└── package.json

All we're really interested in are abc_bg.js and abc_bg.wasm … the .js file exports the values that need to be passed in as the imports object to WebAssembly.instantiateStreaming or WebAssembly.instantiate or new WebAssembly.Instance. The contents of abc.js (no _bg in the file name) looks like this:

import * as wasm from './abc_bg.wasm';
export * from './abc_bg.js';

This is sort of the way of the future, for when ES module integration lands. For now we've got to do the imperative way, so in parcel-plugin-wasm-pack I swap the first line of abc_bg.js and tack on an intializer function. The initializer function gathers up the imports object and passes it to a loader (which does the module fetching and instantiation) and then sets the wasm module in the local scope and exports the public interface.

It's a bit of juggling, but I think it's the right thing to do for the time being … and unfortunately we're not quite done. At this point the entry module is trying to access members of a wasm module, but that wasm module is wrapped up in the loader promise.

In my plugin's implementation a packager is added to check if bundles include wasm assets, and if they do it inserts a new entry module that wraps those promises in a Promise.all and when they're all resolved it replaces the module in Parcel's cache with the wasm results and then calls the original entry module.

More juggling, but again, I think it's the right thing to do … so, I'm really tired now, but I guess my questions are:

1. Is this approach still valid in Parcel 2?
2. Assuming so, where should the equivalent of my plugin's WasmPackAsset and WasmPackPackager go in the new architecture?
3. Should I replace RustAsset, and make it only work on toml files, filter on Cargo.tomls, and forward handling of non-Cargo tomls to TOMLAsset/TOMLTransformer?
4. What of the various wasm-loader.jss?

Anyway, I hope that makes sense. Thanks in advance for your thoughtful consideration. Looking forward to getting this working!

[mysterycommand]

Okay, thank you … I think I've got the start of something here:

v2...mysterycommand:v2-wasm-pack

I still need to figure out how to wire it all together though. The integration test there passes, but seems to indicate that I still only have the 2 assets in the bundle even though I'm returning 3 from the transformer … so I must be doing something wrong there. Couple questions about this:

1. Is there a test util or something that would let me inspect the contents of the bundle?
2. Do I need to do more to add assets to a bundle? new MutableAsset() or something?
3. What's the intended use for asset.addDependency, asset.addIncludedFile, and asset.addURLDependency?

Also, in the v1 plugin, I had to add a custom packager to replace the entry and do some hacky cache-replacement stuff to get this all working.

It's been a while since I worked on that, but iirc it's loading and initializing all the wasm modules and then swapping the result of the load/init promise in the cache, and then calling the original entry module. This seems like kind of a lot of juggling/magic and maybe a little hacky, but is there a better way to do it in v2?

Also any general feedback (specifically help with understanding how to make that integration test more meaningful) on that [WIP] branch would be greatly appreciated.

[mysterycommand]

Alright, I think I've got some progress on my fork/branch, but I'm kinda stuck and looking for help. The integration test I described before is giving me this output:

$ yarn test:integration -g wasm-pack
yarn run v1.22.4
$ yarn workspace @parcel/integration-tests test -g wasm-pack
$ cross-env NODE_ENV=test PARCEL_BUILD_ENV=test mocha -g wasm-pack


  wasm-pack
    1) should work


  0 passing (2s)
  1 failing

  1) wasm-pack
       should work:
     Error: No transformers found for "/Users/matt/Sites/mysterycommand/parcel/packages/core/integration-tests/test/integration/wasm-pack-single/pkg/single_bg.wasm".
      at ParcelConfig._getTransformerNodes (/Users/matt/Sites/mysterycommand/parcel/packages/core/core/src/ParcelConfig.js:217:13)
      at ParcelConfig.getTransformerNames (/Users/matt/Sites/mysterycommand/parcel/packages/core/core/src/ParcelConfig.js:228:29)
      at ConfigLoader.loadParcelConfig (/Users/matt/Sites/mysterycommand/parcel/packages/core/core/src/ConfigLoader.js:63:37)
      at /Users/matt/Sites/mysterycommand/parcel/packages/core/core/src/ConfigLoader.js:43:48
      at processTicksAndRejections (internal/process/task_queues.js:97:5)
      at PromiseQueue._runFn (/Users/matt/Sites/mysterycommand/parcel/packages/core/utils/src/PromiseQueue.js:84:7)
      at PromiseQueue._next (/Users/matt/Sites/mysterycommand/parcel/packages/core/utils/src/PromiseQueue.js:73:5)



error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
error Command failed.
Exit code: 1
Command: /Users/matt/.nvm/versions/node/v12.18.3/bin/node
Arguments: /Users/matt/.nvm/versions/node/v12.18.3/lib/node_modules/yarn/lib/cli.js test -g wasm-pack
Directory: /Users/matt/Sites/mysterycommand/parcel/packages/core/integration-tests
Output:

info Visit https://yarnpkg.com/en/docs/cli/workspace for documentation about this command.
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

The No transformers found error also occurs if I run yarn build from either of the examples I also added (in packages/examples/wasm-multiple-crates and packages/examples/wasm-single-crate … copied mostly from my old plugin implementation).

I think I'm doing everything to load wasm file via url (using import wasmUrl from 'url:./pkg/simple_bg.wasm'; in the "glue code" module I'm adding), so I'm not sure why it's trying to transform the wasm module at all. What I think I want is this:

1. a loader module, copied from packages/transformers/wasm-pack/src/loaders (either browser or node depending on asset.env.isNode())
   • this just asynchronously loads and instantiates the wasm module (using fetch or readFile depending on the loader/context), passing in the imports object
2. a wasm module (this is created by wasm-pack … in the case of the integration test this is simple_bg.wasm)
3. an "initializer" module (this is a modified version of what's produced by wasm-pack at simple_bg.js)
   • it's modified so that it exports a single function that uses the loader from 1. and generates the imports object to be passed in, and then returns the "public names" of wasm-exposed functions after the loader's load/instantiate promise resolves
4. a glue module that gets the init function from 3. and the url of the wasm module described by 2. (via 'url:./pkg/simple_bg.wasm' for example) and does export default init(wasmUrl);

From my WasmPackTransformer's transform function, I'm returning:

[
  {
    filePath: loaderPath, // e.g. './pkg/browser-loader.js'
    type: 'js',
    content: loader,
  },
  {
    filePath: jsPath, // e.g. './pkg/simple_bg.js'
    type: 'js',
    content: initStr,
  },
  {
    filePath: asset.filePath, // e.g. './Cargo.toml' (the path of the file I'm transforming)
    type: 'js',
    content: glueStr, // the "glue code" described by 4. above
  },
]

Anyway, do I need to do something with asset.addDependency, asset.addIncludedFile, and asset.addURLDependency? I'm happy to answer any questions about the setup I've tried to describe. It's kinda convoluted maybe, but I think it's at least fully loaded into my brain at this point. Thanks in advance!

[mysterycommand]

Okay I think I figured the error out … I can't remember where, but there's a comment somewhere in core that reads something like "ignore content if the file already exists in the cache" or something like that, so I just had do await options.inputFS.writeFile(jsPath, initStr); to get the correct output into the bundle (the old version of the file was trying to import the wasm module directly).

Anyway, still not quite working, but it's closer!

[mysterycommand]

In the spirit of "make it work, make it good, make it fast" … this works: #4970 (I make no guarantees about it being good or fast at this time).

[vinhvn]

Any progress on the PR? I'm working on a library project w/ webpack atm but it's not handling wasm's dynamic import correctly for my use-case so I'm looking into parcel as a possible replacement.