[feature]: Shareable Babel "preset" package to simplify build config

[zetlen]

Peregrine and Venia modules should be easy to consume in a third-party app, like any PWA created with the future scaffolding CLI. But in order to consume it, you need to be able to parse and transpile it.

Right now, we have no guarantees about transpiler configuration outside of this repository itself. So we have to keep Peregrine code, Venia code, and any other code that Webpack may bundle, aligned with the features supported in the root babel.config.js...and people consuming our code don't even have access to that.

Remove our workaround

We came up with an intermediate solution for portability in #788: we added an intermediate compilation step which transpiles Peregrine code into slightly more "generic" ES Modules. This comes at a cost though:

• Slower build times
• More complex scripting
• Serious bugs with hot reloading
• No clear definition of the intermediate compilation target

And even this "generic" transpile is still hard to anticipate for third party consumers.

Use a Babel Preset

The right way to do this is a Babel preset. Instead of maintaining complex Babel configuration in our root repository, we should maintain a separate package that other projects can consume and use for its base Babel configuration. Any project which builds Peregrine code would simply have to include the @magento/babel-preset-peregrine module in their devDependencies.

Decide what really needs Babel transpilation

"Build" is a broad category. Lots of our code will require build steps, and some of them are as simple as copying files from one directory to another. Those operations are usually very fast, so this issue is focused on code transpilation via Babel and where/when it should take place.

We use Babel to transpile source code so it will run on a target platform, like Node or a particular browser. Babel is used in the following packages:

• venia-concept
  • Package scripts run webpack, Webpack babel-loader transpiles for browser compatibility
• peregrine
  • Package scripts run babel CLI to build CommonJS modules for third parties who aren't using ESM
  • Package scripts run babel CLI to build "generic" ES modules for venia-concept to consume via @magento/peregrine/esm/* imports
• pwa-buildpack
  • Package scripts run babel CLI to transpile for Node 8-10 compatibility

None of the other packages use Babel at all. (Our test suite transpiles all code on the fly, whether it needs it or not, because of the presence of a root babel.config.js. This is maintenance-free, though it probably slows down our test runs unnecessarily.)

Only Venia really needs a build step. Peregrine code will transpile as a result of being included in modules processed by Webpack. It just needs to use the same set of Babel features. A Babel preset will help us to keep track of that. (We happen to have a build step for Buildpack, whose source code uses a few Babel features. I recommend that we get rid of this eventually.)

Formalize a directory structure for library code, source code, and build artifacts

Most of the time, third-party projects can consume our source modules directly:

import useQuery from `@magento/peregrine/src/hooks/useQuery`

If we keep all of our modules re-exported in index.js files, then third-party projects can use the cleaner API

import { useQuery } from `@magento/peregrine`;

This is nice, because it means the directory structure of the published module is not part of the public API. However, it risks including extra dead code in the bundle unless tree-shaking is set up perfectly. For that reason alone, I think we should continue to expose our directory structure as public API for import.

But this means specifying a full path to the imported module, including its base directory. Since Peregrine was set up with its own build step, the base directory is src: therefore you have to import @magento/peregrine/src/hooks/useQuery instead of @magento/peregrine/hooks/useQuery. This is a minor thing, but it needs to be consistent.

I propose the following rules:

• **All code that expects to be compatible with PWA Studio code should use a new Babel preset @magento/babel-preset-peregrine
• Design all packages to require as little individual build as possible
• Use Babel only when compiling the final app Venia
• Any code meant to be imported, not directly transpiled, should be in a /lib directory relative to package root. This indicates that the files are library code that will be included in a final bundle.
• **Any code meant to be directly transpiled, like the source code of Venia which Venia runs Webpack against, should be in a /src directory relative to package root.
• **Build artifacts should go to a /dist directory relative to package root.

If we create and maintain a Babel preset, and then stick to this convention, we will have faster and simpler repository management and easier-to-maintain APIs.

Please let us know what packages this feature is in regards to:

• venia-concept
• pwa-buildpack
• peregrine
• pwa-devdocs
• upward-js
• upward-spec