Merge pull request #36 from neon-bindings/native-cli

RFC: Streamlined UX

text/0000-streamlined-ux.md

@@ -0,0 +1,125 @@
+- Feature Name: streamlined_ux
+- Start Date: 2020-09-17
+- RFC PR: (leave this empty)
+- Neon Issue: (leave this empty)
+
+# Summary
+[summary]: #summary
+
+This RFC proposes streamlining the directory layout and command-line UX of Neon projects, making the design simpler, more intuitive, and easier to learn and adopt.
+
+# Motivation
+[motivation]: #motivation
+
+This RFC proposes three simplifications to the Neon design:
+
+- **Simplified model:** A Neon project is no longer a JavaScript package that _contains_ a native module; a Neon project _is just_ a native module.
+- **Simplified CLI:** Using Neon no longer requires learning a custom CLI tool; Neon projects are operated with the standard `npm` and `cargo` tools.
+- **Simplified layout:** A Neon project no longer contains a mix of JS and Rust subdirectory structures; it's flattened into a standard Cargo project + a `package.json` to treat it as a Node native module.
+
+This design lightens the cognitive burden of learning Neon, making it less intimidating for new users and easier to use for everyone.
+
+# Guide-level explanation
+[guide-level-explanation]: #guide-level-explanation
+
+## Creating a new Neon project
+
+The `npm init neon` command makes it easy to start a new Neon project. You can use `npm init neon <name>` to create a new directory _name_, containing a simple but fully working Neon project.
+
+## Neon project layout
+
+A Neon project looks like a typical Rust project that you might create with `cargo new`, but it also contains a `package.json` manifest with the proper npm scripts to build the project as a Node module with `npm install`. Using `npm init neon <name>`, an initial Neon project directory looks like this:
+
+```
+<name>
+├── .gitignore
+├── Cargo.toml
+├── README.md
+├── package.json
+└── src
+    └── lib.rs
+```
+
+After running a build with `npm install`, the project also contains build artifacts and lockfiles:
+
+```
+<name>
+├── .gitignore
+├── Cargo.lock
+├── Cargo.toml
+├── README.md
+├── index.node
+├── package.json
+├── package-lock.json
+├── src
+|   └── lib.rs
+└── target
+```
+
+The `.gitignore` file generated by `npm init neon` ensures that build artifacts are ignored by git.
+
+## Building a Neon project
+
+The `package.json` contains an `install` script that runs `cargo build` to build the module from Rust source, and then copies the result into `index.node` in the project root. You can do a complete build from source with the `install` script by running `npm install` (or `npm i` for short) at the command-line. (You can also run the build manually with `cargo build` or `cargo build --release`, but then you will have to find the generated library and copy it manually into `./index.node` yourself.)
+
+# Reference-level explanation
+[reference-level-explanation]: #reference-level-explanation
+
+## Replacing `neon new`
+
+The reason we can now move past `neon new` is that `npm` has added support for custom templates via extensibility of the [`npm init`](https://docs.npmjs.com/cli/init) command. This RFC proposes just one single command, `npm init neon`, which would be implemented with a `create-neon` npm package.
+
+## Replacing `neon build`
+
+There are a couple of key developments that will allow us to move past `neon build`:
+
+1. Once we move to the [N-API backend](https://github.com/neon-bindings/neon/issues/444), we no longer need to invalidate a Neon build when changing between versions of Node. This means we won't need to use the custom invalidation logic (which has proved to be fragile and a source of user frustration).
+2. For the final step of copying the built DLL into a `.node` file, we can create a narrower CLI tool that's not intended for end-user use, but just for the `package.json`'s npm scripts to call into. We have built a simple tool called [`cargo-cp-artifact`](https://github.com/neon-bindings/cargo-cp-artifact/). This command can be used with a standard `cargo build` command to capture the name of the generated DLL and copy it to `index.node`. The syntax would look something like:
+```json
+{
+  "scripts": {
+    "build": "cargo-cp-artifact -nc index.node -- cargo build --message-format=json-render-diagnostics"
+  },
+  "devDependencies": {
+    "cargo-cp-artifact": "^0.1"
+  }
+}
+```
+3. Because of the ABI and API stability of N-API, none of the `electron-build-env` settings should be needed for setting custom targets or header files for Electron builds. And with [dynamic loading](https://github.com/neon-bindings/neon/pull/646), we should also be able to avoid needing custom `.lib` files for Windows Electron builds as well.
+
+## Flattening Neon project layout
+
+Historically, `neon new` generated a project layout that assumed that the addon should always be a submodule of a pure-JS wrapper module. This incurs some cognitive overhead for understanding the directory layout, and it's not clear the wrapper structure is actually providing value in the common case.
+
+Making things even more confusing, there were differences between how you would use Neon for an app, a library, or an Electron app.
+
+We can simplify all of these by eliminating the wrapper entirely, and just creating a super simple project structure that's easy enough to understand that users can feel comfortable understanding and modifying it themselves. And the fact that Electron no longer needs custom builds eliminates the need for custom boilerplate generation for those cases.
+
+# Drawbacks
+[drawbacks]: #drawbacks
+
+Some users report liking the `neon` CLI. We don't need to be in a big hurry to eliminate it, and should at least continue to support it for some time in order to avoid disrupting users. Over time, we can decide whether it's becoming less popular. But there's really no need to couple the decision of what to do with `neon` to the decision of whether to do this RFC. Deprecation and removal are separate questions.
+
+# Rationale and alternatives
+[alternatives]: #alternatives
+
+## Alternative: Only use `cargo`
+
+We could build all the tooling through Cargo generators (e.g. with the [cargo-generate](https://github.com/ashleygwilliams/cargo-generate) Cargo addon). This would double down on the idea that Neon is about using Rust in your Node projects.
+
+However, to consumers of a Neon project, it _is_ a JS package, and you still need to use npm e.g. to install dependencies or to publish to the registry. So it makes sense to use `npm init` for creating a Neon project and `npm i` for building the project (including fetching and building any dependencies it may have). However, it's still useful for `cargo build` to work in the root directory for convenience, especially for cases where the user may add additional build logic to `npm install` but still want to be able to just build the Rust code by itself.
+
+We could use an `@neon` namespace instead of `create-neon-lib` etc. However, this makes for a noisier and less convenient-to-type command-line experience:
+```
+npm init @neon/app my-app
+```
+(Note that as a _programming_ syntax, this might arguably be preferable for the visual distinction. But the command-line is meant to be quick and easy to type and remember.) Another option would be to support both as aliases for each other, but this might just overwhelm users with pointless choices.
+
+We could offer a zero-argument version of `create-neon`, which would initialize an existing project. However, this approach irrevocably modifies a user's existing directory structure, which can leave them in a painful intermediate state on failure. It's a more robust and less stressful user experience to know that `create-neon` will only ever build a new directory structure from scratch, and if anything goes wrong it's safe to delete the generated directory and start over.
+
+Moreover, despite the historical behavior of `npm init`, this approach of generating a new directory structure from scratch is standard practice these days not only in the Rust ecosystem (with `cargo new`), but in the JS ecosystem as well with tools like [`create-react-app`](https://create-react-app.dev)—which was in fact the tool that inspired the extensibility of `npm init`. So we appear to be safely within the bounds of idiomatic JS tooling behavior.
+
+# Unresolved questions
+[unresolved]: #unresolved-questions
+
+None.