docs: document generated repos (#1427)

Fixes #1118

docs/Built-ins.md

@@ -26,28 +26,29 @@ To be run in user's WORKSPACE to install rules_nodejs dependencies.
 This rule sets up node, npm, and yarn.
 
 The versions of these tools can be specified in one of three ways:
-- Normal Usage:
+- Simplest Usage:
 Specify no explicit versions. This will download and use the latest NodeJS & Yarn that were available when the
 version of rules_nodejs you're using was released.
+Note that you can skip calling `node_repositories` in your WORKSPACE file - if you later try to `yarn_install` or `npm_install`,
+we'll automatically select this simple usage for you.
 - Forced version(s):
 You can select the version of NodeJS and/or Yarn to download & use by specifying it when you call node_repositories,
-but you must use a value that matches a known version.
+using a value that matches a known version (see the default values)
 - Using a custom version:
 You can pass in a custom list of NodeJS and/or Yarn repositories and URLs for node_resositories to use.
 - Using a local version:
 To avoid downloads, you can check in vendored copies of NodeJS and/or Yarn and set vendored_node and or vendored_yarn
-to point to those before calling node_repositories.
+to point to those before calling node_repositories. You can also point to a location where node is installed on your computer,
+but we don't recommend this because it leads to version skew between you, your coworkers, and your Continuous Integration environment.
+It also ties your build to a single platform, preventing you from cross-compiling into a Linux docker image on Mac for example.
 
-This rule exposes the `@nodejs` workspace containing some rules the user can call later:
+See the [the repositories documentation](repositories.html) for how to use the resulting repositories.
 
-- Run node: `bazel run @nodejs//:node path/to/program.js`
-- Install dependencies using npm: `bazel run @nodejs//:npm_node_repositories install`
-- Install dependencies using yarn: `bazel run @nodejs//:yarn_node_repositories`
 
-Note that the dependency installation scripts will run in each subpackage indicated by the `package_json` attribute.
+## Creating dependency installation scripts for manually-managed dependencies
 
-This approach uses npm/yarn as the package manager. You could instead have Bazel act as the package manager, running the install behind the scenes.
-See the `npm_install` and `yarn_install` rules, and the discussion in the README.
+You can optionally pass a `package_json` array to node_repositories. This lets you use Bazel's version of yarn or npm, yet always run the package manager yourself.
+This is an advanced scenario you can use in place of the `npm_install` or `yarn_install` rules, but we don't recommend it, and might remove it in the future.
 
 Example:
 
@@ -58,6 +59,8 @@ node_repositories(package_json = ["//:package.json", "//subpkg:package.json"])
 
 Running `bazel run @nodejs//:yarn_node_repositories` in this repo would create `/node_modules` and `/subpkg/node_modules`.
 
+Note that the dependency installation scripts will run in each subpackage indicated by the `package_json` attribute.
+
 
 
 ### Usage
@@ -120,10 +123,11 @@ Defaults to `["https://mirror.bazel.build/nodejs.org/dist/v{version}/{filename}"
 Defaults to `"12.13.0"`
 
 #### `package_json`
-(*[labels]*): a list of labels, which indicate the package.json files that will be installed
+(*[labels]*): (ADVANCED, not recommended)
+            a list of labels, which indicate the package.json files that will be installed
             when you manually run the package manager, e.g. with
             `bazel run @nodejs//:yarn_node_repositories` or `bazel run @nodejs//:npm_node_repositories install`.
-            If you use bazel-managed dependencies, you can omit this attribute.
+            If you use bazel-managed dependencies, you should omit this attribute.
 
 Defaults to `[]`
 

docs/_includes/drawer.html

@@ -10,6 +10,7 @@
           <ul>
             <li><a href="install.html">Installing</a></li>
             <li><a href="Built-ins.html">Built-in rules</a></li>
+            <li><a href="repositories.html">Generated repositories</a></li>
           </ul>
         </li>
         <li>

docs/index.md

@@ -16,7 +16,7 @@ This JavaScript support lets you build and test code that targets a JavaScript r
 ## Quickstart
 
 This is the fastest way to get started for most use cases.
-See [the installation page](install.md) for details and alternative methods.
+See [the installation page](install.html) for details and alternative methods.
 
 ```sh
 $ npm init @bazel

docs/repositories.md

@@ -0,0 +1,76 @@
+---
+title: Generated Repositories
+layout: default
+stylesheet: docs
+---
+
+rules_nodejs produces several repositories for you to reference.
+Bazel represents your workspace as one repository, and code fetched or installed from outside your workspace lives in other repositories.
+These are referenced with the `@repo//` syntax in your BUILD files.
+
+## @nodejs
+
+This repository is created by calling the `node_repositories` function in your `WORKSPACE` file.
+It contains the node, npm, and yarn programs.
+
+As always, `bazel query` is useful for learning about what targets are available.
+
+```sh
+$ bazel query @nodejs//...
+@nodejs//:node
+...
+```
+
+You don't typically need to reference the `@nodejs` repository from your BUILD files because it's used behind the scenes
+to run node and fetch dependencies.
+
+Some ways you can use this:
+
+- Run the Bazel-managed version of node: `bazel run @nodejs//:node path/to/program.js`
+- Run the Bazel-managed version of npm: `bazel run @nodejs//:npm`
+- Run the Bazel-managed version of yarn: `bazel run @nodejs//:yarn`
+- Install dependencies from nested package.json file(s) which were passed to `node_repositories#package.json`
+  - using npm: `bazel run @nodejs//:npm_node_repositories install`
+  - using yarn: `bazel run @nodejs//:yarn_node_repositories`
+
+## @npm
+
+This repository is created by calling the `npm_install` or `yarn_install` function in your `WORKSPACE` file.
+
+The name `@npm` is recommended in the simple case that you install only a single `package.json` file.
+If you have multiple, call the `npm_install` or `yarn_install` multiple times, and give each one a unique name.
+This results in multiple repositories, named whatever you chose, rather than "npm".
+The following applies to any repository created by `npm_install` , or `yarn_install`, just replace `@npm` with the name you chose.
+
+Again, use `bazel query @npm//...` to learn about all the targets declared in this repository.
+
+Our philosophy is to mirror the installed npm dependencies in a way that's idiomatic to reference them in Bazel.
+
+Commonly used ones are:
+
+- Every file that was installed from npm: `@npm//:node_modules`. This target can have a very large number of files and slow down your build, however it's a simple way to skip having to declare more fine-grained inputs to your BUILD targets.
+- If you had a dependency on the `foo` package, you can reference `@npm//foo` to get all the files. We mirror the npm dependency graph, so if `foo` declares a dependency on another package `dep`, Bazel will include that dependency when `foo` is used.
+- If the `foo` package has an executable program `bar`, then `@npm//foo/bin:bar` is a `nodejs_binary` that you can call with `bazel run` or can pass as the `executable` to your own rules.
+- Sometimes you need a UMD bundle, but a package doesn't ship one. For example, the `ts_devserver` rule depends on third-party libraries having a named UMD entry point. The `@npm//foo:foo__umd` target will automatically run Browserify to convert the package's `main` entry into UMD.
+- A helper to install npm packages into their own Bazel repository: `@npm//:install_bazel_dependencies.bzl` provides a `install_bazel_dependencies` function. Some npm packages ship custom bazel rules, for example, the `@bazel/typescript` package provides rules which you should load from `@npm_bazel_typescript//:index.bzl`. The `install_bazel_dependencies` function installs such npm packages into their equivalent Bazel repository. (Note, we expect this could be removed in the future, as `load("@npm//@bazel/typescript:index.bzl")` would be a more natural way to load these rules.)
+
+> One convenient (maybe also confusing) way to understand what BUILD files are generated is to look at our integration test at https://github.com/bazelbuild/rules_nodejs/tree/master/internal/npm_install/test/golden - this directory looks similar to the content of an `@npm` repository.
+
+### Generated macros for npm packages with `bin` entries
+
+Any installed package that has one or more `bin` entries in the package.json get convenient macros generated.
+These are the Bazel equivalent of the `./node_modules/.bin/*` files in your project which the package manager created.
+
+For a package `foo` with some bin entries, we will create a `.bzl` file where you can load rules, at `@npm//foo:index.bzl`
+
+If the foo package contains a bin entry `bar`, the `index.bzl` file will contain `bar` and `bar_test` macros. You can load these two generated rules in your BUILD file:
+
+`load("@npm//foo:index.bzl", "bar", "bar_test")`
+
+The `bar` macro can be called in two ways. If you pass `outs` or `output_dir`, it produces an `npm_package_bin` rule that invokes the tool to transform some inputs to outputs, useful as a dependency of another rule, or with `bazel build`. If you don't pass `outs` or `output_dir`, then it will produce a `nodejs_binary` rule intended for use with `bazel run`. (The latter is identical to the `@npm//foo/bin:bar` target, just giving you a convenient way to alias it with a different label and pass it arguments).
+
+See examples in rules_nodejs. A typical tool to use with `outs` is Babel, while a typical rule with no outputs is `http_server`.
+
+The `bar_test` macro produces a `nodejs_test` that assumes the tool is a test runner, and produces a zero or one exit code, useful as a target with `bazel test`. See the examples of `mocha_test` in rules_nodejs.
+
+You can also read https://dev.to/bazel/layering-in-bazel-for-web-389h to see an end-to-end example of using the generated `bin` macros.

internal/node/node_repositories.bzl

@@ -29,28 +29,28 @@ _DOC = """To be run in user's WORKSPACE to install rules_nodejs dependencies.
 This rule sets up node, npm, and yarn.
 
 The versions of these tools can be specified in one of three ways:
-- Normal Usage:
+- Simplest Usage:
 Specify no explicit versions. This will download and use the latest NodeJS & Yarn that were available when the
 version of rules_nodejs you're using was released.
+Note that you can skip calling `node_repositories` in your WORKSPACE file - if you later try to `yarn_install` or `npm_install`,
+we'll automatically select this simple usage for you.
 - Forced version(s):
 You can select the version of NodeJS and/or Yarn to download & use by specifying it when you call node_repositories,
-but you must use a value that matches a known version.
+using a value that matches a known version (see the default values)
 - Using a custom version:
 You can pass in a custom list of NodeJS and/or Yarn repositories and URLs for node_resositories to use.
 - Using a local version:
 To avoid downloads, you can check in vendored copies of NodeJS and/or Yarn and set vendored_node and or vendored_yarn
-to point to those before calling node_repositories.
+to point to those before calling node_repositories. You can also point to a location where node is installed on your computer,
+but we don't recommend this because it leads to version skew between you, your coworkers, and your Continuous Integration environment.
+It also ties your build to a single platform, preventing you from cross-compiling into a Linux docker image on Mac for example.
 
-This rule exposes the `@nodejs` workspace containing some rules the user can call later:
+See the [the repositories documentation](repositories.html) for how to use the resulting repositories.
 
-- Run node: `bazel run @nodejs//:node path/to/program.js`
-- Install dependencies using npm: `bazel run @nodejs//:npm_node_repositories install`
-- Install dependencies using yarn: `bazel run @nodejs//:yarn_node_repositories`
+## Creating dependency installation scripts for manually-managed dependencies
 
-Note that the dependency installation scripts will run in each subpackage indicated by the `package_json` attribute.
-
-This approach uses npm/yarn as the package manager. You could instead have Bazel act as the package manager, running the install behind the scenes.
-See the `npm_install` and `yarn_install` rules, and the discussion in the README.
+You can optionally pass a `package_json` array to node_repositories. This lets you use Bazel's version of yarn or npm, yet always run the package manager yourself.
+This is an advanced scenario you can use in place of the `npm_install` or `yarn_install` rules, but we don't recommend it, and might remove it in the future.
 
 Example:
 
@@ -60,6 +60,8 @@ node_repositories(package_json = ["//:package.json", "//subpkg:package.json"])
 ```
 
 Running `bazel run @nodejs//:yarn_node_repositories` in this repo would create `/node_modules` and `/subpkg/node_modules`.
+
+Note that the dependency installation scripts will run in each subpackage indicated by the `package_json` attribute.
 """
 
 _ATTRS = {
@@ -157,10 +159,11 @@ and expect that file to have sha256sum `00b7a8426e076e9bf9d12ba2d571312e833fe962
         doc = "the specific version of NodeJS to install or, if vendored_node is specified, the vendored version of node",
     ),
     "package_json": attr.label_list(
-        doc = """a list of labels, which indicate the package.json files that will be installed
+        doc = """(ADVANCED, not recommended)
+            a list of labels, which indicate the package.json files that will be installed
             when you manually run the package manager, e.g. with
             `bazel run @nodejs//:yarn_node_repositories` or `bazel run @nodejs//:npm_node_repositories install`.
-            If you use bazel-managed dependencies, you can omit this attribute.""",
+            If you use bazel-managed dependencies, you should omit this attribute.""",
     ),
     "preserve_symlinks": attr.bool(
         default = True,