Peer dep failure with npm_install but not with yarn_install

[Dig-Doug]

Hi, I'm running into an issue with the npm_install rule. For some reason, the npm_install rule fails on my package.json, but when I switch to the yarn_install rule, everything works fine. Running npm install without bazel in the directory also works.

Steps to reproduce

git clone https://github.com/Dig-Doug/ts-protoc-gen.git
cd ts-protoc-gen
git checkout 1bd65c9d7b710aabbd9d2e179e082692090598db
bazel test //test/bazel/...:all

# Build fails
Starting local Bazel server and connecting to it...
INFO: SHA256 (https://github.com/bazelbuild/rules_nodejs/archive/0.15.1.zip) = a0a91a2e0cee32e9304f1aeea9e6c1b611afba548058c5980217d44ee11e3dd7
INFO: SHA256 (https://codeload.github.com/google/protobuf/zip/48cb18e5c419ddd23d9badcfe4e9df7bde1979b2) = b6b42f90c60b54732f764ae875623a9b05e6eede064173c36c6fea12dd376cdd
ERROR: /home/doug/.cache/bazel/_bazel_doug/b05db80e2dc3f398f49b0e15d16c5b02/external/ts_protoc_gen/bin/BUILD.bazel:5:1: no such package '@ts_protoc_gen_deps//': Traceback (most recent call last):
	File "/home/doug/.cache/bazel/_bazel_doug/b05db80e2dc3f398f49b0e15d16c5b02/external/build_bazel_rules_nodejs/internal/npm_install/npm_install.bzl", line 107
		_create_build_file(repository_ctx, node)
	File "/home/doug/.cache/bazel/_bazel_doug/b05db80e2dc3f398f49b0e15d16c5b02/external/build_bazel_rules_nodejs/internal/npm_install/npm_install.bzl", line 32, in _create_build_file
		fail(("node failed: \nSTDOUT:\n%s\nST...)))
node failed: 
STDOUT:

STDERR:
/home/doug/.cache/bazel/_bazel_doug/b05db80e2dc3f398f49b0e15d16c5b02/external/ts_protoc_gen_deps/generate_build_file.js:283
            throw new Error(`Could not find ${depType} '${targetDep}' of '${dep._dir}'`)
            ^

Error: Could not find peer dependency 'karma' of 'karma-jasmine'
    at Object.keys.map.targetDep (/home/doug/.cache/bazel/_bazel_doug/b05db80e2dc3f398f49b0e15d16c5b02/external/ts_protoc_gen_deps/generate_build_file.js:283:19)
    at Array.map (<anonymous>)
    at findDeps (/home/doug/.cache/bazel/_bazel_doug/b05db80e2dc3f398f49b0e15d16c5b02/external/ts_protoc_gen_deps/generate_build_file.js:267:10)
    at flattenDependencies (/home/doug/.cache/bazel/_bazel_doug/b05db80e2dc3f398f49b0e15d16c5b02/external/ts_protoc_gen_deps/generate_build_file.js:301:3)
    at pkgs.forEach.pkg (/home/doug/.cache/bazel/_bazel_doug/b05db80e2dc3f398f49b0e15d16c5b02/external/ts_protoc_gen_deps/generate_build_file.js:81:23)
    at Array.forEach (<anonymous>)
    at main (/home/doug/.cache/bazel/_bazel_doug/b05db80e2dc3f398f49b0e15d16c5b02/external/ts_protoc_gen_deps/generate_build_file.js:81:8)
    at Object.<anonymous> (/home/doug/.cache/bazel/_bazel_doug/b05db80e2dc3f398f49b0e15d16c5b02/external/ts_protoc_gen_deps/generate_build_file.js:54:3)
    at Module._compile (module.js:635:30)
    at Object.Module._extensions..js (module.js:646:10)
 and referenced by '@ts_protoc_gen//bin:protoc-gen-ts_bin'
ERROR: Analysis of target '//test/bazel/proto:pizza_service_ts_proto' failed; build aborted: no such package '@ts_protoc_gen_deps//': Traceback (most recent call last):
	File "/home/doug/.cache/bazel/_bazel_doug/b05db80e2dc3f398f49b0e15d16c5b02/external/build_bazel_rules_nodejs/internal/npm_install/npm_install.bzl", line 107
		_create_build_file(repository_ctx, node)
	File "/home/doug/.cache/bazel/_bazel_doug/b05db80e2dc3f398f49b0e15d16c5b02/external/build_bazel_rules_nodejs/internal/npm_install/npm_install.bzl", line 32, in _create_build_file
		fail(("node failed: \nSTDOUT:\n%s\nST...)))
node failed: 
STDOUT:

STDERR:
/home/doug/.cache/bazel/_bazel_doug/b05db80e2dc3f398f49b0e15d16c5b02/external/ts_protoc_gen_deps/generate_build_file.js:283
            throw new Error(`Could not find ${depType} '${targetDep}' of '${dep._dir}'`)
            ^

Error: Could not find peer dependency 'karma' of 'karma-jasmine'
    at Object.keys.map.targetDep (/home/doug/.cache/bazel/_bazel_doug/b05db80e2dc3f398f49b0e15d16c5b02/external/ts_protoc_gen_deps/generate_build_file.js:283:19)
    at Array.map (<anonymous>)
    at findDeps (/home/doug/.cache/bazel/_bazel_doug/b05db80e2dc3f398f49b0e15d16c5b02/external/ts_protoc_gen_deps/generate_build_file.js:267:10)
    at flattenDependencies (/home/doug/.cache/bazel/_bazel_doug/b05db80e2dc3f398f49b0e15d16c5b02/external/ts_protoc_gen_deps/generate_build_file.js:301:3)
    at pkgs.forEach.pkg (/home/doug/.cache/bazel/_bazel_doug/b05db80e2dc3f398f49b0e15d16c5b02/external/ts_protoc_gen_deps/generate_build_file.js:81:23)
    at Array.forEach (<anonymous>)
    at main (/home/doug/.cache/bazel/_bazel_doug/b05db80e2dc3f398f49b0e15d16c5b02/external/ts_protoc_gen_deps/generate_build_file.js:81:8)
    at Object.<anonymous> (/home/doug/.cache/bazel/_bazel_doug/b05db80e2dc3f398f49b0e15d16c5b02/external/ts_protoc_gen_deps/generate_build_file.js:54:3)
    at Module._compile (module.js:635:30)
    at Object.Module._extensions..js (module.js:646:10)
INFO: Elapsed time: 23.447s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (470 packages loaded, 20900 targets configured)
FAILED: Build did NOT complete successfully (470 packages loaded, 20900 targets configured)


# Edit ./defs.bzl, at the bottom of the file inside typescript_proto_dependencies()
# Comment out the npm_install and uncomment yarn_install
bazel test //test/bazel/...:all

# Tests pass

Thanks for your help!

[gregmagolan]

Thanks for the easy reproduction.

I looked into this and it looks like npm is laying out the npm modules incorrectly. karma ends up in node_modules/@bazel/karma/node_modules/karma as it is a dep of @bazel/karma but karma-jasmine ends up at node_modules/karma-jasmine and there is no nested node_modules/karma-jasmine/node_modules/karma or root node_modules/karma.

It seems like karma should be placed at the root node_modules/karma and not at node_modules/@bazel/karma/node_modules/karma since it is a peer dep of karma-jasmine and both are deps of @bazel/karma.

I'll think about this some more.

[gregmagolan]

Some more data. npm install does warn that the peer dep is missing as well:

npm WARN [email protected] requires a peer of karma@* but none is installed. You must install peer dependencies yourself.
npm WARN [email protected] requires a peer of karma@>=0.9 but none is installed. You must install peer dependencies yourself.

even tho v1.7.1 it is installed at node_modules/@bazel/karma/node_modules/karma.

I wonder if it has to do with @bazel/karma depending on a github version "karma": "github:alexeagle/karma#fa1a84ac881485b5657cb669e9b4e5da77b79f0a", and not a direct npm release?

[gregmagolan]

Looks like an existing and known issue with npm npm/npm#19877 (comment).

[Dig-Doug]

Thanks for looking into this. From your explanation it doesn't seem like there's an issue in rules_nodejs, so I'm closing this issue.

[mikelnrd]

Update:

It looks like npm are working on the installation of peer dependencies:

https://blog.npmjs.org/post/617484925547986944/npm-v7-series-introduction
https://github.com/npm/rfcs/blob/latest/accepted/0025-install-peer-deps.md