Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

//internal/pkg_npm/test/linking:test fails on Windows with remote caching enabled #1781

Closed
meteorcloudy opened this issue Apr 3, 2020 · 2 comments · Fixed by #1805
Closed
Labels

Comments

@meteorcloudy
Copy link
Collaborator

🐞 bug report

Is this a regression?

No, the test is added recently

Description

//internal/pkg_npm/test/linking:test will fail in a second run with remote caching enabled.

https://buildkite.com/bazel/bazel-at-head-plus-downstream/builds/1446#b6409cc9-0a26-4ddd-8828-74d639131091

🔬 Minimal Reproduction

bazel test //internal/pkg_npm/test/linking:test
bazel clean --expunge & rm -rf node_modules
bazel test //internal/pkg_npm/test/linking:test

🔥 Exception or Error


Executing tests from //internal/pkg_npm/test/linking:test
-----------------------------------------------------------------------------
internal/modules/cjs/loader.js:797
    throw err;
    ^

Error: Cannot find module '@scoped/bar'
Require stack:
- C:\b\s7rxdkmr\execroot\build_bazel_rules_nodejs\bazel-out\x64_windows-fastbuild\bin\internal\pkg_npm\test\linking\main.js
    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:794:15)
    at Function.Module._load (internal/modules/cjs/loader.js:687:27)
    at Module.require (internal/modules/cjs/loader.js:849:19)
    at require (internal/modules/cjs/helpers.js:74:18)
    at Object. (C:\b\s7rxdkmr\execroot\build_bazel_rules_nodejs\bazel-out\x64_windows-fastbuild\bin\internal\pkg_npm\test\linking\main.js:3:13)
    at Module._compile (internal/modules/cjs/loader.js:956:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:973:10)
    at Module.load (internal/modules/cjs/loader.js:812:32)
    at Function.Module._load (internal/modules/cjs/loader.js:724:14)
    at Function.Module.runMain (internal/modules/cjs/loader.js:1025:10) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [
    'C:\\b\\s7rxdkmr\\execroot\\build_bazel_rules_nodejs\\bazel-out\\x64_windows-fastbuild\\bin\\internal\\pkg_npm\\test\\linking\\main.js'
  ]
}

🌍 Your Environment

Operating System:

  
Windows
  

Output of bazel version:

  
3.0.0rc2
  

Rules_nodejs version:

(Please check that you have matching versions between WORKSPACE file and @bazel/* npm packages.)

  
HEAD
  
@meteorcloudy
Copy link
Collaborator Author

/cc @alexeagle @gregmagolan

@gregmagolan
Copy link
Collaborator

gregmagolan commented Apr 5, 2020

Thanks for the repo steps @meteorcloudy. I am able to reproduce on Windows. Doesn't happen on OSX. Haven't tried linux.

gregmagolan added a commit to gregmagolan/rules_nodejs that referenced this issue Apr 5, 2020
gregmagolan added a commit to gregmagolan/rules_nodejs that referenced this issue Apr 5, 2020
This is a partial fix for linker issue on Windows when all build actions are cached bazel-contrib#1781. This fixes incorrectly symlinks but it does not actually address the underlying issue of programs (tests) running outside of the sandbox and directly out of the execroot on Windows (either out of execroot sources or execroot bazel-out).

If all build actions are cached then the linker only runs for the test that is executed. If there are no third-part npm deps then it makes a node_modules folder in the runfiles for that test and puts its symlinks in there. The problem is that when the test runs, the .js spec is in execroot bazel-out and it doesn't see the node_modules created in the runfiles folder. Since no build actions have run that may have created the symlinks needed for the test in the execroot node_modules, the symlinks that it needs do not exist.
gregmagolan added a commit to gregmagolan/rules_nodejs that referenced this issue Apr 6, 2020
This is a partial fix for linker issue on Windows when all build actions are cached bazel-contrib#1781. This fixes incorrectly symlinks but it does not actually address the underlying issue of programs (tests) running outside of the sandbox and directly out of the execroot on Windows (either out of execroot sources or execroot bazel-out).

If all build actions are cached then the linker only runs for the test that is executed. If there are no third-part npm deps then it makes a node_modules folder in the runfiles for that test and puts its symlinks in there. The problem is that when the test runs, the .js spec is in execroot bazel-out and it doesn't see the node_modules created in the runfiles folder. Since no build actions have run that may have created the symlinks needed for the test in the execroot node_modules, the symlinks that it needs do not exist.
gregmagolan added a commit to gregmagolan/rules_nodejs that referenced this issue Apr 6, 2020
This is a partial fix for linker issue on Windows when all build actions are cached bazel-contrib#1781. This fixes incorrectly symlinks but it does not actually address the underlying issue of programs (tests) running outside of the sandbox and directly out of the execroot on Windows (either out of execroot sources or execroot bazel-out).

If all build actions are cached then the linker only runs for the test that is executed. If there are no third-part npm deps then it makes a node_modules folder in the runfiles for that test and puts its symlinks in there. The problem is that when the test runs, the .js spec is in execroot bazel-out and it doesn't see the node_modules created in the runfiles folder. Since no build actions have run that may have created the symlinks needed for the test in the execroot node_modules, the symlinks that it needs do not exist.
gregmagolan added a commit to gregmagolan/rules_nodejs that referenced this issue Apr 6, 2020
Fix 1 of 2 for linker issue on Windows when all build actions are cached bazel-contrib#1781. This fixes incorrectly symlinks but it does not actually address the underlying issue of programs (tests) running outside of the sandbox and directly out of the execroot on Windows (either out of execroot sources or execroot bazel-out).

If all build actions are cached then the linker only runs for the test that is executed. If there are no third-part npm deps then it makes a node_modules folder in the runfiles for that test and puts its symlinks in there. The problem is that when the test runs, the .js spec is in execroot bazel-out and it doesn't see the node_modules created in the runfiles folder. Since no build actions have run that may have created the symlinks needed for the test in the execroot node_modules, the symlinks that it needs do not exist.

The 2nd fix (still coming) seems to be to always symlink into the execroot which is below the runfiles folder. Some mental gymnastics needed to figure out if this is correct when in the sandbox as well and if there are third-party deps or not. I'll investigate.
gregmagolan added a commit to gregmagolan/rules_nodejs that referenced this issue Apr 7, 2020
Fix 1 of 2 for linker issue on Windows when all build actions are cached bazel-contrib#1781. This fixes incorrectly symlinks but it does not actually address the underlying issue of programs (tests) running outside of the sandbox and directly out of the execroot on Windows (either out of execroot sources or execroot bazel-out).

If all build actions are cached then the linker only runs for the test that is executed. If there are no third-part npm deps then it makes a node_modules folder in the runfiles for that test and puts its symlinks in there. The problem is that when the test runs, the .js spec is in execroot bazel-out and it doesn't see the node_modules created in the runfiles folder. Since no build actions have run that may have created the symlinks needed for the test in the execroot node_modules, the symlinks that it needs do not exist.

The 2nd fix (still coming) seems to be to always symlink into the execroot which is below the runfiles folder. Some mental gymnastics needed to figure out if this is correct when in the sandbox as well and if there are third-party deps or not. I'll investigate.
gregmagolan added a commit to gregmagolan/rules_nodejs that referenced this issue Apr 8, 2020
Fix 1 of 2 for linker issue on Windows when all build actions are cached bazel-contrib#1781. This fixes incorrectly symlinks but it does not actually address the underlying issue of programs (tests) running outside of the sandbox and directly out of the execroot on Windows (either out of execroot sources or execroot bazel-out).

If all build actions are cached then the linker only runs for the test that is executed. If there are no third-part npm deps then it makes a node_modules folder in the runfiles for that test and puts its symlinks in there. The problem is that when the test runs, the .js spec is in execroot bazel-out and it doesn't see the node_modules created in the runfiles folder. Since no build actions have run that may have created the symlinks needed for the test in the execroot node_modules, the symlinks that it needs do not exist.

The 2nd fix (still coming) seems to be to always symlink into the execroot which is below the runfiles folder. Some mental gymnastics needed to figure out if this is correct when in the sandbox as well and if there are third-party deps or not. I'll investigate.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
3 participants