Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

document convenient way to lay out all files in bazel-out #1771

Closed
mrmeku opened this issue Apr 1, 2020 · 6 comments
Closed

document convenient way to lay out all files in bazel-out #1771

mrmeku opened this issue Apr 1, 2020 · 6 comments

Comments

@mrmeku
Copy link
Collaborator

mrmeku commented Apr 1, 2020

🐞 bug report

Affected Rule

The issue is caused by the rule: ts_project

Is this a regression?

No

Description

Consider what happens when you try to bundle a index.js emited by ts_project.

index.js is placed in execroot/workspace/bazel-out/k8-fastbuild/bin/index.js and tries to do a relative import of a non-generated file something like ./styles.css (other common scenarios would be a local .js, or .json file)

styles.css is placed in execroot/workspace/styles.css

So the relative import fails since the bundler doesn't know how to resolve things living bazel-out vs not in bazel-out

In order to fix this sort of thing we'll either need to layout the directory structure differently or provide some documentation on ways to configure bundlers to resolve these imports.

A clear and concise description of the problem...

🔬 Minimal Reproduction

#1770

🔥 Exception or Error


ERROR in ./bazel-out/k8-fastbuild/bin/index.js
Module not found: Error: Can't resolve './styles.css' in '.../execroot/react_webpack/bazel-out/k8-fastbuild/bin'
 @ ./bazel-out/k8-fastbuild/bin/index.js 5:13-36

🌍 Your Environment

Output of bazel version:

  
Build label: 2.2.0
Build target: bazel-out/k8-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Tue Mar 3 09:26:12 2020 (1583227572)
Build timestamp: 1583227572
Build timestamp as int: 1583227572
  

Rules_nodejs version:

(Please check that you have matching versions between WORKSPACE file and @bazel/* npm packages.)

  
http_archive(
    name = "build_bazel_rules_nodejs",
    sha256 = "d0c4bb8b902c1658f42eb5563809c70a06e46015d64057d25560b0eb4bdc9007",
    urls = ["https://github.com/bazelbuild/rules_nodejs/releases/download/1.5.0/rules_nodejs-1.5.0.tar.gz"],
)
  
@alexeagle
Copy link
Collaborator

js_library has a built-in copy_to_bin action on InputArtifacts so you get relative imports together again. That's how @gregmagolan solved this in the linker PR.

@alexeagle alexeagle changed the title ts_project emited files unable to resolve files not in bazel-out document convenient way to lay out all files in bazel-out Apr 1, 2020
@alexeagle
Copy link
Collaborator

certainly not a ts_project bug, it's doing the right thing with its inputs and outputs.
The question is how the user should express that they want other sources of the bundler to be copied to the output tree. You could manually add copy_to_bin rules for it, but maybe we should make the js_library rule part of the public API, as it now does this https://github.com/bazelbuild/rules_nodejs/blob/1.5.0/internal/js_library/js_library.bzl#L57-L63

@alexeagle
Copy link
Collaborator

I'm doing some work right now with cra/react-scripts and copy_to_bin works pretty well, just use it as a drop-in replacement for filegroup and you get a self-contained output directory

@alexeagle
Copy link
Collaborator

We should add a bit of docs somewhere for 2.0 to explain the current situation.
IF js_library becomes public API then it changes what we tell you to do...

@alexeagle
Copy link
Collaborator

js_library not going public in time for 2.0, so I think this should slip too.

@alexeagle alexeagle removed this from the 2.0 milestone Jun 18, 2020
@alexeagle
Copy link
Collaborator

Documenting this now in the js_library rule

alexeagle pushed a commit to alexeagle/rules_nodejs that referenced this issue Sep 10, 2020
alexeagle pushed a commit to alexeagle/rules_nodejs that referenced this issue Sep 10, 2020
alexeagle pushed a commit to alexeagle/rules_nodejs that referenced this issue Sep 10, 2020
lherman-cs pushed a commit to lherman-cs/rules_nodejs that referenced this issue Jan 15, 2021
fix: remove duplicate Importing

chore: upgrade rollup to ^2.3.0

fix: coverage (bazel-contrib#2100)

* test: add coverage e2e test

* fix(builtin): fix coverage lcov merger script jumping out of sandbox and not being able to find c8

* test: don't test coverage on Windows

docs: use @npm//@bazel for example load sites

fix(cypress): allow for async cypress plugins

cypress_repository now fails if cypress verify fails
set the HOME env variable since cypress writes files to it
remove unused includeScreenshots / includeVideos variables
browserify files are no longer included in test output files
screenshots/videos are stored directory in TEST_UNDECLARED_OUTPUTS_DIR

docs: update docs for release

chore(release): 2.0.2

chore: update lock files for release

feat(example): add targets in angular_bazel_architect for production serve and build

fix(examples): use ./ prefix on babel config file

Otherwise if it's in a subfolder, babel will think it's an npm package

chore: update dependency com_google_protobuf to v3.13.0 (bazel-contrib#2103)

refactor(rollup): use rollup loadConfigFile API

chore: update dependency bazel_toolchains to v3.4.1

build(deps): bump elliptic in /e2e/fine_grained_symlinks

Bumps [elliptic](https://github.com/indutny/elliptic) from 6.4.1 to 6.5.3.
- [Release notes](https://github.com/indutny/elliptic/releases)
- [Commits](indutny/elliptic@v6.4.1...v6.5.3)

Signed-off-by: dependabot[bot] <[email protected]>

fix(typescript): only expect .js outs for .tsx? srcs (bazel-contrib#2118)

Fixes bazel-contrib#2115

feat(builtin): new js_library rule (bazel-contrib#2109)

Removes `node_module_library` private API and renames it to `js_library`

the latter will be promoted to a public API in a followup change

fix(typescript): produce .d.ts as default output rather than empty (bazel-contrib#2117)

This lets you type-check a ts_project(emit_declaration_only=True) just by building it (its default outputs)

Fixes bazel-contrib#2116

added script to generate NodeJS versions for node_repositories and added most up-to-date versions (bazel-contrib#2126)

Co-authored-by: Matt Insler <[email protected]>

refactor: move node version list into its own file (bazel-contrib#2128)

Add a package.json script for updating it

docs: update docs for release

chore(release): 2.0.3

chore: update lock files for release

fix(typescript): add the tsBuildInfoFile option to ts_project (bazel-contrib#2138)

Like other options that affect tsc outputs, Bazel needs to know the value from the tsconfig if its set.
Add it to our validator so there's a buildozer command to sync the option value into the BUILD file.

Fixes bazel-contrib#2137

refactor(typescript): remove private _TsConfigInfo in ts_project (bazel-contrib#2139)

Instead use the same TsConfigInfo that ts_library uses.

Fixes bazel-contrib#1931

docs: refresh docsite

docs: minor cleanup of builtins page

don't include the defaults which are giant dictionaries as they render poorly

docs: fix spacing and link colours on rhs sidenav

Fix circleci status badges to pin to stable branch

CircleCI docs claim that they should use the default branch by default, but this icon is green while stable is red, so I think it's still reporting on master.

ci: all angular examples timeout=long

chore: update dependency bazel_toolchains to v3.4.2

chore: update circleci image to latest to pick up newer chrome

docs: add search to docsite using gcse

feat(typescript): generate tsconfig.json for ts_project (bazel-contrib#2130)

This is an experimental, opt-in feature where ts_project will generate a tsconfig.json file in place of
the user specifying one in the source directory.

I expect a long tail of compatibility bugs since any path appearing in this generated file needs to point from
bazel-out back to the source directory.

Fixes bazel-contrib#2058

docs: replace references to //packages/ with the @npm//@bazel/ equivalent (bazel-contrib#2154)

feat(builtin): accept any stamp vars in pkg_npm

NOTE: instead of replacing the replace_with_version with 0.0.0 in unstamped builds,
we now omit the replacement.

See bazel-contrib#1694

docs(rollup): add stamping and debug to rollup doc

Also move most of the content above the list of rules. It shows up better in the TOC since rollup_bundle
can be its own entry later in the page, and avoids the API doc table of attributes just appearing with no header.

docs: document the node_context_data attribute

It can be useful if you need to turn on/off stamping per-target.
`java_binary` gives a dedicated stamp attribute for this purpose

Fixes bazel-contrib#1693

chore: hide generated .html files in code review

feat(builtin): support for substitutions

fix: use golden_file_test instead

refactor: cleanup pkg_web stamping feature

chore: update dependency bazel_toolchains to v3.5.0 (bazel-contrib#2168)

Updated nodejs versions (14.9.0) (bazel-contrib#2169)

docs: update docs for release

chore(release): 2.1.0

chore: update lock files for release

chore: try github actions version of stale bot

The probot one never triggered despite being setup correctly AFAICT.
This bot seems better supported and I sow it working on bazel-watcher repo.

build(deps): bump http-proxy from 1.18.0 to 1.18.1

Bumps [http-proxy](https://github.com/http-party/node-http-proxy) from 1.18.0 to 1.18.1.
- [Release notes](https://github.com/http-party/node-http-proxy/releases)
- [Changelog](https://github.com/http-party/node-http-proxy/blob/master/CHANGELOG.md)
- [Commits](http-party/node-http-proxy@1.18.0...1.18.1)

Signed-off-by: dependabot[bot] <[email protected]>

build(deps): bump yargs-parser in /e2e/ts_devserver

Bumps [yargs-parser](https://github.com/yargs/yargs-parser) from 13.1.1 to 13.1.2.
- [Release notes](https://github.com/yargs/yargs-parser/releases)
- [Changelog](https://github.com/yargs/yargs-parser/blob/master/docs/CHANGELOG-full.md)
- [Commits](https://github.com/yargs/yargs-parser/commits)

Signed-off-by: dependabot[bot] <[email protected]>

ci: stamp our releases with a stable key

fix(builtin): don't set --preserve-symlinks-main by default (bazel-contrib#2176)

feat: add link_workspace_root to nodejs_binary, npm_package_bin, rollup_bundle, terser_minified, ts_project

Link the workspace root to the bin_dir to support absolute requires like 'my_wksp/path/to/file'.
If source files need to be required then they can be copied to the bin_dir with copy_to_bin.

fix(builtin): fix bazel coverage masking test failures

test(builtin): test failing test with bazel coverage

build(deps): bump http-proxy in /examples/protocol_buffers

Bumps [http-proxy](https://github.com/http-party/node-http-proxy) from 1.18.0 to 1.18.1.
- [Release notes](https://github.com/http-party/node-http-proxy/releases)
- [Changelog](https://github.com/http-party/node-http-proxy/blob/master/CHANGELOG.md)
- [Commits](http-party/node-http-proxy@1.18.0...1.18.1)

Signed-off-by: dependabot[bot] <[email protected]>

build(deps): bump http-proxy in /examples/web_testing

Bumps [http-proxy](https://github.com/http-party/node-http-proxy) from 1.18.0 to 1.18.1.
- [Release notes](https://github.com/http-party/node-http-proxy/releases)
- [Changelog](https://github.com/http-party/node-http-proxy/blob/master/CHANGELOG.md)
- [Commits](http-party/node-http-proxy@1.18.0...1.18.1)

Signed-off-by: dependabot[bot] <[email protected]>

test(rollup): fix test failing due to missing dep

fix(rollup): allow config files to override default onwarn method

Fixes bazel-contrib#2084

feat: link_workspace_root not needed in terser_minified

docs: add spica to adopter organization list (bazel-contrib#2184)

* Add spica to adopter organization list (bazel-contrib#2183)
* docs: update the 'send a PR' link to point to the stable branch

Co-authored-by: Sahin Yort <[email protected]>

feat: promote js_library to public API

Fixes bazel-contrib#149
Fixes bazel-contrib#1771

docs: update docs for release

chore(release): 2.2.0

chore: update lock files for release

Update readme (bazel-contrib#2190)

Fix missing `}` and indentation

feat: add strict_visibility to npm_install / yarn_install rules (bazel-contrib#2193)

With strict_visibility enabled (currently defaults false), only dependencies within the given `package.json` file are given public visibility.

All transitive dependencies are given limited visibility, enforcing that all direct dependencies are listed in the `package.json` file.

closes bazel-contrib#2110

fix: don't glob yarn or node files when using vendored_node or vendored_yarn

- Fixes a visibility issue with rollup-worker

fix(examples): prevent ibazel EOF

Fixes bazel-contrib#2143

docs(karma): document how to use non-headless Chrome (bazel-contrib#2202)

Fixes bazel-contrib#1881

docs(labs): produce stardoc API for labs

Fixes bazel-contrib#2195

fix(karma): allow custom browsers to specify args (fixes bazel-contrib#595)

Today, the args as specified in the various manifests of the rules_webtesting
browsers never make it into the generated karma.conf.js. This results in these
arguments never being used when launching Chrome, preventing customization of
browsers such as window size, enabling remote debugging, or other flag-based
options. This PR fixes this by reading those arguments from the manifest, and
adding them to the browsers list in the generated karma.conf.js.

Also, this PR includes a change to generated_file_test to allow a golden file
to represent a substring of the generated output.

Also Also: This PR includes a golden file test that verified that the generated
karma.conf.js does read in the specified value.

Furthermore, the effect of this can be verified manually via:
```
VERBOSE_LOGS=1 bazel run packages/karma/test/karma:testing_custom_chrome
```
Note the appearance of the additional flags in the new debug output.

fix(builtin): js_library: correctly propagate DeclarationInfos

Transitive typings are added to typings_depsets, but we were checking
the typings array instead.

Signed-off-by: Duarte Nunes <[email protected]>

feat: update nodejs versions (bazel-contrib#2207)

include 10.22.1, 12.18.4, 14.10.0, 14.10.1, 14.11.0 and 14.12.0 in node_versions

Added s390x support

docs: update docs for release

chore(release): 2.2.1

chore: update lock files for release

ci: tell Angular CLI not to update webdriver

It can get ahead of the Chrome version on our CI machine

ci: replace --webdriver-update=false with --no-webdriverUpdate

Architect CLI uses minimist under the hood to parse arguments. Minimist will not convert boolean like values unless they are known options or the `{boolean: true}` option is configured (Which is not in Architect CLI).

feat(karma): use Trusted Types policy when loading scripts for Karma

When the Karma plugin is used in a testing environment that enforces
Trusted Types, its loadFile functionality currently fails due to a
Trusted Types violation when assigning to script.textContent. This makes
it impossible to use the plugin with integration tests that ensure an
application is compatible with Trusted Types.

This is fixed by creating a Trusted Types policy specifically for the
Karma plugin, and use it to promote any loaded scripts to a TrustedScript
before assigning to script.textContent. This is done in a way that is
backwards compatible:
- The policy is `null` in browsers that don't yet support Trusted Types,
  in which case the original script string is used as before.
- When Trusted Types are supported in the browser but not enforced by
  the application, the browser treats the TrustedScript as if it were a
  string when it is assigned to script.textContent.

chore: update dependency bazel_toolchains to v3.6.0

docs: remove HTML tables so docs can be authored in markdown without need for html escaping

docs(rollup): improve clarity

feat(example): add full pwa support

feat(example): service worker update handling

fix(exmaple): add docstring to ngsw_config rule

fix(example): remove server side compression

fix(example): remove index.html from prodapp srcs

fix(example): remove compression dependencies

fix(builtin): js_library supports --output_groups=types

If providing DeclarationInfo we should also give this way for filegroup/bazel CLI to select them

chore: docgen

docs: update docs for release

chore(release): 2.2.2

chore: update lock files for release

fix(builtin): give a longer timeout for _create_build_files

A user reports that their build file generation takes longer than 600 seconds.
It's probably a bug that we are so slow, but let's address that separately (maybe in our work for npm v7)

Also note we could have made this timeout user-configurable, but I think the extra API surface isn't worth the benefit.

Fixes bazel-contrib#2231

fix(typescript): don't include _valid_options marker file in outs

Tested by building //packages/angular:npm_package and don't get an extra .d.ts file in the package.
Also tested the negative case by making an invalid option and check that it's still reported.

Fixes bazel-contrib#2078

chore: allow node versions <= 14.14.0

chore: update rules_docker to 0.14.4

docs: minor update for readability and formatting (bazel-contrib#2243)

* docs: minor update for readibility and formating

Fix small typo in changelog

chore(docs): move ts_library docs into ts_library

Also recommend ts_project for new uses

chore(karma): remove dependency on tmp

feat(typescript): add allow_js support to ts_project

fix(exmaples/nestjs): add module_name field in ts_library

docs: fix a typo of angular-architect example (bazel-contrib#2272)

chore: bump days before closing stale issues to 90

Fix select with conditons on darwin.

Bazel is changing how @bazel_tools/conditions:darwin is implemented.
With old implementation based on flags, it was ok to have darwin and
darwin_x86_64 in a select. This was based on flag --cpu=darwin and
--cpu=dawin_x86_64.

New implementation is based on constraints, where "darwin" means OS (and
any CPU), while "darwin_x86_64" mean only specific CPU. As such they
cannot be used in the same select, because the selection would be
ambiguous.

build: add 'fix-windows' tag due to windows flaky tests

Will revisit the linker and test, but this is blocking PRs from being merged

feat(typescript): worker mode for ts_project (bazel-contrib#2136)

* feat(typescript): worker mode for ts_project

* chore: cleanup and declare protobufjs dependency

* fix: do not pass --watch for non-worker mode

* fix: do not test worker on windows

* fix: log on standalone failures

* chore: docs improvements

Co-authored-by: Alex Eagle <[email protected]>
Co-authored-by: Dan Muller <[email protected]>

feat(node_repositories): Added auth option for downloading nodejs and yarn

fix: npm_package.pack should work in windows os

feat(examples): adds example for running jest with typescript (bazel-contrib#2245)

fix(typescript): specify rootDir as absolute path

Also allow non-ts files across src/bin dir, since they are not emitted we don't need the rootdir
calculation to take them into account. This lets the Angular compiler accept .ts sources from
source dir even if the css inputs are generated.

refactor: ts_project cleanup

feat(cypress): remove browiserify preprocessor

docs: update docs for release

chore(release): 2.3.0

chore: update lock files for release

fix: npm_package.pack on Windows should not generate undefined.tgz

chore: update nodejs versions (bazel-contrib#2284)

perf(cypress): pack cypress runfiles into a single tar

chore: add GitHub Workflow to update NodeJS versions every night

chore: fix a link

Current link returns 404, it seems that the url was somehow changed to .html from .md.

chore: fix the link again

haha

fix(builtin): make linker deterministic when resolving from manifest & fix link_workspace_root with no runfiles

chore: re-lock jest dependencies

It was using a non-standard repository, which had downtime this morning

chore(release): 2.3.1

chore: update lock files for release

chore: update dependency com_google_protobuf to v3.14.0

chore: update dependency bazel_toolchains to v3.7.0

fix launcher script terminating before child terminates on SIGTERM

chore: update dependency bazel_toolchains to v3.7.1

fix(examples): fix jest example on windows

Fixes bazel-contrib#1454

fix(builtin): give better error when linker runs on Node <10

Example how this looks now:

```
[20 / 21] Bundling JavaScript bundle.js [parcel]; 4s remote-cache, darwin-sandbox
ERROR: /private/var/folders/r7/2kp53v7n091gcz93xlt7wtd80000gn/T/tmp-48948DHy4HrXTwhRy/BUILD.bazel:4:1: Bundling JavaScript bundle.js [parcel] failed (Exit 1) parcel.sh failed: error executing command bazel-out/host/bin/external/npm/parcel-bundler/bin/parcel.sh build foo.js --out-dir bazel-out/darwin-fastbuild/bin --out-file bundle.js ... (remaining 2 argument(s) skipped)

Use --sandbox_debug to see verbose messages from the sandbox
ERROR: rules_nodejs linker requires Node v10 or greater, but is running on 8.11.2
Note that earlier Node versions are no longer in long-term-support, see
https://nodejs.org/en/about/releases/
INFO: Elapsed time: 40.360s, Critical Path: 12.97s
INFO: 0 processes.
FAILED: Build did NOT complete successfully
//:test                                                               NO STATUS
```

Fixes bazel-contrib#2304

docs(typescript): link some user-contributed doc (bazel-contrib#2322)

Fixes bazel-contrib#2298

chore: allow node versions >=12 <=14 (bazel-contrib#2332)

chore: allow node versions >=12 <=14

feat: add esbuild package

chore: add module_mapping support via the aspect instead of manual implementing it

chore: sqash

feat: update to latest esbuild, add additional flags when not minifying

fix: consume both JSEcmaScriptModuleInfo and JSModuleInfo to support js_library

fix: add missing bzl libraries and publish helpers

test: use runfiles helper in test

fix: use exe for Windows
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants