Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bazel: update to 3.5.0 #97497

Closed
epigramengineer opened this issue Sep 9, 2020 · 12 comments
Closed

bazel: update to 3.5.0 #97497

epigramengineer opened this issue Sep 9, 2020 · 12 comments
Labels
0.kind: packaging request Request for a new package to be added

Comments

@epigramengineer
Copy link

When attempting to follow the instructions in pkgs/development/tools/build-managers/bazel/bazel_3/default.nix I ran into an issue with eval $(nix-build -A bazel.updater):

❯ eval $(nix-build -A bazel.updater)
these derivations will be built:
  /nix/store/m8275a3529g9252jd4h66rdawp3lmw3c-bazel-deps.json.drv
  /nix/store/3yf5bgp4jr1h98m0gajn0zmjfzw07683-update-bazel-deps.sh.drv
building '/nix/store/m8275a3529g9252jd4h66rdawp3lmw3c-bazel-deps.json.drv'...
Archive:  /nix/store/z6whc45g0yhcgf8595whhk47ww7f8inq-bazel-3.5.0-dist.zip
  inflating: WORKSPACE
Traceback (most recent call last):
  File "/nix/store/d5h0g8zr573ksxgs31vg3fi7sdbn1pp1-update-srcDeps.py", line 54, in <module>
    exec(f.read())
  File "<string>", line 1050, in <module>
NameError: name 'grpc_deps' is not defined

I'm unsure how to fix that myself; apologies.

See: https://github.com/bazelbuild/bazel/releases

@epigramengineer epigramengineer added the 0.kind: packaging request Request for a new package to be added label Sep 9, 2020
@doronbehar
Copy link
Contributor

cc @mboes

@dmivankov
Copy link
Contributor

`Does 3.4.1 work? It seems bazel is making some breaking changes among 3.x versions, like bazelbuild/rules_docker#1620

@dmivankov
Copy link
Contributor

@dmivankov
Copy link
Contributor

dmivankov commented Oct 6, 2020

Using bazel itself (so, older but recent enough version or one from a binary distribution)

bazel query 'kind(http_archive, //external:all) + kind(http_file, //external:all) + kind(distdir_tar, //external:all) + kind(git_repository, //external:all)' --output build

will produce the list of dependencies without fancy language features

Notes:

  • not sure whether we need to handle git_repository's
  • there's also xml output option, but build format looks easier to handle
  • bazel query will download multiple GBs the first time, including j2obc, jdk and other things

@dmivankov
Copy link
Contributor

dmivankov commented Oct 6, 2020

Further complications include

@dmivankov
Copy link
Contributor

trying to fix non-checksummed dependencies on bazel side bazelbuild/bazel#12226

@dmivankov
Copy link
Contributor

bazel 3.6.1 should be compatible, but not released yet

@divanorama
Copy link
Contributor

Actually 3.7.0 is still incompatible, waiting for 3.8.0 (November)
Alternatively, I managed to compile it from recent git sources (git sources don't include some generated bootstrap sources, so we need to change compilation process a little), will publish a PR later.

@divanorama
Copy link
Contributor

3.7.0+ / 3.9999 / bazel_git PR #101292

@lambdadog
Copy link
Contributor

Do we have any reasonable solution to just using a bazel version higher than 3.3.1 even if it's not included into nixpkgs? I need bazel>=3.4.0 for the dependency of a project I'm working on and I'm trying to riddle out how I can manage that using nix.

@divanorama
Copy link
Contributor

divanorama commented Nov 4, 2020

I have WIP branch for bazel_git with 3.7.0+ https://github.com/divanorama/nixpkgs/commits/bazel_git
bazel 3.8.0 should only happen in ~1month, we should be able to use that version in nixpkgs then.
It may be possible to make 3.4..3.7 compilable with nix, hopefully it amounts to just manually pinning boringssl and skylib dependencies, but still some work to do.
nixpkgs.bazelisk isn't functional
There also are some docker images, which may be an option
https://console.cloud.google.com/gcr/images/cloud-marketplace/GLOBAL/google/bazel?gcrImageListsize=30
or
https://github.com/GoogleCloudPlatform/cloud-builders/tree/master/bazel

divanorama added a commit to divanorama/nixpkgs that referenced this issue Jan 27, 2021
Bazel 4 is going to be a long term support release.

Latest version in NixPkgs so far was 3.3.1
There's a need for more recent version
NixOS#97497

All versions from 3.5.0 to 3.7.1 had some reproducibility issues
as noted in issue above, but there also seems to be
a working PR for 3.7.1 now at
NixOS#105439

Notable changes from bazel_3 setup:
- put python to default bash path

  For autodetecting python toolchain
  with strict action_env on and without this change
  bazel would fail to autodetect host python.

  There are some repos that define hermetic python
  toolchains, but they aren't easy to use yet. Also
  telling python paths to bazel isn't a 1-liner it
  seems:
  - action_env=PATH would affect cache
  - declaring toolchain via BUILD&WORKSPACE files
    is not per-user but more like per-repo and
    affects cache too

  Using python from nixpkgs shouldn't be too bad
  in the lack of simpler hermetic python toolchain
  options

- bazel_4.updater is bazel on `bazel query` to support
  new constructs in WORKSPACE (load of vars, transitive
  load etc). This is more robust but requires bazel
  to run the updater, using bazel_3 for now. This is
  only needed to bump package version, doesn't introduce
  bazel_4 build dependency on bazel_3

https://blog.bazel.build/2020/11/10/bazel-4.0-announce.html
https://blog.bazel.build/2020/11/10/long-term-support-release.html
bazelbuild/bazel#12455
https://github.com/bazelbuild/bazel/releases/tag/4.0.0
https://blog.bazel.build/2021/01/19/bazel-4-0.html
divanorama added a commit to divanorama/nixpkgs that referenced this issue Feb 4, 2021
Bazel 4 is going to be a long term support release.

Latest version in NixPkgs so far was 3.3.1
There's a need for more recent version
NixOS#97497

All versions from 3.5.0 to 3.7.1 had some reproducibility issues
as noted in issue above, but there also seems to be
a working PR for 3.7.1 now at
NixOS#105439

Notable changes from bazel_3 setup:
- put python to default bash path

  For autodetecting python toolchain
  with strict action_env on and without this change
  bazel would fail to autodetect host python.

  There are some repos that define hermetic python
  toolchains, but they aren't easy to use yet. Also
  telling python paths to bazel isn't a 1-liner it
  seems:
  - action_env=PATH would affect cache
  - declaring toolchain via BUILD&WORKSPACE files
    is not per-user but more like per-repo and
    affects cache too

  Using python from nixpkgs shouldn't be too bad
  in the lack of simpler hermetic python toolchain
  options

- bazel_4.updater is bazel on `bazel query` to support
  new constructs in WORKSPACE (load of vars, transitive
  load etc). This is more robust but requires bazel
  to run the updater, using bazel_3 for now. This is
  only needed to bump package version, doesn't introduce
  bazel_4 build dependency on bazel_3

https://blog.bazel.build/2020/11/10/bazel-4.0-announce.html
https://blog.bazel.build/2020/11/10/long-term-support-release.html
bazelbuild/bazel#12455
https://github.com/bazelbuild/bazel/releases/tag/4.0.0
https://blog.bazel.build/2021/01/19/bazel-4-0.html
@divanorama
Copy link
Contributor

bazel_3 is now at 3.7.2 and bazel_4 is at 4.0.0 (bump to 4.1.0 pending), so I think this issue can be closed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
0.kind: packaging request Request for a new package to be added
Projects
None yet
Development

No branches or pull requests

6 participants