bazel: 3.3.1 -> 3.7.2 #105439
Merged
bazel: 3.3.1 -> 3.7.2 #105439
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Fails to build on darwin: |
|
@SuperSandro2000 That seems to be the same error you got here: #102230 (comment) for libretro.gambatte Are you sure your mac is still working properly? (just asking since this error seems to be really rare, and you hit it twice already -- without anyone else seeing it... ❓ ) Seems like libmodplug broke in eval 1628634, when the changes for big sur where merged: #98541 (changes) Could this be a reason for those linker errors? |
kalbasit
reviewed
Nov 30, 2020
|
I get a build failure on macOS: |
|
@uri-canva seems to be the same error as #105439 (comment) I think if we find out why libmodplug fails with that error too since 21. november, that can be fixed for bazel also?! |
|
You're right, in fact that build failure exists regardless of this PR. |
uri-canva
approved these changes
Dec 1, 2020
Could be but I am not that deep into the big sur update to fully understand it. |
|
Since bazel 3.3.1 fails with the same error on hydra, should we just mark this package as broken on darwin? |
|
If we mark it as broken it will stop being built right? That means that even if it gets fixed by a more general big sur fix we won't notice. Since it was broken by a more general change, it's likely it will be fixed by a general change too. |
Right, but bazel is quite huge, compared to libmodplug for example. We could monitor libmodplug builds (I created #105537 to track this) and could re-enable bazel when libmodplug no longer fails. I guess this would reduce power consumption by some order... |
You can't build it on darwin at all.
Yeah but otherwise we get failures until it is fixed which is not great especially when other PRs see failing builds in their nixpkgs-review.
I wouldn't count on that. If you mark it as broken on darwin I would merge it. |
👌 Done. |
@SuperSandro2000 I've pushed a commit to address envoy too. |
|
This is a semi-automatic executed nixpkgs-review with nixpkgs-review-checks extension. It is checked by a human on a best effort basis and does not build all packages (e.g. lumo, tensorflow or pytorch). Result of 12 packages marked as broken and skipped:
1 package failed to build and are new build failure:
1 package built:
The following issues got detected with the above build packages. bazel: Please consider this feature to be alpha. A substituteInPlace with an unmatched pattern got detected: Please check the offending substituteInPlace for typos or changes in source. Do you have an idea about bazel_remote? If not we could mark it broken on darwin. |
|
Is this related? |
Seems to be the same problem as with bazel-watcher. (see #105669 (comment)) I tried to build bazel-watcher on top of this PR and it failed in the same way - cannot find a standard C++ header. This might indicate a serious problem with bazel on Darwin and probably needs attention separately. I'll mark it as broken for now... |
Let me have a look... Oh, yes of course. tensorflow 2 was moved / renamed to tensorflow. |
* update sha256 for gvisor, tensorflow{,WithCuda}, bazel-watcher, bazel-remote
and envoy
Use the same JDK for building bazel and for its runtime.
Effectively, the former `toolchain_hostjdk8` java toolchain has been deprecated
and should no longer be used (in newer bazel)[1]:
```
# Deprecated, do not use.
# It will be removed after migration to Java toolchain resolution.
default_java_toolchain(
name = "toolchain_hostjdk8",
...
)
```
[1]: https://github.com/bazelbuild/bazel/blob/4fc48680653a71aacbfd555436ba8f9a0742d3d9/tools/jdk/BUILD.tools#L384-L387
* use default stdenv (clang 7) * add no-arc.patch to make the xcode_locate tool compile without libarc-lite * remove the `-mmacosx-version-min=10.9` flag from the bootstrap compile script
Co-authored-by: Greg Roodt <[email protected]>
|
I have fixed up the "bazel: 3.7.1 -> 3.7.2" commit with changed hashes for tensorflow{,WithCuda}. @SuperSandro2000 PTAL ... and I have marked bazel-remote as broken on Darwin. |
|
@avdv I think Sandro might want you to fix the last commit according to the style guide.
|
SuperSandro2000
requested changes
Feb 3, 2021
It currently fails during compilation because C++ standard header files cannot be found.
|
This is a semi-automatic executed nixpkgs-review with nixpkgs-review-checks extension. It is checked by a human on a best effort basis and does not build all packages (e.g. lumo, tensorflow or pytorch). Result of 14 packages marked as broken and skipped:
4 packages built:
The following issues got detected with the above build packages. bazel: Please consider this feature to be alpha. A substituteInPlace with an unmatched pattern got detected: Please check the offending substituteInPlace for typos or changes in source. Fine for me. |
teto
approved these changes
Feb 4, 2021
|
This is a semi-automatic executed nixpkgs-review with nixpkgs-review-checks extension. It is checked by a human on a best effort basis and does not build all packages (e.g. lumo, tensorflow or pytorch). Result of 13 packages marked as broken and skipped:
1 package built:
The following issues got detected with the above build packages. bazel: Please consider this feature to be alpha. A substituteInPlace with an unmatched pattern got detected: Please check the offending substituteInPlace for typos or changes in source. |
divanorama
added a commit
to divanorama/nixpkgs
that referenced
this pull request
Feb 4, 2021
Bazel 4 is going to be a long term support release. Latest version in NixPkgs so far was 3.3.1 There's a need for more recent version NixOS#97497 All versions from 3.5.0 to 3.7.1 had some reproducibility issues as noted in issue above, but there also seems to be a working PR for 3.7.1 now at NixOS#105439 Notable changes from bazel_3 setup: - put python to default bash path For autodetecting python toolchain with strict action_env on and without this change bazel would fail to autodetect host python. There are some repos that define hermetic python toolchains, but they aren't easy to use yet. Also telling python paths to bazel isn't a 1-liner it seems: - action_env=PATH would affect cache - declaring toolchain via BUILD&WORKSPACE files is not per-user but more like per-repo and affects cache too Using python from nixpkgs shouldn't be too bad in the lack of simpler hermetic python toolchain options - bazel_4.updater is bazel on `bazel query` to support new constructs in WORKSPACE (load of vars, transitive load etc). This is more robust but requires bazel to run the updater, using bazel_3 for now. This is only needed to bump package version, doesn't introduce bazel_4 build dependency on bazel_3 https://blog.bazel.build/2020/11/10/bazel-4.0-announce.html https://blog.bazel.build/2020/11/10/long-term-support-release.html bazelbuild/bazel#12455 https://github.com/bazelbuild/bazel/releases/tag/4.0.0 https://blog.bazel.build/2021/01/19/bazel-4-0.html
|
Thank you @SuperSandro2000 ! |
|
I am trying to add execlog to our bazel package in #111975 , if you have advice on how to fix my error, advice welcome :) |
|
Congratulations @avdv, this was quite the challenging upgrade! Thanks for sticking with it. |
|
Wow, amazing work all! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation for this change
The
rules_scalafor Bazel has added support for Scala 2.13 lately. According to their compatibility matrix it requires at least Bazel 3.5.Upgrading bazel in nixpkgs is blocked currently on dm-sonnet, since it needs the
--incompatible_disable_deprecated_attr_params=falsebeing passed to bazel, which has been removed in bazel 3.4.0 (bazelbuild/bazel@1ed9a07).Currently, in order to build dm-sonnet, which depends on tensorflow 1, which in turn requires bazel 0.26, you have to build bazel 0.26 and bazel 3. The same is true for tensorflow-probability.
Using the same old bazel version to build dm-sonnet and tensorflow-probability which is a hard requirement for tensorflow 1 unblocks the situation for bazel 3.
Note that building dm-sonnet and tensorflow-probability is impossible since tensorflow 1 is broken right now: #104801
Things done
sandboxinnix.confon non-NixOS linux)nix-shell -p nixpkgs-review --run "nixpkgs-review wip"(bazel-remote, bazel-watcher, gvisor build successfully -- I ran out of disk space when building tensorflow 2)./result/bin/)nix path-info -Sbefore and after)