Emit nogo facts into a separate archive #3789

fmeum · 2023-12-16T07:44:59Z

What type of PR is this?

Refactoring

What does this PR do? Why is it needed?

Stores nogo facts in a separate archive in preparation for running nogo in a separate action. This gets rid of a lot of code related to packaging nogo facts into the regular export archive.

Note: This commit breaks compatibility with Go 1.16 and earlier.

Which issues(s) does this PR fix?

Other notes for review

go/tools/builders/compilepkg.go

go/tools/builders/utils.go

sluongng

This is a super cool and important PR. It addressed one of the most intriguing TODO in rules_go that I know of.

The result is much cleaner than original and pave way for future refactoring of nogo. Great job!

go/private/actions/archive.bzl

go/private/actions/compilepkg.bzl

tyler-french

Tested this at Uber. We were able to get it working by adjusting certain rules to not depend on things being in the archives, and using the export files instead. We may want to add some notes to the release with this.

This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | [io_bazel_rules_go](https://togithub.com/bazelbuild/rules_go) | http_archive | minor | `v0.45.1` -> `v0.46.0` | --- ### Release Notes <details> <summary>bazelbuild/rules_go (io_bazel_rules_go)</summary> ### [`v0.46.0`](https://togithub.com/bazelbuild/rules_go/releases/tag/v0.46.0) [Compare Source](https://togithub.com/bazelbuild/rules_go/compare/v0.45.1...v0.46.0) #### `WORKSPACE` code load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive") http_archive( name = "io_bazel_rules_go", sha256 = "80a98277ad1311dacd837f9b16db62887702e9f1d1c4c9f796d0121a46c8e184", urls = [ "https://mirror.bazel.build/github.com/bazelbuild/rules_go/releases/download/v0.46.0/rules_go-v0.46.0.zip", "https://github.com/bazelbuild/rules_go/releases/download/v0.46.0/rules_go-v0.46.0.zip", ], ) load("@io_bazel_rules_go//go:deps.bzl", "go_register_toolchains", "go_rules_dependencies") go_rules_dependencies() go_register_toolchains(version = "1.22.0") #### What's Changed - Support custom `GOARM` architecture levels via platform constraints by [@LINKIWI](https://togithub.com/LINKIWI) in [https://github.com/bazelbuild/rules_go/pull/3837](https://togithub.com/bazelbuild/rules_go/pull/3837) - Emit nogo facts into a separate archive by [@fmeum](https://togithub.com/fmeum) in [https://github.com/bazelbuild/rules_go/pull/3789](https://togithub.com/bazelbuild/rules_go/pull/3789) - go_test: ensure external source compilation has data by [@sluongng](https://togithub.com/sluongng) in [https://github.com/bazelbuild/rules_go/pull/3848](https://togithub.com/bazelbuild/rules_go/pull/3848) - Fix invocation of assembler for go1.22 by [@jquirke](https://togithub.com/jquirke) in [https://github.com/bazelbuild/rules_go/pull/3756](https://togithub.com/bazelbuild/rules_go/pull/3756) - nogo: Create a go_register_nogo wrapper for WORKSPACE users. by [@DolceTriade](https://togithub.com/DolceTriade) in [https://github.com/bazelbuild/rules_go/pull/3842](https://togithub.com/bazelbuild/rules_go/pull/3842) - prepare minor release 0.46 by [@tyler-french](https://togithub.com/tyler-french) in [https://github.com/bazelbuild/rules_go/pull/3854](https://togithub.com/bazelbuild/rules_go/pull/3854) #### New Contributors - [@LINKIWI](https://togithub.com/LINKIWI) made their first contribution in [https://github.com/bazelbuild/rules_go/pull/3837](https://togithub.com/bazelbuild/rules_go/pull/3837) - [@jquirke](https://togithub.com/jquirke) made their first contribution in [https://github.com/bazelbuild/rules_go/pull/3756](https://togithub.com/bazelbuild/rules_go/pull/3756) **Full Changelog**: bazel-contrib/rules_go@v0.45.1...v0.46.0 </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Whenever PR is behind base branch, or you tick the rebase/retry checkbox. 👻 **Immortal**: This PR will be recreated if closed unmerged. Get [config help](https://togithub.com/renovatebot/renovate/discussions) if that's undesired. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://togithub.com/renovatebot/renovate).  Co-authored-by: Self-hosted Renovate Bot <361546+cgrindel-self-hosted-renovate[bot]@users.noreply.github.enterprise.com>

[![Mend Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com) This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | [io_bazel_rules_go](https://togithub.com/bazelbuild/rules_go) | http_archive | minor | `v0.45.1` -> `v0.46.0` | --- > [!WARNING] > Some dependencies could not be looked up. Check the Dependency Dashboard for more information. --- ### Release Notes <details> <summary>bazelbuild/rules_go (io_bazel_rules_go)</summary> ### [`v0.46.0`](https://togithub.com/bazelbuild/rules_go/releases/tag/v0.46.0) [Compare Source](https://togithub.com/bazelbuild/rules_go/compare/v0.45.1...v0.46.0) #### `WORKSPACE` code load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive") http_archive( name = "io_bazel_rules_go", sha256 = "80a98277ad1311dacd837f9b16db62887702e9f1d1c4c9f796d0121a46c8e184", urls = [ "https://mirror.bazel.build/github.com/bazelbuild/rules_go/releases/download/v0.46.0/rules_go-v0.46.0.zip", "https://github.com/bazelbuild/rules_go/releases/download/v0.46.0/rules_go-v0.46.0.zip", ], ) load("@io_bazel_rules_go//go:deps.bzl", "go_register_toolchains", "go_rules_dependencies") go_rules_dependencies() go_register_toolchains(version = "1.22.0") #### What's Changed - Support custom `GOARM` architecture levels via platform constraints by [@LINKIWI](https://togithub.com/LINKIWI) in [https://github.com/bazelbuild/rules_go/pull/3837](https://togithub.com/bazelbuild/rules_go/pull/3837) - Emit nogo facts into a separate archive by [@fmeum](https://togithub.com/fmeum) in [https://github.com/bazelbuild/rules_go/pull/3789](https://togithub.com/bazelbuild/rules_go/pull/3789) - go_test: ensure external source compilation has data by [@sluongng](https://togithub.com/sluongng) in [https://github.com/bazelbuild/rules_go/pull/3848](https://togithub.com/bazelbuild/rules_go/pull/3848) - Fix invocation of assembler for go1.22 by [@jquirke](https://togithub.com/jquirke) in [https://github.com/bazelbuild/rules_go/pull/3756](https://togithub.com/bazelbuild/rules_go/pull/3756) - nogo: Create a go_register_nogo wrapper for WORKSPACE users. by [@DolceTriade](https://togithub.com/DolceTriade) in [https://github.com/bazelbuild/rules_go/pull/3842](https://togithub.com/bazelbuild/rules_go/pull/3842) - prepare minor release 0.46 by [@tyler-french](https://togithub.com/tyler-french) in [https://github.com/bazelbuild/rules_go/pull/3854](https://togithub.com/bazelbuild/rules_go/pull/3854) #### New Contributors - [@LINKIWI](https://togithub.com/LINKIWI) made their first contribution in [https://github.com/bazelbuild/rules_go/pull/3837](https://togithub.com/bazelbuild/rules_go/pull/3837) - [@jquirke](https://togithub.com/jquirke) made their first contribution in [https://github.com/bazelbuild/rules_go/pull/3756](https://togithub.com/bazelbuild/rules_go/pull/3756) **Full Changelog**: bazel-contrib/rules_go@v0.45.1...v0.46.0 </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/kreempuff/rules_unreal_engine).

This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | [io_bazel_rules_go](https://togithub.com/bazelbuild/rules_go) | http_archive | minor | `v0.45.1` -> `v0.46.0` | --- ### Release Notes <details> <summary>bazelbuild/rules_go (io_bazel_rules_go)</summary> ### [`v0.46.0`](https://togithub.com/bazelbuild/rules_go/releases/tag/v0.46.0) [Compare Source](https://togithub.com/bazelbuild/rules_go/compare/v0.45.1...v0.46.0) #### `WORKSPACE` code load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive") http_archive( name = "io_bazel_rules_go", sha256 = "80a98277ad1311dacd837f9b16db62887702e9f1d1c4c9f796d0121a46c8e184", urls = [ "https://mirror.bazel.build/github.com/bazelbuild/rules_go/releases/download/v0.46.0/rules_go-v0.46.0.zip", "https://github.com/bazelbuild/rules_go/releases/download/v0.46.0/rules_go-v0.46.0.zip", ], ) load("@io_bazel_rules_go//go:deps.bzl", "go_register_toolchains", "go_rules_dependencies") go_rules_dependencies() go_register_toolchains(version = "1.22.0") #### What's Changed - Support custom `GOARM` architecture levels via platform constraints by [@LINKIWI](https://togithub.com/LINKIWI) in [https://github.com/bazelbuild/rules_go/pull/3837](https://togithub.com/bazelbuild/rules_go/pull/3837) - Emit nogo facts into a separate archive by [@fmeum](https://togithub.com/fmeum) in [https://github.com/bazelbuild/rules_go/pull/3789](https://togithub.com/bazelbuild/rules_go/pull/3789) - go_test: ensure external source compilation has data by [@sluongng](https://togithub.com/sluongng) in [https://github.com/bazelbuild/rules_go/pull/3848](https://togithub.com/bazelbuild/rules_go/pull/3848) - Fix invocation of assembler for go1.22 by [@jquirke](https://togithub.com/jquirke) in [https://github.com/bazelbuild/rules_go/pull/3756](https://togithub.com/bazelbuild/rules_go/pull/3756) - nogo: Create a go_register_nogo wrapper for WORKSPACE users. by [@DolceTriade](https://togithub.com/DolceTriade) in [https://github.com/bazelbuild/rules_go/pull/3842](https://togithub.com/bazelbuild/rules_go/pull/3842) - prepare minor release 0.46 by [@tyler-french](https://togithub.com/tyler-french) in [https://github.com/bazelbuild/rules_go/pull/3854](https://togithub.com/bazelbuild/rules_go/pull/3854) #### New Contributors - [@LINKIWI](https://togithub.com/LINKIWI) made their first contribution in [https://github.com/bazelbuild/rules_go/pull/3837](https://togithub.com/bazelbuild/rules_go/pull/3837) - [@jquirke](https://togithub.com/jquirke) made their first contribution in [https://github.com/bazelbuild/rules_go/pull/3756](https://togithub.com/bazelbuild/rules_go/pull/3756) **Full Changelog**: bazel-contrib/rules_go@v0.45.1...v0.46.0 </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Whenever PR is behind base branch, or you tick the rebase/retry checkbox. 👻 **Immortal**: This PR will be recreated if closed unmerged. Get [config help](https://togithub.com/renovatebot/renovate/discussions) if that's undesired. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://togithub.com/renovatebot/renovate).  Co-authored-by: Self-hosted Renovate Bot <361546+cgrindel-self-hosted-renovate[bot]@users.noreply.github.enterprise.com>

bazel-contrib/rules_go#3789 beakes gVisor nogo tests.

avagin · 2024-02-27T19:38:30Z

This pull request has not been tested and broke users. The motivation of the change is unclear. In gVisor, we have a few addition nogo analyzers that require to have access to object files. Here is one of them: https://cs.opensource.google/gvisor/gvisor/+/master:tools/checkescape/checkescape.go.

fmeum · 2024-02-28T15:39:49Z

I can understand that it's annoying to have your project broken by this change, but it has been tested and there is a reason for it: We are working on decoupling nogo from the compilation action to improve caching. If nogo only needs the export files, nogo actions would have to rerun for more than one target if you only change the implementation.

We could introduce a knob to also forward the object files, but that would nullify most of the gains. Instead, could you change your analyzer so that it emits all the information required for downstream targets into the facts files?

fmeum force-pushed the extract-nogo branch 4 times, most recently from 91df820 to 6e3d725 Compare December 16, 2023 12:56

fmeum requested review from tyler-french and linzhp December 16, 2023 12:57

fmeum marked this pull request as ready for review December 16, 2023 12:57

fmeum mentioned this pull request Dec 16, 2023

nogo: use bazel validations and don't fail compile #3707

Closed

linzhp approved these changes Dec 17, 2023

View reviewed changes

go/tools/builders/compilepkg.go Outdated Show resolved Hide resolved

fmeum force-pushed the extract-nogo branch 2 times, most recently from fcb7222 to 36c2aca Compare December 17, 2023 20:00

linzhp approved these changes Dec 18, 2023

View reviewed changes