Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bazel: gomock bottlenecks build significantly #71854

Closed
irfansharif opened this issue Oct 21, 2021 · 1 comment · Fixed by #75806
Closed

bazel: gomock bottlenecks build significantly #71854

irfansharif opened this issue Oct 21, 2021 · 1 comment · Fixed by #75806
Labels
A-build-system C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) T-dev-inf

Comments

@irfansharif
Copy link
Contributor

irfansharif commented Oct 21, 2021

Describe the problem

We hooked up gomock into our bazel build recently (#67226); it's pretty slow.

To Reproduce

To grab a profile, I started with a fresh bazel build, switched to a branch rebased on top of the original (and adding a few changes).

$ ./dev build cockroach-short -- --profile=bazel-profile.gz

For some super unscientific analysis looking to our worst offendors (the re-build took 100s+, and incremental make buildshort was ~40s).

$ bazel analyze-profile bazel-profile.gz  
Critical path (148.265 s):
...
       Time Percentage   Descriptiona'
   13.467 s    9.08%   action 'GoLink pkg/kv/kvclient/kvcoord/mock_kvcoord_gomock_prog_bin_/mock_kvcoord_gomock_prog_bin'
    3.255 s    2.20%   action 'GoMockReflectExecOnlyGen pkg/kv/kvclient/kvcoord/mocks_generated.go'
    9.320 s    6.29%   action 'GoPath pkg/kv/kvclient/rangefeed/mock_rangefeed_gomock_gopath'
    7.300 s    4.92%   action 'Action pkg/kv/kvclient/rangefeed/mocks_generated.go'
...
   26.920 s   18.16%   action 'GoLink pkg/cmd/cockroach-short/cockroach-short_/cockroach-short'

Another iteration of the exercise showed the same.

    4.968 s    3.43%   action 'GoCompilePkg pkg/roachpb/roachpb.a'
    5.439 s    3.75%   action 'GoCompilePkg pkg/roachpb/with-mocks.a'
    8.227 s    5.67%   action 'GoLink pkg/kv/kvclient/kvcoord/mock_kvcoord_gomock_prog_bin_/mock_kvcoord_gomock_prog_bin'
    7.008 s    4.83%   action 'GoPath pkg/kv/kvclient/rangefeed/mock_rangefeed_gomock_gopath'
    4.574 s    3.15%   action 'Action pkg/kv/kvclient/rangefeed/mocks_generated.go'

Expected behavior

This is probably one of many performance bottlenecks in our build-system. As we start using Bazel in earnest, it'd be good to squash these out.

Epic CRDB-8036

@irfansharif irfansharif added C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) A-build-system labels Oct 21, 2021
@ajwerner
Copy link
Contributor

ajwerner commented Feb 1, 2022

This seems to be due almost entirely to the the use of source-only gomock generation. We do this in two places because of unexported interfaces.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-build-system C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) T-dev-inf
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants