Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

--local_test_jobs causes bazel to ignore local resource limitations #16786

Closed
drewmacrae opened this issue Nov 17, 2022 · 3 comments
Closed

--local_test_jobs causes bazel to ignore local resource limitations #16786

drewmacrae opened this issue Nov 17, 2022 · 3 comments
Assignees
Labels
P3 We're not considering working on this, but happy to review a PR. (No assignee) team-Local-Exec Issues and PRs for the Execution (Local) team type: bug

Comments

@drewmacrae
Copy link
Contributor

drewmacrae commented Nov 17, 2022

Description of the bug:

I apply a cpu:4 tag to a set of sh_tests that requires 4 cores to run without timing out. When I set --local_tests_jobs the tests are dispatched regardless of the requirement.

What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

Run 2 sh_tests that "sleep 10" with a cpu:1000 tag and --local_tests_jobs=2 on a multicore system and observe that they finish in ~10s total.

Which operating system are you running Bazel on?

Rolling debian testing Rodette

What is the output of bazel info release?

release 5.3.2

If bazel info release returns development version or (@non-git), tell us how you built Bazel.

No response

What's the output of git remote get-url origin; git rev-parse master; git rev-parse HEAD ?

https://github.com/bazelbuild/bazel
d2aaf0cd66f10745aedee16330c2c053b848a507
ea1646d8d58f5252e55bb891b5eaee67778f23c7

Have you found anything relevant by searching the web?

No response

Any other information, logs, or outputs that you want to share?

This is related to and motivated by work on #16785 but affects bazel in general and the cpu tag in an undocumented way.

@sgowroji sgowroji added type: bug untriaged team-Local-Exec Issues and PRs for the Execution (Local) team labels Nov 18, 2022
@coeuvre coeuvre added P3 We're not considering working on this, but happy to review a PR. (No assignee) and removed untriaged labels Nov 22, 2022
@drewmacrae drewmacrae changed the title --local_test_jobs causes bazel to ignore resource limitations --local_test_jobs causes bazel to ignore local resource limitations Nov 22, 2022
@drewmacrae
Copy link
Contributor Author

Here's what it says in the docs:
"--local_test_jobs=<an integer, or a keyword ("auto", "HOST_CPUS", "HOST_RAM"), optionally followed by an operation ([-|]) eg. "auto", "HOST_CPUS.5"> default: "auto"
The max number of local test jobs to run concurrently. Takes an integer, or a keyword ("auto", "HOST_CPUS", "HOST_RAM"), optionally followed by an operation ([-|]) eg. "auto", "HOST_CPUS.5". 0 means local resources will limit the number of local test jobs to run concurrently instead. Setting this greater than the value for --jobs is ineffectual."

It does document that if 0, local resource limitations will limit the number, which implies that if non-zero they might not, but it's not explicitly stated, and having the cpu:n tags become a no-op isn't so user friendly and contradicts the spec here: https://bazel.build/reference/test-encyclopedia#other-resources

Copy link

Thank you for contributing to the Bazel repository! This issue has been marked as stale since it has not had any activity in the last 1+ years. It will be closed in the next 90 days unless any other activity occurs. If you think this issue is still relevant and should stay open, please post any comment here and the issue will no longer be marked as stale.

@github-actions github-actions bot added the stale Issues or PRs that are stale (no activity for 30 days) label Jan 27, 2024
@DavidANeil
Copy link
Contributor

I ran into this too. I only figured out what was going on by reading the bazel source code.

@github-actions github-actions bot removed the stale Issues or PRs that are stale (no activity for 30 days) label Mar 9, 2024
@wilwell wilwell closed this as completed Aug 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
P3 We're not considering working on this, but happy to review a PR. (No assignee) team-Local-Exec Issues and PRs for the Execution (Local) team type: bug
Projects
None yet
Development

No branches or pull requests

5 participants