Compile tests for multiple packages #1828

fhahn · 2015-07-24T11:11:12Z

This PR for #1528 is still pretty rough (and so far only added multiple package support for tests), but I wanted to make sure the overall approach is fine.

I have made some progress and cargo is now able to compile and execute tests for multiple packages.

In order to execute the doc tests for multiple packages as well, I added to_doc_test to Compilation (used here). Previously it only executed the doc tests for Compilation.package. Another option would be to change Compilation.package to be a Vec<Package>, because in order to execute the tests of multiple packages, multiple packages are compiled as "top level package". But this would require a changes at a couple of other places.

rust-highfive · 2015-07-24T11:11:24Z

r? @alexcrichton

(rust_highfive has picked a reviewer for you, use r? to override)

alexcrichton · 2015-07-24T16:25:12Z

src/bin/test.rs

@@ -26,7 +26,7 @@ pub const USAGE: &'static str = "
 Execute all unit and integration tests of a local package

 Usage:
-    cargo test [options] [--] [<args>...]
+    cargo test [options] [-p SPEC --package SPEC]... [--] [<args>...]


cc @BurntSushi, you mentioned on docopt/docopt#275 that you were thinking of proposing the ability to do this in the options section instead of up here in the usage section, do you know if there's been progress on that? I wouldn't mind proposing an idea or two on that thread to see if I can get discussion rolling again, it'd be great to fix :)

@alexcrichton Thanks for the reminder! I have a candidate implementation working in 0.6.69. See my proposal for details: docopt/docopt#275 (Even though it's working in the Rust implementation, I probably wouldn't add it into Cargo quite yet in case the proposal changes. If it's ultimately rejected, I might still leave it in because I think it's really really useful.)

Sounds good to me, thanks @BurntSushi!

alexcrichton · 2015-07-24T17:18:54Z

Yeah this all looks like a pretty good approach to me!

bors · 2015-08-04T23:02:49Z

☔ The latest upstream changes (presumably #1830) made this pull request unmergeable. Please resolve the merge conflicts.

fhahn · 2015-09-06T21:39:38Z

I finally had time to work on this PR again. I have started adding tests, but I am not sure how to test cargo test and cargo bench, because the output order is non deterministic for multiple packages (e.g. test_cargo_test::test_multiple_packages) Any ideas whats a good way to test this would be?

bors · 2015-09-08T20:44:36Z

☔ The latest upstream changes (presumably #1973) made this pull request unmergeable. Please resolve the merge conflicts.

alexcrichton · 2015-09-09T16:07:04Z

The nondeterminism may be solved by -j1, but if that doesn't work then just testing that they're accepted should be fine.

alexcrichton · 2015-09-09T16:10:10Z

src/bin/bench.rs

@@ -26,7 +26,7 @@ pub const USAGE: &'static str = "
 Execute all benchmarks of a local package

 Usage:
-    cargo bench [options] [--] [<args>...]
+    cargo bench [options] [-p SPEC --package SPEC]... [--] [<args>...]


I think that new syntax in docopt may actually allow for nicer definitions of this, something like:

Options: -p, --package=SPEC ... Description ...

I believe it's still somewhat experimental, but this may be a good place to get a good start on it?

fhahn · 2015-09-30T19:31:20Z

I've squashed the commits down to 6, but I could reduce the number even further if necessary.

closes rust-lang#1528

alexcrichton · 2015-09-30T21:43:56Z

Looks like this may have accidentally picked up some other commits, perhaps a rebase is in order? I'll give this one last look-through but I'll probably just send it to bors.

alexcrichton · 2015-09-30T23:26:49Z

Now that I've actually gotten around to taking a peek, this looks fantastic to me. Thanks so much again for pushing on this @fhahn!

@bors: r+ d9615d7

This PR for #1528 is still pretty rough (and so far only added multiple package support for tests), but I wanted to make sure the overall approach is fine. I have made some progress and cargo is now able to compile and execute tests for multiple packages. In order to execute the doc tests for multiple packages as well, I added `to_doc_test` to `Compilation` (used [here](fhahn@8b8df3e#diff-417e085367d0ce027505dfaa26a089d3R33)). Previously it only executed the doc tests for `Compilation.package`. Another option would be to change `Compilation.package` to be a `Vec<Package>`, because in order to execute the tests of multiple packages, multiple packages are compiled as "top level package". But this would require a changes at a couple of other places.

bors · 2015-09-30T23:26:51Z

⌛ Testing commit d9615d7 with merge 7931e58...

bors · 2015-09-30T23:45:09Z

☀️ Test successful - cargo-linux-32, cargo-linux-64, cargo-mac-32, cargo-mac-64, cargo-win-gnu-32, cargo-win-gnu-64, cargo-win-msvc-32, cargo-win-msvc-64

fhahn · 2015-10-01T09:30:24Z

Great, thanks @alexcrichton for your patience :)

Upgrade Cargo to get rust-lang/cargo#1828, and use it for unit tests. This allows Cargo to get some more parallelism when compiling the test crates’ dependencies. `touch components/util/lib.rs && mach test-unit` on my machine goes from 149 seconds to 124.

Run all unit tests with just one Cargo command. Upgrade Cargo to get rust-lang/cargo#1828, and use it for unit tests. This allows Cargo to get some more parallelism when compiling the test crates’ dependencies. `touch components/util/lib.rs && mach test-unit` on my machine goes from 149 seconds to 124.  [<img src="https://reviewable.io/review_button.png" height=40 alt="Review on Reviewable"/>](https://reviewable.io/reviews/servo/servo/7813)

Upgrade Cargo to get rust-lang/cargo#1828, and use it for unit tests. This allows Cargo to get some more parallelism when compiling the test crates’ dependencies. `touch components/util/lib.rs && mach test-unit` on my machine goes from 149 seconds to 124.

Run all unit tests with just one Cargo command. Upgrade Cargo to get rust-lang/cargo#1828, and use it for unit tests. This allows Cargo to get some more parallelism when compiling the test crates’ dependencies. `touch components/util/lib.rs && mach test-unit` on my machine goes from 149 seconds to 124.  [<img src="https://reviewable.io/review_button.png" height=40 alt="Review on Reviewable"/>](https://reviewable.io/reviews/servo/servo/7813)

…from servo:parallel-unit-tests); r=mbrubeck Upgrade Cargo to get rust-lang/cargo#1828, and use it for unit tests. This allows Cargo to get some more parallelism when compiling the test crates’ dependencies. `touch components/util/lib.rs && mach test-unit` on my machine goes from 149 seconds to 124. Source-Repo: https://github.com/servo/servo Source-Revision: ba2714f4f607da77bd7200f88cfa16c1d10da9cd

@alexcrichton

Allow `cargo doc --open` with multiple packages. If `cargo doc --open` builds multiple packages, open the first one. This seems pretty natural to me (the first one on the command line, or the first default member if `default-members` is specified, or the root of a workspace). Rustdoc shows a list of crates in the sidebar, so if it doesn't open the one the user wants, it's a trivial matter of clicking on the crate name. @alexcrichton specifically asked for an error [here](#1828 (diff)). However, at the time I don't think rustdoc dynamically generated the "Crates" listing in the sidebar. Alex, I wonder if your stance still holds? Closes #5145

…from servo:parallel-unit-tests); r=mbrubeck Upgrade Cargo to get rust-lang/cargo#1828, and use it for unit tests. This allows Cargo to get some more parallelism when compiling the test crates’ dependencies. `touch components/util/lib.rs && mach test-unit` on my machine goes from 149 seconds to 124. Source-Repo: https://github.com/servo/servo Source-Revision: ba2714f4f607da77bd7200f88cfa16c1d10da9cd UltraBlame original commit: a747ff057fac88586b57a543cf00052e24c93afe

rust-highfive assigned alexcrichton Jul 24, 2015

fhahn force-pushed the multiple-package-parameters branch from 8b8df3e to 64a8cb0 Compare July 24, 2015 11:23

alexcrichton reviewed Jul 24, 2015
View reviewed changes

fhahn force-pushed the multiple-package-parameters branch from 64a8cb0 to 44957f6 Compare July 24, 2015 21:08

fhahn force-pushed the multiple-package-parameters branch from a56fd80 to 38335d8 Compare August 2, 2015 22:04

fhahn force-pushed the multiple-package-parameters branch 3 times, most recently from 35c6833 to 4f7f409 Compare September 6, 2015 21:05

alexcrichton reviewed Sep 9, 2015
View reviewed changes

fhahn added 3 commits September 30, 2015 21:30

Add support for multiple -p options to cargo {bench, build, doc}

b3ade7c

Use docopt's new syntax for repeatable options

9d301d6

Add support for multiple -p options to cargo clean

dad79c9

fhahn force-pushed the multiple-package-parameters branch from 2ebf892 to 016e971 Compare September 30, 2015 19:31

fhahn added 2 commits September 30, 2015 21:42

Add support for multiple -p options to cargo update

f0647ea

closes rust-lang#1528

Use std::env::EXE_SUFFIX in new tests

d9615d7

fhahn force-pushed the multiple-package-parameters branch from 016e971 to d9615d7 Compare September 30, 2015 19:42

bors merged commit d9615d7 into rust-lang:master Sep 30, 2015

bors mentioned this pull request Sep 30, 2015

Add cargo rustdoc for passing arbitrary flags to rustdoc #1977

Closed

fhahn deleted the multiple-package-parameters branch October 1, 2015 09:28

SimonSapin mentioned this pull request Oct 1, 2015

Run all unit tests with just one Cargo command. servo/servo#7813

Merged

ehuss mentioned this pull request Mar 31, 2019

Allow cargo doc --open with multiple packages. #6803

Merged

RinChanNOWWW mentioned this pull request May 28, 2022

cargo test: continue other packages' tests when one package's test is failed. #10712

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Compile tests for multiple packages #1828

Compile tests for multiple packages #1828

fhahn commented Jul 24, 2015

rust-highfive commented Jul 24, 2015

alexcrichton Jul 24, 2015

BurntSushi Jul 28, 2015

alexcrichton Jul 28, 2015

alexcrichton commented Jul 24, 2015

bors commented Aug 4, 2015

fhahn commented Sep 6, 2015

bors commented Sep 8, 2015

alexcrichton commented Sep 9, 2015

alexcrichton Sep 9, 2015

fhahn commented Sep 30, 2015

alexcrichton commented Sep 30, 2015

alexcrichton commented Sep 30, 2015

bors commented Sep 30, 2015

bors commented Sep 30, 2015

fhahn commented Oct 1, 2015

Compile tests for multiple packages #1828

Compile tests for multiple packages #1828

Conversation

fhahn commented Jul 24, 2015

rust-highfive commented Jul 24, 2015

alexcrichton Jul 24, 2015

Choose a reason for hiding this comment

BurntSushi Jul 28, 2015

Choose a reason for hiding this comment

alexcrichton Jul 28, 2015

Choose a reason for hiding this comment

alexcrichton commented Jul 24, 2015

bors commented Aug 4, 2015

fhahn commented Sep 6, 2015

bors commented Sep 8, 2015

alexcrichton commented Sep 9, 2015

alexcrichton Sep 9, 2015

Choose a reason for hiding this comment

fhahn commented Sep 30, 2015

alexcrichton commented Sep 30, 2015

alexcrichton commented Sep 30, 2015

bors commented Sep 30, 2015

bors commented Sep 30, 2015

fhahn commented Oct 1, 2015