cargo test does not run documentation tests on binary projects

[bltavares]

Hi there folks,

I was preparing an introduction to cargo to share with some coworkers and I wanted to show how the documentation testing works.

So I created a binary project to show a 'Hello world' and added a documentation code that should fail to exemplify the documentation testing, but it didn't run with cargo test.

Here are examples to reproduce:

cargo new --bin binary
cd binary
echo '///```
///assert_eq!(1, 2);
///```
fn main() {
    println!("Hello, world!");
}
' > src/main.rs
cargo test
# Expected: 1 doc-test failed, found 0 failures

cargo new library
cd library
echo '///```
///assert_eq!(1, 2);
///```
fn lib() {
    println!("Hello, world!");
}
' > src/lib.rs
cargo test
# Expected: 1 doc-test failed, found 1 failure

Thanks and keep the good work folks! (:

[steveklabnik]

This is actually a rustdoc 'bug', though I can't seem to find it. Basically, Rustdoc, in its current form, needs to link to your library, and so this is just an inherent limitation of it right now.

/cc @alexcrichton do you remember where that bug is?

[alexcrichton]

Yeah @steveklabnik hit this right on the money, in the case of a binary there's no library to link against (because it's a binary), so this is an inherent limitation of rustdoc itself. If you end up writing lots of functions and documenting heavily in a binary, moving them to a library in the same repo should do the trick!

[bltavares]

Thanks for the explanation folks. Would it make sense to add this information on the documentation about documentation? I think it would be interesting to explain why there is such limitation currently and suggest to move the logic into a library on the same like @alexcrichton said.

I could try to do that later if it makes sense.

Cheers.

[alexcrichton]

Oh dear, this should definitely be in the docs! I'll leave this open to ensure a mention can be added there.

[steveklabnik]

I believe it's already mentioned in Rust's docs. I don't know how much behavior we want to duplicate across both Rust and Cargo though.

[bltavares]

@steveklabnik: now that you mentioned, I found it already documented on Rust. I was about to send a PR on documentation Rust.

I think this could be closed. Sorry about that.

[steveklabnik]

It's all good! I think that this is just hard in general, right? It's part of having tools that are composed together. We have a behavior of rustdoc that's called by cargo, which is documented in rust...

[Rudxain]

Is this related to modules? I have doc-tests in a module file, but rust-doc can't find them.
Terminal:

cargo test
   Compiling xorsum v4.0.0 (/home/rudxain/Documents/GitHub/xorsum)
    Finished test [unoptimized + debuginfo] target(s) in 1.08s
     Running unittests src/lib.rs (target/debug/deps/xorsum-be73721f08e2bcae)

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

     Running unittests src/main.rs (target/debug/deps/xorsum-46e9ecbb076492d7)

running 2 tests
test module::tests::hex_cmp ... ok
test module::test_hasher ... ok

test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

   Doc-tests xorsum

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

The weird thing is that rust-analyzer detects doc-tests, and shows a "Run Doctest" button in VSCode, but when I press it, it says "0 tests run"