Allow using reverse-dependencies for -Zrustdoc-scrape-examples, not just examples

[jyn514]

Problem

Many repositories don't have examples written, or the examples are not very long and don't show what the call looks like in context.

Proposed Solution

However, almost any library will have people actually using it. What cargo could do is instead of running rustdoc like this:

rustc --emit=metadata --name=foo crate_a/src/lib.rs
rustc --emit=metadata --name=foo crate_b/src/lib.rs
rustdoc --scrape-examples-target-crate=crate_a --scrape-examples-target-crate=crate_b examples/foo.rs
rustdoc --with-examples=foo.examples crate_a/src/lib.rs

Scrape the examples from the other crates in the workspace as well:

rustc --emit=metadata --name=foo crate_a/src/lib.rs
rustc --emit=metadata --name=foo crate_b/src/lib.rs
rustdoc --scrape-examples-target-crate=crate_a crate_b/src/lib.rs
rustdoc --with-examples=crate_ b.examples crate_a/src/lib.rs

This can be extended to all the dependencies in the whole tree if desired. This does have the drawback that it will take about twice as long to run rustdoc, but since it's opt-in I think the tradeoff is worth it.

Notes

Here's a proof of concept which runs those commands by hand on cargo-deadlinks:

[willcrichton]

Yeah this is definitely a good idea to have. I think the key question is imagining the Cargo workflow. First thought is something like

cd crate_a
cargo scrape-examples --out-dir /tmp/examples
cd ../crate_b
cargo scrape-examples --out-dir /tmp/examples

cd ../crate_c
cargo doc --with-examples-in-dir /tmp/examples

So we'd need a new command specifically for generating the *.examples files, and then add a top-level doc flag for including user-defined scraped examples that aren't part of the current workspace.

[jyn514]

@willcrichton hmm, is there a reason this would be a separate command for each crate? I was imagining that you could use cargo doc -Zscrape-examples=workspace and cargo would automatically generate examples for each crate (and similarly -Zscrape-examples=dependencies for things not in the workspace)

[willcrichton]

Oh =all already has the semantics you're describing for =workspace. The issue I'm thinking of is how =dependencies would work.

[jyn514]

Oh =all already has the semantics you're describing for =workspace.

Oh hmm - I do see it running rustdoc --edition=2018 --crate-type bin --crate-name cargo_deadlinks src/bin/cargo-deadlinks.rs -o /home/jnelson/src/cargo-deadlinks/target/doc --scrape-examples-output-path /home/jnelson/src/cargo-deadlinks/target/debug/deps/cargo-deadlinks-0b4f32ef0b71752d.examples --scrape-examples-target-crate cargo-deadlinks, but it's not displaying those docs. Maybe I need a version of cargo with #10037 ?

[jyn514]

The issue I'm thinking of is how =dependencies would work.

I think it can use basically the same approach, have cargo run rustdoc in reverse order on all the dependencies (and pass --scrape-examples-target-crate with each crate it depends on, unconditionally - maybe we can add a flag to rustdoc for this so the command line doesn't get super big?)

[willcrichton]

but it's not displaying those docs.

I think you're running into #10035 because cargo-deadlinks has a dash in it. Make sure you build cargo from source on master.

have cargo run rustdoc in reverse order on all the dependencies

Wait when you say "dependencies" are you talking about reverse-dependencies or actual dependencies? Like if we're documenting A where A depends on B and C depends on A (and both B and C are in different workspaces). Is your goal that examples of B in A show up in the documentation for B, or that examples of A in C show up in the documentation for A?

[jyn514]

If I read your example right, it's C <- A <- B, where <- means "depends on". So I intend for both to show up, B should have docs using A's examples, and A should have docs using C's examples (not just at the outer-most level where C is a path dependency, but also if A and B are from crates.io).

[willcrichton]

Ok that makes sense. Then the issue is specifying which reverse-dependencies to pull in, assuming you don't want all of them. Maybe a Cargo.toml configuration like this?

[package]
scrape-examples-from = ["crate_a", "crate_b", "etc"]

[jyn514]

Oh uhh I was actually expecting cargo would pull in all of them 😅 my worry is that if we make it opt-in then people won't actually configure it; a lot of the benefit comes from it working 'automagically' with only a single opt-in from the end user.

[willcrichton]

The current implementation of scrape-examples now allows for scraping reverse-dependencies in your current workspace, so this can be closed. (@weihanglo)