Skip to content

Latest commit

 

History

History
142 lines (96 loc) · 3.91 KB

0000-doctest-name.md

File metadata and controls

142 lines (96 loc) · 3.91 KB
Raw

Summary

Allow giving documentation tests names for identification in test runner output by including name=IDENT in doctest code block info strings.

Motivation

When doctests run, the test runner prints the name of the module that contains them and the line number that they start on. If a module contains many doctests, it is hard to identify which doctest is which in the test runner output by line number alone.

By giving users the option to give names to doctests, identifying individual doctests in test runner output will be easier. Additionally, users could re-run specific tests using cargo test --doc -- $TEST_NAME.

Guide-level explanation

Doctests are written as fenced markdown code blocks that start and end with lines containing three backticks.

The first set of backticks may be followed by an info string. For example, to tell the Rust compiler to ignore the doctest:

// This doctest won't run.
```ignore
assert_eq!(2 + 2, 4);
```

Normally, doctests do not have names, and when run, the test runner will identify them by their test binary, module name, and line number:

   Doc-tests foo

running 1 test
test src/lib.rs - bar (line 37) ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out

If you'd like to have the test runner print a more informative name, perhaps because there are many doctests in a single file, you can put name=IDENT, where IDENT is the name the test should have, which must be an identifier, in the doctest's info string.

```name=arithmetic
assert_eq!(2 + 2, 4);
```
   Doc-tests foo

running 1 test
test src/lib.rs - bar::arithmetic (line 37) ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out

Reference-level explanation

The set of valid words that may appear in code block info strings is extended to include patterns of the form name=IDENT.

This IDENT will be the name of the documentation test generated by the code block, and must be a valid Rust identifier, which includes raw r#-prefixed identifiers.

When the test runner runs the documentation test, the test will be identified by this name, in addition to the test binary, module, and line number.

Multiple name=IDENT words may not appear in a single code block's info string.

Non-unique IDENTs and IDENTS that conflict with other items within the current namespace will produce a warn-by-default error that may become a hard error in the future.

name=IDENT may be combined with existing annotations like rust or should_panic by separating the annotations with commas:


```rust,name=foo
assert!(true)
```

The ability to include multiple comma-separated annotations is an existing feature.

Drawbacks

  • Adds complexity to code block parsing.

  • Requires parsing multi-word info strings, which are standard markdown, but which does increase the complexity of the markdown parser.

Rationale and alternatives

This design seems like the simplest design possible, although viable alternatives may have been overlooked.

The impact of adopting this RFC is minimal. Doctests test output will remain a little hard to make sense of.

Prior art

Unit and integration tests have names. This change would bring doctests closer to parity, by allowing features like test filtering to work with doctests.

Unresolved questions

None.

Future possibilities

None.