Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implementing "<test_binary> --list --format json" for use by IDE test explorers / runners #107307

Closed
parthopdas opened this issue Jan 25, 2023 · 3 comments · Fixed by #108148
Closed
Labels
A-libtest Area: `#[test]` / the `test` library

Comments

@parthopdas
Copy link
Contributor

[consider the default output of cargo new a --lib]

Issue

Test explorers in editors (e.g. VSCode, see pic below) are unable to navigate to test location in the editor from the test explorer.

Cause

cargo test --list -q used by the editors to get list of tests do not have the source and line / column info.

Description

Currently cargo test --list -q shows the following output

tests::it_works: test

This is used by editor extensions e.g. vscode rust test adapter to show & run tests from the tests pane in VS Code. e.g.
image

While it is possible to run and debug the tests from the test explorer, it is not possible to navigate back to the test location as output above is missing that information. E.g. you cannot double click on a test on the test explorer to navigate to its definition in the editor.

Suggestion is to add the source info to the output of --list as follows

tests::it_works: test,<path>|<start_line>|<start_col>|<end_line>|<end_col>

Fix proposal

Proposal for fix (& I only have a plan A, happy to receive further guidance) is to update test::TestDescAndFn as follows

// https://github.com/rust-lang/rust/blob/master/library/test/src/types.rs#L118
#[derive(Clone, Debug, PartialEq, Eq, Hash)]
pub struct TestDesc {
    pub name: TestName,
    pub ignore: bool,
    pub ignore_message: Option<&'static str>,
    pub should_panic: options::ShouldPanic,
    pub compile_fail: bool,
    pub no_run: bool,
    pub test_type: TestType,
    pub source_path: &'static str,     // <- Added
    pub start_line: i32,               // <- Added
    pub start_column: i32,             // <- Added
    pub end_line: i32,                 // <- Added
    pub end_column: i32,               // <- Added
}

Once this is available, the corresponding changes can be made to compiler/rustc_builtin_macros/src/test.rs to emit the above info and then in library/test/src/console.rs to write it out.

@parthopdas
Copy link
Contributor Author

I'd be happy to raise a PR.

@Noratrieb Noratrieb added the A-libtest Area: `#[test]` / the `test` library label Jan 26, 2023
@parthopdas
Copy link
Contributor Author

Folks would anyone be able to give me some hints on how to track down the source file & line number from the compiler's internal object model?

parthopdas added a commit to parthopdas/rust that referenced this issue Feb 8, 2023
… output

earlier:
  p2dchecks::fibonacci_test::case_1: test

now:
  // test test_type | ignored_or_not | location_info
  p2dchecks::fibonacci_test::case_1: test | false | src\lib.rs:57:8: 57:19
parthopdas added a commit to parthopdas/rust that referenced this issue Feb 8, 2023
parthopdas added a commit to parthopdas/rust that referenced this issue Feb 8, 2023
parthopdas added a commit to parthopdas/rust that referenced this issue Feb 8, 2023
parthopdas added a commit to parthopdas/rust that referenced this issue Feb 8, 2023
parthopdas added a commit to parthopdas/rust that referenced this issue Feb 8, 2023
parthopdas added a commit to parthopdas/rust that referenced this issue Feb 8, 2023
parthopdas added a commit to parthopdas/rust that referenced this issue Feb 8, 2023
parthopdas added a commit to parthopdas/rust that referenced this issue Feb 8, 2023
parthopdas added a commit to parthopdas/rust that referenced this issue Feb 8, 2023
@parthopdas
Copy link
Contributor Author

@parthopdas parthopdas changed the title Adding file path and line number to "cargo test --list" output Implementing "<test_binary> --list --format json" for use by IDE test explorers / runners Feb 17, 2023
matthiaskrgr added a commit to matthiaskrgr/rust that referenced this issue Mar 8, 2023
rust-lang#107307 Implementing "<test_binary> --list --format json" for use by IDE test explorers / runners

PR 1 of 2 - wiring up just the new information + implement the command line changes i.e. --format json + tests

upcoming:
PR 2 of 2 - clean up "#[cfg(not(bootstrap))]" from PR 1

As per the discussions on
- MCP: https://rust-lang.zulipchat.com/#narrow/stream/233931-t-compiler.2Fmajor-changes/topic/Implementing.20.22.3Ctest_binary.3E.20--list.20--form.E2.80.A6.20compiler-team.23592/near/328747548
- preRFC: https://internals.rust-lang.org/t/pre-rfc-implementing-test-binary-list-format-json-for-use-by-ide-test-explorers-runners/18308
- FYI on Discord: https://discord.com/channels/442252698964721669/459149169546887178/1075581549409484820
matthiaskrgr added a commit to matthiaskrgr/rust that referenced this issue Mar 8, 2023
rust-lang#107307 Implementing "<test_binary> --list --format json" for use by IDE test explorers / runners

PR 1 of 2 - wiring up just the new information + implement the command line changes i.e. --format json + tests

upcoming:
PR 2 of 2 - clean up "#[cfg(not(bootstrap))]" from PR 1

As per the discussions on
- MCP: https://rust-lang.zulipchat.com/#narrow/stream/233931-t-compiler.2Fmajor-changes/topic/Implementing.20.22.3Ctest_binary.3E.20--list.20--form.E2.80.A6.20compiler-team.23592/near/328747548
- preRFC: https://internals.rust-lang.org/t/pre-rfc-implementing-test-binary-list-format-json-for-use-by-ide-test-explorers-runners/18308
- FYI on Discord: https://discord.com/channels/442252698964721669/459149169546887178/1075581549409484820
GuillaumeGomez added a commit to GuillaumeGomez/rust that referenced this issue Mar 8, 2023
rust-lang#107307 Implementing "<test_binary> --list --format json" for use by IDE test explorers / runners

PR 1 of 2 - wiring up just the new information + implement the command line changes i.e. --format json + tests

upcoming:
PR 2 of 2 - clean up "#[cfg(not(bootstrap))]" from PR 1

As per the discussions on
- MCP: https://rust-lang.zulipchat.com/#narrow/stream/233931-t-compiler.2Fmajor-changes/topic/Implementing.20.22.3Ctest_binary.3E.20--list.20--form.E2.80.A6.20compiler-team.23592/near/328747548
- preRFC: https://internals.rust-lang.org/t/pre-rfc-implementing-test-binary-list-format-json-for-use-by-ide-test-explorers-runners/18308
- FYI on Discord: https://discord.com/channels/442252698964721669/459149169546887178/1075581549409484820
matthiaskrgr added a commit to matthiaskrgr/rust that referenced this issue Mar 10, 2023
rust-lang#107307 Implementing "<test_binary> --list --format json" for use by IDE test explorers / runners

PR 1 of 2 - wiring up just the new information + implement the command line changes i.e. --format json + tests

upcoming:
PR 2 of 2 - clean up "#[cfg(not(bootstrap))]" from PR 1

As per the discussions on
- MCP: https://rust-lang.zulipchat.com/#narrow/stream/233931-t-compiler.2Fmajor-changes/topic/Implementing.20.22.3Ctest_binary.3E.20--list.20--form.E2.80.A6.20compiler-team.23592/near/328747548
- preRFC: https://internals.rust-lang.org/t/pre-rfc-implementing-test-binary-list-format-json-for-use-by-ide-test-explorers-runners/18308
- FYI on Discord: https://discord.com/channels/442252698964721669/459149169546887178/1075581549409484820
matthiaskrgr added a commit to matthiaskrgr/rust that referenced this issue Mar 17, 2023
Implementing "<test_binary> --list --format json" for use by IDE test explorers / runners

Fixes rust-lang#107307

PR 1 of 2 - wiring up just the new information + implement the command line changes i.e. --format json + tests

upcoming:
PR 2 of 2 - clean up "#[cfg(not(bootstrap))]" from PR 1

As per the discussions on
- MCP: https://rust-lang.zulipchat.com/#narrow/stream/233931-t-compiler.2Fmajor-changes/topic/Implementing.20.22.3Ctest_binary.3E.20--list.20--form.E2.80.A6.20compiler-team.23592/near/328747548
- preRFC: https://internals.rust-lang.org/t/pre-rfc-implementing-test-binary-list-format-json-for-use-by-ide-test-explorers-runners/18308
- FYI on Discord: https://discord.com/channels/442252698964721669/459149169546887178/1075581549409484820
@bors bors closed this as completed in 9d0eac4 Mar 20, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-libtest Area: `#[test]` / the `test` library
Projects
None yet
2 participants