Skip to content

Commit

Permalink
Resolve naming conflict with internal system
Browse files Browse the repository at this point in the history
Summary:
This CL changes the summary output that Scrut prints after `test` and `update` operations.

# What?

The `pretty` renderer used to generate summaries such as:

```
1 file(s) with 1 test(s): 0 succeeded, 1 failed and 0 skipped
```

Now it creates instead summaries that start with the word `Result`:

```
Result: 1 file(s) with 1 test(s): 0 succeeded, 1 failed and 0 skipped
```

# Why?

The word `Summary` interferes with internal systems. This leads to wrong / corrupted signals.

Reviewed By: uepoch

Differential Revision: D55635336

fbshipit-source-id: dcb27074906058602a1bec7b18403e142dc27c21
  • Loading branch information
ukautz authored and facebook-github-bot committed Apr 3, 2024
1 parent b8c78cd commit c6cf5ca
Show file tree
Hide file tree
Showing 40 changed files with 62 additions and 81 deletions.
6 changes: 3 additions & 3 deletions py/pytest/parsertest.py
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ def test_validate_valid_input(self) -> None:
ok, error = self._testcases()[0].validate(output, "the-location")
self.assertEqual(True, ok)
self.assertEqual(
"Summary: 1 file(s) with 1 test(s): 1 succeeded, 0 failed and 0 skipped\n",
"Result: 1 file(s) with 1 test(s): 1 succeeded, 0 failed and 0 skipped\n",
error,
)

Expand All @@ -53,7 +53,7 @@ def test_fail_on_invalid_exit_code(self) -> None:
## STDERR
Summary: 1 file(s) with 1 test(s): 0 succeeded, 1 failed and 0 skipped
Result: 1 file(s) with 1 test(s): 0 succeeded, 1 failed and 0 skipped
""".format(
expected_line=self._expect_line()
),
Expand All @@ -78,7 +78,7 @@ def test_validation_invalid_input(self) -> None:
1 | + Wrong
Summary: 1 file(s) with 1 test(s): 0 succeeded, 1 failed and 0 skipped
Result: 1 file(s) with 1 test(s): 0 succeeded, 1 failed and 0 skipped
""".format(
expected_line=self._expect_line()
),
Expand Down
4 changes: 2 additions & 2 deletions selftest/complex/append/test.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ Scrut supports appending of tests in front of a list of other files.

```scrut
$ "${SCRUT_BIN}" test --work-directory "$(pwd)" --match-markdown "*.mdtest" "$TESTDIR/actual.mdtest"
Summary: 1 file(s) with 1 test(s): 1 succeeded, 0 failed and 0 skipped
Result: 1 file(s) with 1 test(s): 1 succeeded, 0 failed and 0 skipped
```

There should be a file still in place afterwards
Expand All @@ -20,7 +20,7 @@ File exists

```scrut
$ "${SCRUT_BIN}" test --work-directory "$(pwd)" --match-markdown "*.mdtest" --append-test-file-paths "$TESTDIR/append.mdtest" "$TESTDIR/actual.mdtest"
Summary: 0 file(s) with 0 test(s): 0 succeeded, 0 failed and 0 skipped
Result: 0 file(s) with 0 test(s): 0 succeeded, 0 failed and 0 skipped
```

There should be no file around afterwards
Expand Down
8 changes: 4 additions & 4 deletions selftest/complex/combine-output/test-combine-output.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,27 +18,27 @@ $ "$SCRUT_BIN" test --match-markdown "*.mdtest" "$TESTDIR/test.mdtest"
2 | - standard error
Summary: 1 file(s) with 1 test(s): 0 succeeded, 1 failed and 0 skipped
Result: 1 file(s) with 1 test(s): 0 succeeded, 1 failed and 0 skipped
[50]
```

## Run test with combined output

```scrut
$ "$SCRUT_BIN" test --combine-output --match-markdown "*.mdtest" "$TESTDIR/test.mdtest"
Summary: 1 file(s) with 1 test(s): 1 succeeded, 0 failed and 0 skipped
Result: 1 file(s) with 1 test(s): 1 succeeded, 0 failed and 0 skipped
```

## Run test with combined output (global)

```scrut
$ "$SCRUT_BIN" --combine-output test --match-markdown "*.mdtest" "$TESTDIR/test.mdtest"
Summary: 1 file(s) with 1 test(s): 1 succeeded, 0 failed and 0 skipped
Result: 1 file(s) with 1 test(s): 1 succeeded, 0 failed and 0 skipped
```

## Run test with cram compat enabling combined output

```scrut
$ "$SCRUT_BIN" --cram-compat test --match-markdown "*.mdtest" "$TESTDIR/test.mdtest"
Summary: 1 file(s) with 1 test(s): 1 succeeded, 0 failed and 0 skipped
Result: 1 file(s) with 1 test(s): 1 succeeded, 0 failed and 0 skipped
```
6 changes: 3 additions & 3 deletions selftest/complex/cram-compat/test.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ Assure that the `--cram-compat` flag enables Cram backwards compatibility featur

```scrut
$ $SCRUT_BIN test --match-markdown "*.mdtest" "$TESTDIR/test-without-cr.mdtest" 2>&1
Summary: 1 file(s) with 1 test(s): 1 succeeded, 0 failed and 0 skipped
Result: 1 file(s) with 1 test(s): 1 succeeded, 0 failed and 0 skipped
```

Scrut treats `\r` (LF) and `\r\n` (CRLF) as line breaks.
Expand All @@ -15,7 +15,7 @@ Scrut treats `\r` (LF) and `\r\n` (CRLF) as line breaks.

```scrut
$ $SCRUT_BIN test --match-markdown "*.mdtest" --cram-compat "$TESTDIR/test-with-cr.mdtest" 2>&1
Summary: 1 file(s) with 1 test(s): 1 succeeded, 0 failed and 0 skipped
Result: 1 file(s) with 1 test(s): 1 succeeded, 0 failed and 0 skipped
```

Scrut requires explicit handling of `\r` (CR) line breaks.
Expand All @@ -24,7 +24,7 @@ Scrut requires explicit handling of `\r` (CR) line breaks.

```scrut
$ $SCRUT_BIN test --match-cram "*.cram" "$TESTDIR/test.cram" 2>&1
Summary: 1 file(s) with 1 test(s): 1 succeeded, 0 failed and 0 skipped
Result: 1 file(s) with 1 test(s): 1 succeeded, 0 failed and 0 skipped
```

Cram files auto-enable cram compat mode
8 changes: 4 additions & 4 deletions selftest/complex/crlf-support/test.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,28 +20,28 @@ $ "$SCRUT_BIN" test --match-markdown "*.mdtest" "$TESTDIR/test.mdtest"
2 | + Newlines
Summary: 1 file(s) with 1 test(s): 0 succeeded, 1 failed and 0 skipped
Result: 1 file(s) with 1 test(s): 0 succeeded, 1 failed and 0 skipped
[50]
```

## Run with explicit CRLF handling

```scrut
$ "$SCRUT_BIN" test --keep-output-crlf --match-markdown "*.mdtest" "$TESTDIR/test.mdtest"
Summary: 1 file(s) with 1 test(s): 1 succeeded, 0 failed and 0 skipped
Result: 1 file(s) with 1 test(s): 1 succeeded, 0 failed and 0 skipped
```

## Run with explicit CRLF handling (global)

```scrut
$ "$SCRUT_BIN" --keep-output-crlf test --match-markdown "*.mdtest" "$TESTDIR/test.mdtest"
Summary: 1 file(s) with 1 test(s): 1 succeeded, 0 failed and 0 skipped
Result: 1 file(s) with 1 test(s): 1 succeeded, 0 failed and 0 skipped
```


## Run with cram compat enabling CRLF handling

```scrut
$ "$SCRUT_BIN" --cram-compat test --match-markdown "*.mdtest" "$TESTDIR/test.mdtest"
Summary: 1 file(s) with 1 test(s): 1 succeeded, 0 failed and 0 skipped
Result: 1 file(s) with 1 test(s): 1 succeeded, 0 failed and 0 skipped
```
6 changes: 3 additions & 3 deletions selftest/complex/custom-shell/test-custom-shell.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,20 +24,20 @@ $ "$SCRUT_BIN" test --match-markdown "*.mdtest" "$TESTDIR/test.mdtest"
1 | + This is from a custom shell: no
Summary: 1 file(s) with 1 test(s): 0 succeeded, 1 failed and 0 skipped
Result: 1 file(s) with 1 test(s): 0 succeeded, 1 failed and 0 skipped
[50]
```

## Run test with custom shell

```scrut
$ "$SCRUT_BIN" test --shell "$TESTDIR/shell.sh" --match-markdown "*.mdtest" "$TESTDIR/test.mdtest"
Summary: 1 file(s) with 1 test(s): 1 succeeded, 0 failed and 0 skipped
Result: 1 file(s) with 1 test(s): 1 succeeded, 0 failed and 0 skipped
```

## Run test with custom shell (global)

```scrut
$ "$SCRUT_BIN" --shell "$TESTDIR/shell.sh" test --match-markdown "*.mdtest" "$TESTDIR/test.mdtest"
Summary: 1 file(s) with 1 test(s): 1 succeeded, 0 failed and 0 skipped
Result: 1 file(s) with 1 test(s): 1 succeeded, 0 failed and 0 skipped
```
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ $ "$SCRUT_BIN" test --match-markdown "*.mdtest" "$TESTDIR/test.mdtest"
2 | - test-me.fixture
Summary: 1 file(s) with 1 test(s): 0 succeeded, 1 failed and 0 skipped
Result: 1 file(s) with 1 test(s): 0 succeeded, 1 failed and 0 skipped
[50]
```

Expand All @@ -31,14 +31,14 @@ $ mkdir $TMPDIR/foo && touch $TMPDIR/foo/test-me.fixture

```scrut
$ "$SCRUT_BIN" test --work-directory "$TMPDIR/foo" --match-markdown "*.mdtest" "$TESTDIR/test.mdtest"
Summary: 1 file(s) with 1 test(s): 1 succeeded, 0 failed and 0 skipped
Result: 1 file(s) with 1 test(s): 1 succeeded, 0 failed and 0 skipped
```

## Run test with custom work directory (global)

```scrut
$ "$SCRUT_BIN" --work-directory "$TMPDIR/foo" test --match-markdown "*.mdtest" "$TESTDIR/test.mdtest"
Summary: 1 file(s) with 1 test(s): 1 succeeded, 0 failed and 0 skipped
Result: 1 file(s) with 1 test(s): 1 succeeded, 0 failed and 0 skipped
```

Ensure idempotent execution by cleaning up fixtures directory
Expand Down
4 changes: 2 additions & 2 deletions selftest/complex/glob-greed/test.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ Assure the glob rule is only as greed as it should be

```scrut
$ "$SCRUT_BIN" test --match-markdown "*.mdtest" "$TESTDIR/greedy-good.mdtest"
Summary: 1 file(s) with 1 test(s): 1 succeeded, 0 failed and 0 skipped
Result: 1 file(s) with 1 test(s): 1 succeeded, 0 failed and 0 skipped
```

## Fail with invalid concrete match sandwiched between greedy multiline
Expand All @@ -25,6 +25,6 @@ $ "$SCRUT_BIN" test --match-markdown "*.mdtest" "$TESTDIR/greedy-bad.mdtest"
2 | - bla
Summary: 1 file(s) with 1 test(s): 0 succeeded, 1 failed and 0 skipped
Result: 1 file(s) with 1 test(s): 0 succeeded, 1 failed and 0 skipped
[50]
```
4 changes: 2 additions & 2 deletions selftest/complex/inline-configuration/test-document-append.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ $ scrut_test "$TESTDIR"/test-document-append.mdtest
1 | + FooInAppended1
Summary: 1 file(s) with 1 test(s): 0 succeeded, 1 failed and 0 skipped
Result: 1 file(s) with 1 test(s): 0 succeeded, 1 failed and 0 skipped
[50]
```

Expand Down Expand Up @@ -54,7 +54,7 @@ $ scrut_test "$TESTDIR"/test-document-append.mdtest --append-test-file-paths "$T
1 | + FooInAppended2
Summary: 1 file(s) with 2 test(s): 0 succeeded, 2 failed and 0 skipped
Result: 1 file(s) with 2 test(s): 0 succeeded, 2 failed and 0 skipped
[50]
```

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ $ scrut_test "$TESTDIR"/test-document-prepend.mdtest
1 | + FooInPrepended1
Summary: 1 file(s) with 1 test(s): 0 succeeded, 1 failed and 0 skipped
Result: 1 file(s) with 1 test(s): 0 succeeded, 1 failed and 0 skipped
[50]
```

Expand Down Expand Up @@ -54,7 +54,7 @@ $ scrut_test "$TESTDIR"/test-document-prepend.mdtest --prepend-test-file-paths "
1 | + FooInPrepended1
Summary: 1 file(s) with 2 test(s): 0 succeeded, 2 failed and 0 skipped
Result: 1 file(s) with 2 test(s): 0 succeeded, 2 failed and 0 skipped
[50]
```

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ $ alias scrut_test='$SCRUT_BIN test --match-markdown="*.mdtest"'

```scrut
$ scrut_test "$TESTDIR"/test-document-total-no-timeout.mdtest
Summary: 1 file(s) with 3 test(s): 3 succeeded, 0 failed and 0 skipped
Result: 1 file(s) with 3 test(s): 3 succeeded, 0 failed and 0 skipped
```

## Timeout with inline config
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ word\r (escaped)

```scrut
$ scrut_test "$TESTDIR"/test-testcase-keep-crlf-disabled.mdtest
Summary: 1 file(s) with 1 test(s): 1 succeeded, 0 failed and 0 skipped
Result: 1 file(s) with 1 test(s): 1 succeeded, 0 failed and 0 skipped
```

## Explicitly enabled
Expand All @@ -43,6 +43,6 @@ $ scrut_test "$TESTDIR"/test-testcase-keep-crlf-disabled.mdtest --keep-output-cr
1 | + word\r (escaped) (equal)
Summary: 1 file(s) with 1 test(s): 0 succeeded, 1 failed and 0 skipped
Result: 1 file(s) with 1 test(s): 0 succeeded, 1 failed and 0 skipped
[50]
```
Original file line number Diff line number Diff line change
Expand Up @@ -10,12 +10,12 @@ $ alias scrut_test='$SCRUT_BIN test --match-markdown="*.mdtest"'

```scrut
$ scrut_test "$TESTDIR"/test-testcase-skip-document-code-default.mdtest
Summary: 1 file(s) with 3 test(s): 0 succeeded, 0 failed and 3 skipped
Result: 1 file(s) with 3 test(s): 0 succeeded, 0 failed and 3 skipped
```

## Custom skip document code

```scrut
$ scrut_test "$TESTDIR"/test-testcase-skip-document-code-custom.mdtest
Summary: 1 file(s) with 3 test(s): 0 succeeded, 0 failed and 3 skipped
Result: 1 file(s) with 3 test(s): 0 succeeded, 0 failed and 3 skipped
```
2 changes: 1 addition & 1 deletion selftest/complex/markdown-multi-command/test.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,6 @@ $ "${SCRUT_BIN}" test --match-markdown "*.mdtest" "$TESTDIR"/no-multiple-command
3 | - Bar
Summary: 1 file(s) with 1 test(s): 0 succeeded, 1 failed and 0 skipped
Result: 1 file(s) with 1 test(s): 0 succeeded, 1 failed and 0 skipped
[50]
```
4 changes: 2 additions & 2 deletions selftest/complex/prepend/test.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,13 +18,13 @@ $ "${SCRUT_BIN}" test --match-markdown "*.mdtest" "$TESTDIR/actual.mdtest"
1 | + Good things are undefined
Summary: 1 file(s) with 1 test(s): 0 succeeded, 1 failed and 0 skipped
Result: 1 file(s) with 1 test(s): 0 succeeded, 1 failed and 0 skipped
[50]
```

## Run test with prepend should make it succeed

```scrut
$ "${SCRUT_BIN}" test --match-markdown "*.mdtest" --prepend-test-file-paths "$TESTDIR/prepend.mdtest" "$TESTDIR/actual.mdtest"
Summary: 0 file(s) with 0 test(s): 0 succeeded, 0 failed and 0 skipped
Result: 0 file(s) with 0 test(s): 0 succeeded, 0 failed and 0 skipped
```
2 changes: 1 addition & 1 deletion selftest/complex/tailing-spaces/test.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,6 @@ $ $SCRUT_BIN test --match-markdown "*.mdtest" "$TESTDIR"/test.mdtest
1 | + foo\t (escaped) (equal)
Summary: 1 file(s) with 4 test(s): 1 succeeded, 3 failed and 0 skipped
Result: 1 file(s) with 4 test(s): 1 succeeded, 3 failed and 0 skipped
[50]
```
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ Command executes successfully

```scrut
$ "${SCRUT_BIN}" update --convert markdown test.t
Summary: 1 file(s) of which 1 updated, 0 skipped and 0 unchanged
Result: 1 file(s) of which 1 updated, 0 skipped and 0 unchanged
```

Converted Markdown test file looks like this:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ Hello World

```scrut
$ "${SCRUT_BIN}" update --convert cram test.md
Summary: 1 file(s) of which 1 updated, 0 skipped and 0 unchanged
Result: 1 file(s) of which 1 updated, 0 skipped and 0 unchanged
```

A new Cram file should have been created and the tests should look fine:
Expand Down
2 changes: 1 addition & 1 deletion src/bin/commands/update.rs
Original file line number Diff line number Diff line change
Expand Up @@ -354,7 +354,7 @@ impl Args {
}

fn render_summary(&self, updated: usize, skipped: usize, unchanged: usize) -> String {
let summary = "Summary".underline();
let summary = "Result".underline();
let total = updated + skipped + unchanged;
let files = format!("{} file(s)", total).bold();
let mut updated_fmt = format!("{} updated", updated).green();
Expand Down
4 changes: 2 additions & 2 deletions src/renderers/pretty.rs
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ pub struct PrettyColorRenderer {

impl PrettyColorRenderer {
fn render_summary(&self, files: usize, ok: usize, errors: usize, ignored: usize) -> String {
let summary = "Summary".underline();
let summary = "Result".underline();
let total = ok + errors + ignored;
let tests = format!("{} test(s)", total).bold();
let mut succeeded = format!("{} succeeded", ok).green();
Expand Down Expand Up @@ -406,7 +406,7 @@ mod tests {
}])
.expect("render succeeds");
assert_eq!(
"Summary: 1 file(s) with 1 test(s): 1 succeeded, 0 failed and 0 skipped\n", &rendered,
"Result: 1 file(s) with 1 test(s): 1 succeeded, 0 failed and 0 skipped\n", &rendered,
"success results are not rendered",
);
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,5 +20,4 @@ unexpected exit code
#> the stderr (no-eol)


Summary: 1 file(s) with 1 test(s): 0 succeeded, 1 failed and 0 skipped

Result: 1 file(s) with 1 test(s): 0 succeeded, 1 failed and 0 skipped
Original file line number Diff line number Diff line change
Expand Up @@ -13,5 +13,4 @@ expression: rendered
error: something failed


Summary: 0 file(s) with 1 test(s): 0 succeeded, 1 failed and 0 skipped

Result: 0 file(s) with 1 test(s): 0 succeeded, 1 failed and 0 skipped
Original file line number Diff line number Diff line change
Expand Up @@ -20,5 +20,4 @@ unexpected exit code
#> the stderr (no-eol)


Summary: 0 file(s) with 1 test(s): 0 succeeded, 1 failed and 0 skipped

Result: 0 file(s) with 1 test(s): 0 succeeded, 1 failed and 0 skipped
Original file line number Diff line number Diff line change
Expand Up @@ -20,5 +20,4 @@ unexpected exit code
#> the stderr (no-eol)


Summary: 1 file(s) with 1 test(s): 0 succeeded, 1 failed and 0 skipped

Result: 1 file(s) with 1 test(s): 0 succeeded, 1 failed and 0 skipped
Original file line number Diff line number Diff line change
Expand Up @@ -20,5 +20,4 @@ unexpected exit code
#> the stderr (no-eol)


Summary: 0 file(s) with 1 test(s): 0 succeeded, 1 failed and 0 skipped

Result: 0 file(s) with 1 test(s): 0 succeeded, 1 failed and 0 skipped
Original file line number Diff line number Diff line change
Expand Up @@ -20,5 +20,4 @@ unexpected exit code
#> the stderr (no-eol)


Summary: 1 file(s) with 1 test(s): 0 succeeded, 1 failed and 0 skipped

Result: 1 file(s) with 1 test(s): 0 succeeded, 1 failed and 0 skipped
Loading

0 comments on commit c6cf5ca

Please sign in to comment.