Diff is not generated for assert.deepEqual(obj, undefined, message) with non-empty message

[Antonius-S]

Prerequisites

• [x ] Checked that your issue hasn't already been filed by cross-referencing issues with the faq label
• [ x] Checked next-gen ES issues and syntax problems by using the same environment and/or transpiler configuration without Mocha to ensure it isn't just a feature that actually isn't supported in the environment in question or a bug in your code.
• [x ] 'Smoke tested' the code to be tested by running it outside the real test suite to get a better sense of whether the problem is in the code under test, your usage of Mocha, or Mocha itself
• [x ] Ensured that there is no discrepancy between the locally and globally installed versions of Mocha. You can find them with: node node_modules/.bin/mocha --version(Local) and mocha --version(Global). We recommend avoiding the use of globally installed Mocha.

Description

Steps to Reproduce

'use strict';
const assert = require('assert');

suite('suite', function()
{
	test('Diff OK', function()
	{
		assert.deepStrictEqual({foo: "bar"}, {a:1}, 'msg');
	});

	test('Diff OK', function()
	{
		assert.deepStrictEqual({foo: "bar"}, undefined);
	});

	test('Diff OK', function()
	{
		assert.deepStrictEqual(undefined, {foo: "bar"});
	});

	test('No Diff', function()
	{
		assert.deepStrictEqual({foo: "bar"}, undefined, 'msg');
	});

	test('No Diff', function()
	{
		assert.deepStrictEqual(undefined, {foo: "bar"}, 'msg');
	});
});

Expected behavior: Expect diff generated for all cases

Actual behavior: Diff not generated if message is non-empty

  suite
    1) Diff OK
    2) Diff OK
    3) Diff OK
    4) No Diff
    5) No Diff


  0 passing (21ms)
  5 failing

  1) suite
       Diff OK:

      AssertionError [ERR_ASSERTION]: msg
      + expected - actual

       {
      -  "foo": "bar"
      +  "a": 1
       }

      at Context.<anonymous> (test-eq.js:8:10)

  2) suite
       Diff OK:
     AssertionError [ERR_ASSERTION]: Input A expected to strictly deep-equal inp
ut B:
+ expected - actual

- {
-   foo: 'bar'
- }
+ undefined
      at Context.<anonymous> (test-eq.js:13:10)

  3) suite
       Diff OK:
     AssertionError [ERR_ASSERTION]: Input A expected to strictly deep-equal inp
ut B:
+ expected - actual

- undefined
+ {
+   foo: 'bar'
+ }
      at Context.<anonymous> (test-eq.js:18:10)

  4) suite
       No Diff:
     AssertionError [ERR_ASSERTION]: msg
      at Context.<anonymous> (test-eq.js:23:10)

  5) suite
       No Diff:
     AssertionError [ERR_ASSERTION]: msg
      at Context.<anonymous> (test-eq.js:28:10)

Reproduces how often: always

Versions

• The output of mocha --version and node node_modules/.bin/mocha --version: 6.0.0-1
• The output of node --version: v10.15.1
• The version and architecture of your operating system: Win7 x64
• Your shell (bash, zsh, PowerShell, cmd, etc.): cmd
• Your browser and version (if running browser tests):
• Any other third party Mocha related modules (with versions):
• The code transpiler being used:

Additional Information

[craigtaub]

@Antonius-S
Mocha has logic to not generate a diff if the expectation is undefined (18d2e05).

So I believe this is a question for the assert module.
Mocha makes use of the thrown error.message for its output, and as you can see below the message that it returns isnt consistent.

Run as node test-assert.js

const assert = require('assert');

console.log('With message')
try {
  assert.deepStrictEqual({ foo: "bar" }, undefined, 'msg');
} catch (e) {
  console.log('message: ', e.message)
}
console.log('-----')
console.log('Without message')
try {
  assert.deepStrictEqual({ foo: "bar" }, undefined);
} catch (e) {
  console.log('message: ', e.message)
}

Results:

With message
message:  msg
-----
Without message
message:  Input A expected to strictly deep-equal input B:
+ expected - actual
- {
-   foo: 'bar'
- }
+ undefined

Basically Mocha doesn't and shouldn't generate the diff, someone else does here tho.

[Antonius-S]

Well, you're partially right. Diff in your example is generated by Node and seems no printing diff with message defined it is by design:

If the values are not equal, an AssertionError is thrown with a message property set equal to the value of the message parameter. If the message parameter is undefined, a default error message is assigned.

Nevertheless, there's one incontinence. While bare assert.deepStrictEqual({ foo: "bar" }, { baz: "foo" }, 'msg'); will not generate diff, the mochified check will. Anyway functions unifiedDiff and inlineDiff in \lib\reporters\base.js are meant to do something :)

[craigtaub]

While bare assert.deepStrictEqual({ foo: "bar" }, { baz: "foo" }, 'msg'); will not generate diff, the mochified check will.

So in this scenario it passes criteria for Mocha to generate its own diff.

[juergba]

1. test: outputs only Mocha's diff
2. test: only Node's AssertionError diff, no Mocha diff
3. test: only Node's AssertionError diff, no Mocha diff
4. + 5. test: no diff at all, neither Mocha nor Node

I will investigate further.

[juergba]

The output is consistent and not the result of a bug.

Node's diff:

message <string>: If provided, the error message is going to be set to this value.

As mentionned above err.message does not include a diff when you pass your own error message.
Therefore only 2. and 3. test output a Node's diff.

Mocha's diff:
Mocha does not generate a diff if the err.expected is undefined and/or err.actual/err.expected are of different type. Therefore only the 1. test shows a Mocha's diff.

Mocha supports various third party assertion libraries, each of them can differ in its diff logic. Mocha generates its own diff independently of the assertion library used.

[Antonius-S]

2 questions I still have
1 - why Node doesn't generate diff when a message defned
2 - why mocha doesn't generate diff when err.expected is undefined and/or err.actual/err.expected are of different type

[craigtaub]

1 - why Node doesn't generate diff when a message defned

If there is a message and acceptable expected/actual value, a diff will be generated by mocha.

2 - why mocha doesn't generate diff when err.expected is undefined and/or err.actual/err.expected are of different type

• expected is undefined reason => 18d2e05
• different types reason => Don't diff different object types #900