The unittest module diff is missing/forgetting/not putting newline before + and ? for some inputs #79868

addonszz · 2019-01-08T21:57:48Z

BPO	35687
Nosy	@pfmoore, @rbtcollins, @tjguk, @ned-deily, @ezio-melotti, @voidspace, @zware, @tirkarthi, @addonszz

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2019-01-08.21:57:48.034>
labels = ['type-bug', 'tests', 'OS-windows']
title = 'The unittest module diff is missing/forgetting/not putting newline before + and ? for some inputs'
updated_at = <Date 2019-01-09.11:04:13.571>
user = 'https://github.com/addonszz'

bugs.python.org fields:

activity = <Date 2019-01-09.11:04:13.571>
actor = 'addons_zz'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Tests', 'Windows']
creation = <Date 2019-01-08.21:57:48.034>
creator = 'addons_zz'
dependencies = []
files = []
hgrepos = []
issue_num = 35687
keywords = []
message_count = 8.0
messages = ['333258', '333261', '333265', '333272', '333273', '333298', '333299', '333300']
nosy_count = 9.0
nosy_names = ['paul.moore', 'rbcollins', 'tim.golden', 'ned.deily', 'ezio.melotti', 'michael.foord', 'zach.ware', 'xtreak', 'addons_zz']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue35687'
versions = ['Python 3.6']

addonszz · 2019-01-08T21:57:48Z

Create this program and run with Python 3.6.3:

import unittest

class StdErrUnitTests(unittest.TestCase):

    def test_function_name(self):
        expected = "testing.main_unit_tests.test_dictionaryBasicLogging:416 - dictionary\n" \
                "testing.main_unit_tests.test_dictionaryBasicLogging:417 - dictionary {1: 'defined_chunk'}"

        actual = "15:49:35:912.348986 - testing.main_unit_tests - dictionary\n" \
                "15:49:35:918.879986 - testing.main_unit_tests - dictionary {1: 'defined_chunk'}"

        self.assertEqual(expected, actual)

if __name__ == '__main__':
    unittest.main()

### Actual output

F
======================================================================
FAIL: test_function_name (__main__.StdErrUnitTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\bug_assert_unittests.py", line 13, in test_function_name
    self.assertEqual(expected, actual)
AssertionError: "testing.main_unit_tests.test_dictionaryBa[114 chars]nk'}" != "15:49:35:912.348986 - testing.main_unit_t[94 chars]nk'}"
- testing.main_unit_tests.test_dictionaryBasicLogging:416 - dictionary
- testing.main_unit_tests.test_dictionaryBasicLogging:417 - dictionary {1: 'defined_chunk'}+ 15:49:35:912.348986 - testing.main_unit_tests - dictionary
+ 15:49:35:918.879986 - testing.main_unit_tests - dictionary {1: 'defined_chunk'}

----------------------------------------------------------------------
Ran 1 test in 0.001s

FAILED (failures=1)

### Expected output

F
======================================================================
FAIL: test_function_name (__main__.StdErrUnitTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\bug_assert_unittests.py", line 13, in test_function_name
    self.assertEqual(expected, actual)
AssertionError: "testing.main_unit_tests.test_dictionaryBa[114 chars]nk'}" != "15:49:35:912.348986 - testing.main_unit_t[94 chars]nk'}"
- testing.main_unit_tests.test_dictionaryBasicLogging:416 - dictionary
- testing.main_unit_tests.test_dictionaryBasicLogging:417 - dictionary {1: 'defined_chunk'}
+ 15:49:35:912.348986 - testing.main_unit_tests - dictionary
+ 15:49:35:918.879986 - testing.main_unit_tests - dictionary {1: 'defined_chunk'}

----------------------------------------------------------------------
Ran 1 test in 0.001s

FAILED (failures=1)

### Differences between actual and expected output

@@ -7,7 +7,8 @@
     self.assertEqual(expected, actual)
 AssertionError: "testing.main_unit_tests.test_dictionaryBa[114 chars]nk'}" != "15:49:35:912.348986 - testing.main_unit_t[94 chars]nk'}"
 - testing.main_unit_tests.test_dictionaryBasicLogging:416 - dictionary
-- testing.main_unit_tests.test_dictionaryBasicLogging:417 - dictionary {1: 'defined_chunk'}+ 15:49:35:912.348986 - testing.main_unit_tests - dictionary
+- testing.main_unit_tests.test_dictionaryBasicLogging:417 - dictionary {1: 'defined_chunk'}
++ 15:49:35:912.348986 - testing.main_unit_tests - dictionary
 + 15:49:35:918.879986 - testing.main_unit_tests - dictionary {1: 'defined_chunk'}

This kind of bug frequently happens. On this case, it is not putting a new line before the +. Other cases, it is not putting a new line before the ?.

zooba · 2019-01-08T22:38:50Z

This doesn't appear to be Windows-specific or related to our test suite, so I updated the tags and added the unittest experts.

ned-deily · 2019-01-08T23:06:59Z

Also, if you can, please verify that you can reproduce the problem with a current version of Python 3. 3.7.2 is the current maintenance release and 3.6.8 was the final maintenance release of 3.6.x.

tirkarthi · 2019-01-09T01:44:43Z

This seems to exist on master. Since they are multilines assertMultiLineEqual is used and is there something incorrect during the diff calculation using ndiff due to absence of '\n'? The current diff is calculated as below with difflib.ndiff and seems to produce incorrect output due to absence of newline. The last change was done for single string comparison with bpo-9174.

difflib.ndiff calculation done internally for the below reproducer

$ ./python.exe
Python 3.8.0a0 (heads/master:a234e14839, Jan  8 2019, 21:57:35)
[Clang 7.0.2 (clang-700.1.81)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import difflib
>>> print(''.join(difflib.ndiff(['a\n', 'b'], ['c\n', 'd'])))
- a
- b+ c
+ d
>>> print(''.join(difflib.ndiff(['a\n', 'b\n'], ['c\n', 'd\n']))) # Possible correct candidate?
- a
- b
+ c
+ d

A simpler reproducer

import unittest

class StdErrUnitTests(unittest.TestCase):

    def test_function_name(self):
        expected = "a\nb"
        actual = "c\nd"

        self.assertEqual(expected, actual)

    def test_function_name_newlines_end(self):
        expected = "a\nb\n"
        actual = "c\nd\n"

        self.assertEqual(expected, actual) # produces extra new line at the diff in the end with \d\n


if __name__ == '__main__':
    unittest.main()

$ ./python.exe ../backups/bpo35687_1.py
FF

======================================================================
FAIL: test_function_name (main.StdErrUnitTests)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "../backups/bpo35687_1.py", line 9, in test_function_name
    self.assertEqual(expected, actual)
AssertionError: 'a\nb' != 'c\nd'
- a
- b+ c
+ d

======================================================================
FAIL: test_function_name_newlines_end (main.StdErrUnitTests)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "../backups/bpo35687_1.py", line 15, in test_function_name_newlines_end
    self.assertEqual(expected, actual)
AssertionError: 'a\nb\n' != 'c\nd\n'
- a
- b
+ c
+ d

Ran 2 tests in 0.003s

FAILED (failures=2)

tirkarthi · 2019-01-09T01:52:44Z

Searching further this seems to be reported earlier with bpo-24780 with caret displayed wrongly which was also due to newline missing. But the sample case reported here is also listed at one of the examples in https://bugs.python.org/issue24780#msg258466 . There is an open patch with review in the issue. I guess this is a duplicate of bpo-24780 then?

Related sample case at msg258466

======================================================================
FAIL: test_notrailingnewline_2 (main.AssertEqualTest)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "test.py", line 18, in test_notrailingnewline_2
    self.assertEqual("a\nbcdf", "a\nbddg")
AssertionError: 'a\nbcdf' != 'a\nbddg'
  a
- bcdf+ bddg

addonszz · 2019-01-09T10:50:34Z

The issue is also reproducible, by directly using difflib:

import difflib

def bugged_diff(expected, actual):
    expected = expected.splitlines( 1 )
    actual = actual.splitlines( 1 )

    # diff = difflib.ndiff( expected, actual )
    if expected != actual:
        diff = difflib.context_diff( expected, actual, fromfile='expected input', tofile='actual output', lineterm='\n' )
        return '\n' + ''.join( diff )


if __name__ == '__main__':
    expected = "testing.main_unit_tests.test_dictionaryBasicLogging:416 - dictionary\n" \
            "testing.main_unit_tests.test_dictionaryBasicLogging:417 - dictionary {1: 'defined_chunk'}"

    actual = "15:49:35:912.348986 - testing.main_unit_tests - dictionary\n" \
            "15:49:35:918.879986 - testing.main_unit_tests - dictionary {1: 'defined_chunk'}"

    print( expected, actual )

It outputs:

testing.main_unit_tests.test_dictionaryBasicLogging:416 - dictionary
testing.main_unit_tests.test_dictionaryBasicLogging:417 - dictionary {1: 'defined_chunk'} 15:49:35:912.348986 - testing.main_unit_tests - dictionary
15:49:35:918.879986 - testing.main_unit_tests - dictionary {1: 'defined_chunk'}

But it should be: (still missing the new line on the same place as assertEqual does)

testing.main_unit_tests.test_dictionaryBasicLogging:416 - dictionary
testing.main_unit_tests.test_dictionaryBasicLogging:417 - dictionary {1: 'defined_chunk'} 
15:49:35:912.348986 - testing.main_unit_tests - dictionary
15:49:35:918.879986 - testing.main_unit_tests - dictionary {1: 'defined_chunk'}

addonszz · 2019-01-09T10:56:09Z

Correction

import difflib

def bugged_diff(expected, actual):
    expected = expected.splitlines( 1 )
    actual = actual.splitlines( 1 )

    # diff = difflib.ndiff( expected, actual )
    if expected != actual:
        diff = difflib.context_diff( expected, actual, fromfile='expected input', tofile='actual output', lineterm='\n' )
        return '\n' + ''.join( diff )


if __name__ == '__main__':
    expected = "testing.main_unit_tests.test_dictionaryBasicLogging:416 - dictionary\n" \
            "testing.main_unit_tests.test_dictionaryBasicLogging:417 - dictionary {1: 'defined_chunk'}"

    actual = "15:49:35:912.348986 - testing.main_unit_tests - dictionary\n" \
            "15:49:35:918.879986 - testing.main_unit_tests - dictionary {1: 'defined_chunk'}"

    print( bugged_diff( expected, actual ) )

Outputs:

*** expected input
--- actual output
***************
*** 1,2 ****
! testing.main_unit_tests.test_dictionaryBasicLogging:416 - dictionary
! testing.main_unit_tests.test_dictionaryBasicLogging:417 - dictionary {1: 'defined_chunk'}--- 1,2 ----
! 15:49:35:912.348986 - testing.main_unit_tests - dictionary
! 15:49:35:918.879986 - testing.main_unit_tests - dictionary {1: 'defined_chunk'}

May be it should be:


*** expected input
--- actual output
***************
*** 1,2 ****
! testing.main_unit_tests.test_dictionaryBasicLogging:416 - dictionary
! testing.main_unit_tests.test_dictionaryBasicLogging:417 - dictionary {1: 'defined_chunk'}
--- 1,2 ----
! 15:49:35:912.348986 - testing.main_unit_tests - dictionary
! 15:49:35:918.879986 - testing.main_unit_tests - dictionary {1: 'defined_chunk'}

addonszz · 2019-01-09T11:04:14Z

I applied the patch from https://bugs.python.org/file44679 and it fixed the issue for this example using the unittest module:

import unittest

class StdErrUnitTests(unittest.TestCase):

    def test_function_name(self):
        expected = "testing.main_unit_tests.test_dictionaryBasicLogging:416 - dictionary\n" \
                "testing.main_unit_tests.test_dictionaryBasicLogging:417 - dictionary {1: 'defined_chunk'}"

        actual = "15:49:35:912.348986 - testing.main_unit_tests - dictionary\n" \
                "15:49:35:918.879986 - testing.main_unit_tests - dictionary {1: 'defined_chunk'}"

        self.assertEqual(expected, actual)

if __name__ == '__main__':
    unittest.main()

evandrocoan · 2022-12-09T01:03:26Z

Related to: #68968

addonszz mannequin added tests Tests in the Lib/test dir OS-windows type-bug An unexpected behavior, bug, or error labels Jan 8, 2019

ezio-melotti transferred this issue from another repository Apr 10, 2022

serhiy-storchaka added this to Unittest issues May 20, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

The unittest module diff is missing/forgetting/not putting newline before + and ? for some inputs #79868

The unittest module diff is missing/forgetting/not putting newline before + and ? for some inputs #79868

addonszz mannequin commented Jan 8, 2019

addonszz mannequin commented Jan 8, 2019

zooba commented Jan 8, 2019

ned-deily commented Jan 8, 2019

tirkarthi commented Jan 9, 2019

tirkarthi commented Jan 9, 2019

addonszz mannequin commented Jan 9, 2019

addonszz mannequin commented Jan 9, 2019

addonszz mannequin commented Jan 9, 2019

evandrocoan commented Dec 9, 2022

The unittest module diff is missing/forgetting/not putting newline before + and ? for some inputs #79868

The unittest module diff is missing/forgetting/not putting newline before + and ? for some inputs #79868

Comments

addonszz mannequin commented Jan 8, 2019

addonszz mannequin commented Jan 8, 2019

zooba commented Jan 8, 2019

ned-deily commented Jan 8, 2019

tirkarthi commented Jan 9, 2019

tirkarthi commented Jan 9, 2019

addonszz mannequin commented Jan 9, 2019

addonszz mannequin commented Jan 9, 2019

addonszz mannequin commented Jan 9, 2019

evandrocoan commented Dec 9, 2022