Black crashes

[hallazzang]

Describe the bug
Black crashes with an error message.

To Reproduce

1. Take this file(test.py):

foo = (1, 2, 3,)
bar = (
    (
        1,
        2,
        3,
    )
)

2. Run Black on it with these arguments:

$ black --diff test.py

3. See error:

error: cannot format test.py: INTERNAL ERROR: Black produced different code on the second pass of the formatter.  Please report a bug on https://github.com/psf/black/issues.  This diff might be helpful: /tmp/blk_7zvxucd6.log
Oh no! 💥 💔 💥
1 file failed to reformat.

Expected behavior
Black should reformat the code like this:

foo = (1, 2, 3)
bar = (1, 2, 3)

Environment (please complete the following information):

• Version: black, version 19.10b1.dev5+gde6f4b1
• OS and Python version: Ubuntu 18.04.3 LTS, Python 3.8.0

Does this bug also happen on master?
Yes. It happens when I clone this repository(master branch) and install it manually(python setup.py test fails with a NameError though).
But in online version, it doesn't crash but this time it produces weird result.
Given the same source input, output looks like this:

foo = (
    1,
    2,
    3,
)
bar = (1, 2, 3,)

Then if I use this output as input again, this output pops out:

foo = (
    1,
    2,
    3,
)
bar = (
    1,
    2,
    3,
)

Additional context Add any other context about the problem here.
Log message(from /tmp/blk_7zvxucd6.log):

--- source
+++ first pass
@@ -1,8 +1,7 @@
-foo = (1, 2, 3,)
-bar = (
-    (
-        1,
-        2,
-        3,
-    )
+foo = (
+    1,
+    2,
+    3,
 )
+bar = (1, 2, 3,)
+
--- first pass
+++ second pass
@@ -1,7 +1,11 @@
 foo = (
     1,
     2,
     3,
 )
-bar = (1, 2, 3,)
+bar = (
+    1,
+    2,
+    3,
+)

[JelleZijlstra]

Thanks for the report! This is probably a result of #826.

Could you give more details on the NameError you see when running the tests though?

[hallazzang]

@JelleZijlstra

Here's full log:

(venv) $ python setup.py test
/usr/lib/python3.8/distutils/dist.py:274: UserWarning: Unknown distribution option: 'long_description_content_type'
  warnings.warn(msg)
running test
running egg_info
writing black.egg-info/PKG-INFO
writing dependency_links to black.egg-info/dependency_links.txt
writing entry points to black.egg-info/entry_points.txt
writing requirements to black.egg-info/requires.txt
writing top-level names to black.egg-info/top_level.txt
reading manifest template 'MANIFEST.in'
warning: no files found matching '*.rst'
warning: no files found matching '*.txt' under directory 'tests'
writing manifest file 'black.egg-info/SOURCES.txt'
running build_ext
Traceback (most recent call last):
  File "setup.py", line 46, in <module>
    setup(
  File "/home/ubuntu/workspace/tmp/venv/lib/python3.8/site-packages/setuptools/__init__.py", line 129, in setup
    return distutils.core.setup(**attrs)
  File "/usr/lib/python3.8/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/usr/lib/python3.8/distutils/dist.py", line 966, in run_commands
    self.run_command(cmd)
  File "/usr/lib/python3.8/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/home/ubuntu/workspace/tmp/venv/lib/python3.8/site-packages/setuptools/command/test.py", line 226, in run
    self.run_tests()
  File "/home/ubuntu/workspace/tmp/venv/lib/python3.8/site-packages/setuptools/command/test.py", line 244, in run_tests
    test = unittest.main(
  File "/usr/lib/python3.8/unittest/main.py", line 100, in __init__
    self.parseArgs(argv)
  File "/usr/lib/python3.8/unittest/main.py", line 147, in parseArgs
    self.createTests()
  File "/usr/lib/python3.8/unittest/main.py", line 158, in createTests
    self.test = self.testLoader.loadTestsFromNames(self.testNames,
  File "/usr/lib/python3.8/unittest/loader.py", line 220, in loadTestsFromNames
    suites = [self.loadTestsFromName(name, module) for name in names]
  File "/usr/lib/python3.8/unittest/loader.py", line 220, in <listcomp>
    suites = [self.loadTestsFromName(name, module) for name in names]
  File "/usr/lib/python3.8/unittest/loader.py", line 154, in loadTestsFromName
    module = __import__(module_name)
  File "/home/ubuntu/workspace/tmp/black/tests/test_black.py", line 1618, in <module>
    class BlackDTestCase(AioHTTPTestCase):
NameError: name 'AioHTTPTestCase' is not defined

I opened tests/test_black.py and found this:

try:
    import blackd
    from aiohttp.test_utils import AioHTTPTestCase, unittest_run_loop
    from aiohttp import web
except ImportError:
    has_blackd_deps = False
else:
    has_blackd_deps = True

[vemel]

I am working on this, I also had this issue.

[vemel]

Fixed in my PR

[ichard26]

This has been fixed in the latest release, version 20.8b1:

R:\Programming>type black\temp.py
foo = (1, 2, 3,)
bar = (
    (
        1,
        2,
        3,
    )
)
R:\Programming>black black/temp.py --diff --color
--- black\temp.py       2020-09-04 21:27:53.309225 +0000
+++ black\temp.py       2020-09-04 21:30:38.473714 +0000
@@ -1,8 +1,10 @@
-foo = (1, 2, 3,)
+foo = (
+    1,
+    2,
+    3,
+)
 bar = (
-    (
-        1,
-        2,
-        3,
-    )
+    1,
+    2,
+    3,
 )
would reformat black\temp.py
All done! ✨ 🍰 ✨
1 file would be reformatted.

My environment

• Black version: 20.8b1
• Python version: CPython 3.8.1
• OS version: Windows 10 Home Edition Build 18363

The reason why the collection literals are still exploded is the "magic trailing comma" feature. This feature was introduced in 19.10b0 and then was fixed / improved in 20.8b0 and 20.8b1 to make it work properly so it can be generally usable. To learn more, please read its documentation.

Thank you @hallazzang for reporting this bug!