Cannot format file (black produced different code on the second pass of the formatter)

[rickystewart]

I'm unable to format some files with black version 20.8b1.

Steps to reproduce:

1. Use https://hg.mozilla.org/mozilla-central/raw-file/d4e1e62aed03db1d447f6b31b8d1347c2ad2e43b/testing/web-platform/tests/tools/ci/tests/test_jobs.py (we have other examples of similarly failing Python files in-tree, but this is the shortest example I was able to find)

2. Run black test_jobs.py

3. See error:

error: cannot format testing/web-platform/tests/tools/ci/tests/test_jobs.py: INTERNAL ERROR: Black produced different code on the second pass of the formatter.  Please report a bug on https://github.com/psf/black/issues.  This diff might be helpful: /tmp/blk_p0353uz5.log
Oh no! 💥 💔 💥
1 file failed to reformat.

Expected behavior: No failure to reformat.

Environment:

• Version: 20.8b1 (installed with Pip)
• OS and Python version: Ubuntu 20.04.1 LTS, Python 3.8.5

This bug also occurs on master (tested on https://black.now.sh/?version=master)

[hugovk]

Here's a smaller repro with latest master (black, version 20.8b2.dev36+g4070527):

def test_stability():
    assert jobs.get_jobs(["css/build-css-testsuite.sh",
                          "css/CSS21/test-001.html"],
                         includes=["stability"]) == {"stability"}

Log:

Mode(target_versions={<TargetVersion.PY38: 8>, <TargetVersion.PY37: 7>, <TargetVersion.PY36: 6>}, line_length=88, string_normalization=True, experimental_string_processing=False, is_pyi=False)
--- source
+++ first pass
@@ -1,4 +1,5 @@
 def test_stability():
-    assert jobs.get_jobs(["css/build-css-testsuite.sh",
-                          "css/CSS21/test-001.html"],
-                         includes=["stability"]) == {"stability"}
+    assert jobs.get_jobs(
+        ["css/build-css-testsuite.sh", "css/CSS21/test-001.html"],
+        includes=["stability"],
+    ) == {"stability"}
--- first pass
+++ second pass
@@ -1,5 +1,8 @@
 def test_stability():
-    assert jobs.get_jobs(
-        ["css/build-css-testsuite.sh", "css/CSS21/test-001.html"],
-        includes=["stability"],
-    ) == {"stability"}
+    assert (
+        jobs.get_jobs(
+            ["css/build-css-testsuite.sh", "css/CSS21/test-001.html"],
+            includes=["stability"],
+        )
+        == {"stability"}
+    )

[globau]

This is caused by the changes in 586d242 (cc @ambv)

Amusingly the commit message calls out this exact problem as a pain point:

This form of user-controlled formatting is brittle so we have to be careful not to cause a scenario where Black first formats code without trailing commas in one way, and then looks at the same file with pre-existing trailing commas (that it itself put on the previous run) and decides to format the code again.

[jli]

I have what seems like a similar failure case. Occurs with version 20.8b1 from pip, also happens on https://black.now.sh/?version=master. OS X, Python 3.8.6.

Apologies if it's not useful, but just in case it is:

def test_do_a_thing():
    assert "belongs_to" in do_a_thing({
        "series": {"unitId": 4},
        "data": ["2001-01-01", "2001-12-31"]
    }, add_belongs_to=True)[0]

Gives the error: INTERNAL ERROR: Black produced different code on the second pass of the formatter.
Here's the diff log:

Mode(target_versions=set(), line_length=88, string_normalization=True, experimental_string_processing=False, is_pyi=False)
--- source
+++ first pass
@@ -1,5 +1,8 @@
 def test_do_a_thing():
-    assert "belongs_to" in do_a_thing({
-        "series": {"unitId": 4},
-        "data": ["2001-01-01", "2001-12-31"]
-    }, add_belongs_to=True)[0]
+    assert (
+        "belongs_to"
+        in do_a_thing(
+            {"series": {"unitId": 4}, "data": ["2001-01-01", "2001-12-31"]},
+            add_belongs_to=True,
+        )[0]
+    )
--- first pass
+++ second pass
@@ -1,8 +1,5 @@
 def test_do_a_thing():
-    assert (
-        "belongs_to"
-        in do_a_thing(
-            {"series": {"unitId": 4}, "data": ["2001-01-01", "2001-12-31"]},
-            add_belongs_to=True,
-        )[0]
-    )
+    assert "belongs_to" in do_a_thing(
+        {"series": {"unitId": 4}, "data": ["2001-01-01", "2001-12-31"]},
+        add_belongs_to=True,
+    )[0]

This one seems pretty sensitive. Black will succeed if different parts of this example are changed: removing the final [0], removing the add_belongs_to kwarg, removing either series or data entry in the dict all result in avoiding this error.

[ciknight]

I appeared the same problem, which version can work now？

[ichard26]

Black version 19.10b0 may work, no guarantees though.

[ciknight]

Black version 19.10b0 may work, no guarantees though.

Black version 19.10b0 is work, thanks.

[ichard26]

Hello!

All reproduction cases in this issue format without error on master. The fixing commit was 8672af3 from PR GH-2126. I'll be marking this issue as a duplicate of GH-1629 since that's what GH-2126 aimed to fix and it's highly likely this issue falls under GH-1629.

Since we use the issue tracker as a reflection of what's on master, I'll be closing this issue. If you have any issues, especially with the new (but stable) output, please open a new issue. Oh and the fix should be available in a published release soon, see GH-2125 for more info.

Thank you for reporting!