Support Python 3.10 pattern matching syntax

[tirkarthi]

Is your feature request related to a problem? Please describe.

Python 3.10 has added support for pattern matching. It will be good to support it. https://www.python.org/dev/peps/pep-0622/

Describe the solution you'd like .

Describe alternatives you've considered

Reproducer

from dataclasses import dataclass

@dataclass
class Person:
    name: str

person = Person("Kate")

match person:
    case Person(name):
        print(f"Found {name}")
    case _:
        print("Not found")

 black /tmp/dataclass_match.py
error: cannot format /tmp/dataclass_match.py: Cannot parse: 10:6: match person:
Oh no! 💥 💔 💥
1 file failed to reformat.

Additional context Add any other context or screenshots about the feature request
here.

[brandtbucher]

Note that the syntax has changed slightly since PEP 622 was superseded. PEP 634 (the accepted PEP) has the updated grammar.

I'm happy to help review or answer any questions on this once the work begins. I could probably also find time to help out with the implementation, if needed (probably can't take on the whole parser refactor though). I've contributed a small amount of code to Black in the past.

Definitely looking forward to this support!

[JelleZijlstra]

This will probably require #2318.

[kkirsche]

Is there anything needed from the community to support this following the release of Python 3.10?

I apologize if asking this is inappropriate given the status update on Jun 9th from @JelleZijlstra

Thank you for the hard work on Black.

[LordOfPolls]

May I ask if there's any news in regards to this?

[graingert]

Is there anything needed from the community to support this following the release of Python 3.10?

I apologize if asking this is inappropriate given the status update on Jun 9th from @JelleZijlstra

Thank you for the hard work on Black.

I think here is the latest stuff https://docs.google.com/document/d/1QPqm01E7MIRi_l4jrgCGOFVThP_HINnkm4hmVZC_CSg/edit#heading=h.qokppq9m1uzy

(from #2318 )

and here Instagram/LibCST#285 (comment)

you can follow along the work here Instagram/LibCST@main...zsol:parser

[prutheus]

I do not understand all comments to this issue, but I wonder if there is a branch or something already in which black can handle match keywords or is there no workaround yet? (I can not commit to big code bases anymore cause black is crashing with match keywords)

[kennipj]

I spent some time this morning looking for a temporary workaround until a permanent solution is found.

I was able to add limited support to pattern matching, and I think most of the syntax should be supported with my changes, however some more advanced syntax will still crash black.

I have retained the use of match and case as non-keywords, as specified by PEP634, so you should still be able to use match and case as variable names, functions etc. I can't guarantee I caught all cases with my quick hack though, but should work for most cases.

This being a temporary workaround, I can't guarantee that the formatting of some expressions in match/case statements to look how you'd expect (pattern matching syntax can get pretty complex). The primary purpose here was to have black not crash on seeing match/case statements. Although I did work with a couple toy examples, and it usually does a pretty good job.

Anyhow, here is the branch with the limited pattern matching support: https://github.com/argyle-engineering/black/tree/add-minimal-support-for-pattern-matching. Have a look at argyle-engineering#1 for the PR that adds this functionality (also has some context in the body). You can target hash c332b48 directly for the changes mentioned here ie. pip install git+https://github.com/argyle-engineering/black.git@c332b48

Some examples of syntax that this fork supports (taken from PEP636):

match point:
    case Point(x, y) if x == y:
        print(f"Y=X at {x}")
    case Point(x, y):
        print(f"Not on the diagonal")

match command.split():
    case ["go", direction] if direction in current_room.exits:
        current_room = current_room.neighbor(direction)
    case ["go", _]:
        print("Sorry, you can't go that way")

match command.split():
    case ["drop", *objects]:
        for obj in objects:
            character.drop(obj, current_room)

match command.split():
    case ["north"] | ["go", "north"]:
        current_room = current_room.neighbor("north")
    case ["get", obj] | ["pick", "up", obj] | ["pick", obj, "up"]:
        ...  # Code for picking up the given object

Example of syntax not supported:

match command.split():
    case ["go", ("north" | "south" | "east" | "west") as direction]:
        current_room = current_room.neighbor(direction)

I do not intend to make a PR to this repository here, as this is a quick workaround that won't be able to cover every pattern matching use case.

[JelleZijlstra]

Thank you! I heard that @isidentical also came up with a lib2to3 patch that works for most but not all match syntax.

I think we should just go with one of your patches to unblock users from using most match syntax, fix bugs as they come up, and hopefully switch to a more powerful parser in the future.

[isidentical]

Great solution @kennipj!

Example of syntax not supported:

I also had the same problem, so I've simply added <blabla> as <blabla> as a real expression (even though outside of certain contexts, e.g a case statement or with statements) it is not valid. That also allowed parenthesized with statements (which is a new syntax that also depends on the PEG grammar).

I heard that @isidentical also came up with a lib2to3 patch that works for most but not all match syntax.

Here is my implementation for people who are interested: isidentical@c22fca6.

For testing, I also wrote a couple of scripts which can determine how much of the stuff black can actually parse: https://gist.github.com/isidentical/34f7d90e0b0d59d2cc32cc32d58d4f2e.

Currently it parses 315/322 test cases out there (2 fails when case case or case match (probably fixable), and 5 fails about blabla as blabla (not sure why)).

[brandtbucher]

@isidentical might already be using it, but I've written a few thousand lines of crazy pattern matching code for Python's regression test suite. It can probably help stress your implementation and shake out any edge cases!

[kennipj]

I've written a few thousand lines of crazy pattern matching code for Python's regression test suite.

Ah I should have known about this when I started, very useful.

Running @isidentical's test, results in 245/324 test cases parsed (79 failures) with my solution. Your solution is clearly more robust 😉

I probably won't address the failing cases, as it will take considerable to cover the remaining cases, and seeing as @isidentical has a more mature solution, it would probably be better to use that instead.

EDIT: Took a quick look at some of the failures, and with a couple quick fixes, managed to get it down to just 19 tests failing. All cases with as or tests like case case: as mentioned above

[LordOfPolls]

So, re the above. Does this mean black will be getting support for the new syntax shortly? Or are we to switch to using one of the forks?

[JelleZijlstra]

What I'd like to see is that we land support based on @isidentical's patch, but somebody needs to step up to actually make a PR and get it landed. I might work on that if I have time, but I can't give any guarantees.

[isidentical]

I would love to contribute @JelleZijlstra! If the lack of full support and other issues I have mentioned is considered and agreed upon I can convert my PR to a patch. And iterate over it for better support.

[JelleZijlstra]

That would be great! I'd be OK with landing partial support first and iterating on the rest later.

[isidentical]

Cool, will send a patch within this week!

[JelleZijlstra]

I am going to close this because thanks to @isidentical's awesome work we shipped 21.11b1 with basic match support. There may be some cases that don't work, but we should open new issues for those as we run into them.

[AlozoR]

Got the last version and it still fails to parse the match keyword on my end, with the exact same example as the OP, any ideas?
EDIT: My bad, had to explicitly state -t py310

[JelleZijlstra]

@AlozoR make sure you pass --target-version py310 explicitly. We don't use the match grammar by default.

[AlozoR]

Just saw that! Thanks!

[g-berthiaume]

FYI:
If you're using vscode, you can add the following to your settings.json

"python.formatting.blackArgs": [
    "--target-version",
    "py310"
],

[AdriandLiu]

FYI:
If you're using Atom, you can follow this step to point python-black to find the right black:

• Make sure you installed the latest black, doesn't matter which env
• Go to .atom -> packages -> python-black -> lib -> python-black.js -> binPath -> default, change it to the path get from which black
• Restart Atom and try it again

[rgant]

With the online formatter match-case doesn't work unless you turn on "skip temporary sanity checks" even when targeting py3.10.

https://black.vercel.app/?version=stable&state=_Td6WFoAAATm1rRGAgAhARYAAAB0L-Wj4AU_As5dAD2IimZxl1N_WlkPinBFoXIfdFTaTVkGVeHShArYj9yPlDvwBA7LhGo8BvRQqDilPtgsfdKl-ha7EFp0Ma6lY_06IceKiVsJ3BpoICJM9wU1VJLD7l3qd5xTmo78LqThf9uibGWcWCD16LBOn0JK8rhhx_Gf2ClySDJtvm7zQJ1Z-Ipmv9D7I_zhjztfi2UTVsJp7917XToHBm2EoNZqyE8homtGskFIiif5EZthHQvvOj8S2gJx8_t_UpWp1ScpIsD_Xq83LX-B956I_EBIeNoGwZZPFC5zAIoMeiaC1jU-sdOHVucLJM_x-jkzMvK8Utdfvp9MMvKyTfb_BZoe0-FAc2ZVlXEpwYgJVAGdCXv3lQT4bpTXyBwDrDVrUeJDivSSwOvT8tlnuMrXoD1Sk2NZB5SHyNmZsfyAEqLALbUnhkX8hbt5U2yNQRDf1LQhuUIOii6k6H9wnDNRnBiQHUfzKfW1CLiThnuVFjlCxQhJ60u67n3EK38XxHkQdOocJXpBNO51E4-f9z2hj0EDTu_ScuqOiC9cI8qJ4grSZIOnnQLv9WPvmCzx5zib3JacesIxMVvZNQiljq_gL7udm1yeXQjENOrBWbfBEkv1P4izWeAysoJgZUhtZFwKFdoCGt2TXe3xQ-wVZFS5KoMPhGFDZGPKzpK15caQOnWobOHLKaL8eFA-qI44qZrMQ7sSLn04bYeenNR2Vxz7hvK0lJhkgKrpVfUnZrtF-e-ubeeUCThWus4jZbKlFBe2Kroz90Elij_UaYd3YKuJu25o9LadNF0XlRH8pq_gjACDJK1uhpLOtB_wDfoOiJ0KTXLSgtUwAGWTSmhvO9zEP-9EzdJB9eHeXBR07v8NkRwgzuz-UFdvthBsX-ok_-fLFLdnIv0v-La2xLaXEJU2pO9UpwgQUalhX6pQSEs0hVUqfz9HUY8f8bnlu00yzNHohgAAAAAk9gpxkIeipgAB6gXACgAAmQiwx7HEZ_sCAAAAAARZWg==

[JelleZijlstra]

We do not control the Black Playground. Please report issues with it to https://github.com/jpadilla/black-playground. (Most likely it's running Black under a version <3.10 on the backend.)

[harmbuisman]

FYI: If you're using vscode, you can add the following to your settings.json

"python.formatting.blackArgs": [
    "--target-version",
    "py310"
],

Thanks, this also helps in getting the black commit hook to work, e.g. in the pre-commit-config.yaml:

-   repo: https://github.com/psf/black
    rev: 23.12.1
    hooks:
    -   id: black
        args: ["--target-version", "py310"]