Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change in Flag enum behaviour in v3.11rc2 #96865

Closed
philthompson10 opened this issue Sep 16, 2022 · 8 comments
Closed

Change in Flag enum behaviour in v3.11rc2 #96865

philthompson10 opened this issue Sep 16, 2022 · 8 comments
Assignees
Labels
3.11 only security fixes stdlib Python modules in the Lib dir

Comments

@philthompson10
Copy link

Bug report

The default behaviour of Flag enums has chenged between Python v3.10 and v3.11

Your environment

Python v3.11rc2 on macOS

The following script...

from enum import Flag

class MethodHint(Flag):
    HiddenText = 0x10
    DigitsOnly = 0x01
    LettersOnly = 0x02
    OnlyMask = 0x0f

print(MethodHint.HiddenText|MethodHint.OnlyMask)

...raises a ValueError in v3.11 and prints...

MethodHint.HiddenText|OnlyMask|LettersOnly|DigitsOnly

in v3.10.

While it's not particularly sensible to bitwise-or a mask it is a significant change of behaviour without a deprecation period.

It will be common to have a mask value that covers more bits than currently defined when allowing for future expansion of the flags.

@philthompson10 philthompson10 added the type-bug An unexpected behavior, bug, or error label Sep 16, 2022
@AlexWaygood AlexWaygood added stdlib Python modules in the Lib dir 3.11 only security fixes labels Sep 16, 2022
@ethanfurman
Copy link
Member

ethanfurman commented Sep 16, 2022

There were some significant changes to Enum. They were originally planned for 3.10, but then pushed back to 3.11.

There are a few ways to adjust the above code:

  1. use an IntEnum instead (it seems the actual values are important)
  2. specify the boundary; i.e. class MethodHint(Enum, boundary=CONFORM):
  3. use auto() for the values and reorder the members

The result of (1) would be 31.

The result of (2) would be MethodHint.HiddenText|DigitsOnly|LettersOnly.

(3) would be:

>>> from enum import Flag, auto

>>> class MethodHint(Flag):
...     DigitsOnly = auto()
...     LettersOnly = auto()
...     OnlyMask = DigitsOnly|LettersOnly
...     HiddenText = auto()
... 
>>> print(MethodHint.HiddenText|MethodHint.OnlyMask)
MethodHint.DigitsOnly|LettersOnly|HiddenText

@ethanfurman ethanfurman removed the type-bug An unexpected behavior, bug, or error label Sep 16, 2022
@philthompson10
Copy link
Author

philthompson10 commented Sep 16, 2022 via email

@ethanfurman
Copy link
Member

@pablogsal -- Thoughts?

@gpshead
Copy link
Member

gpshead commented Sep 17, 2022

The enum.Flag documentation has never stated this was forbidden and existing code clearly depends on it working so IMNSHO we need to restore the 3.10 enum.Flag behavior. Even if we let this ship in 3.11.0 and just document it as a known issue in the release, it is worthy of fixing before 3.11.1.

3.10 docs: https://docs.python.org/3.10/library/enum.html#flag
3.11 docs: https://docs.python.org/3.11/library/enum.html#enum.Flag

Neither of them proscribe specifics about what the Flag values must be, it just says that operations like | always work and produce a valid result of the same type. The new behavior violates that.

@ethanfurman
Copy link
Member

Agreed on restoring the behavior and having the deprecation period, I was unsure if it should go into 3.11.0. I'll get a patch created for 3.11.1.

@ethanfurman
Copy link
Member

Actually, won't need a deprecation period, since the proper default for Flag is CONFORM as that matches previous behavior.

Thank you, everyone.

@ethanfurman
Copy link
Member

Phil wrote:

Should the default boundary be KEEP for the next version?

I edited my post: KEEP should be CONFORM (no spurious 12 that way).

miss-islington pushed a commit to miss-islington/cpython that referenced this issue Oct 5, 2022
miss-islington added a commit that referenced this issue Oct 5, 2022
(cherry picked from commit b44372e)

Co-authored-by: Ethan Furman <[email protected]>
carljm added a commit to carljm/cpython that referenced this issue Oct 6, 2022
* main:
  pythonGH-88050: fix race in closing subprocess pipe in asyncio  (python#97951)
  pythongh-93738: Disallow pre-v3 syntax in the C domain (python#97962)
  pythongh-95986: Fix the example using match keyword (python#95989)
  pythongh-97897: Prevent os.mkfifo and os.mknod segfaults with macOS 13 SDK (pythonGH-97944)
  pythongh-94808: Cover `PyUnicode_Count` in CAPI (python#96929)
  pythongh-94808: Cover `PyObject_PyBytes` case with custom `__bytes__` method (python#96610)
  pythongh-95691: Doc BufferedWriter and BufferedReader (python#95703)
  pythonGH-88968: Add notes about socket ownership transfers (python#97936)
  pythongh-96865: [Enum] fix Flag to use CONFORM boundary (pythonGH-97528)
carljm added a commit to carljm/cpython that referenced this issue Oct 8, 2022
* main: (53 commits)
  pythongh-94808: Coverage: Test that maximum indentation level is handled (python#95926)
  pythonGH-88050: fix race in closing subprocess pipe in asyncio  (python#97951)
  pythongh-93738: Disallow pre-v3 syntax in the C domain (python#97962)
  pythongh-95986: Fix the example using match keyword (python#95989)
  pythongh-97897: Prevent os.mkfifo and os.mknod segfaults with macOS 13 SDK (pythonGH-97944)
  pythongh-94808: Cover `PyUnicode_Count` in CAPI (python#96929)
  pythongh-94808: Cover `PyObject_PyBytes` case with custom `__bytes__` method (python#96610)
  pythongh-95691: Doc BufferedWriter and BufferedReader (python#95703)
  pythonGH-88968: Add notes about socket ownership transfers (python#97936)
  pythongh-96865: [Enum] fix Flag to use CONFORM boundary (pythonGH-97528)
  pythongh-65961: Raise `DeprecationWarning` when `__package__` differs from `__spec__.parent` (python#97879)
  docs(typing): add "see PEP 675" to LiteralString (python#97926)
  pythongh-97850: Remove all known instances of module_repr() (python#97876)
  I changed my surname early this year (python#96671)
  pythongh-93738: Documentation C syntax (:c:type:<C type> -> :c:expr:<C type>) (python#97768)
  pythongh-91539: improve performance of get_proxies_environment  (python#91566)
  build(deps): bump actions/stale from 5 to 6 (python#97701)
  pythonGH-95172 Make the same version `versionadded` oneline (python#95172)
  pythongh-88050: Fix asyncio subprocess to kill process cleanly when process is blocked (python#32073)
  pythongh-93738: Documentation C syntax (Function glob patterns -> literal markup) (python#97774)
  ...
pablogsal pushed a commit that referenced this issue Oct 22, 2022
(cherry picked from commit b44372e)

Co-authored-by: Ethan Furman <[email protected]>
@ethanfurman
Copy link
Member

STRICT was/is the correct boundary for Flag -- some underlying issues were preventing it from working correctly. Those have now been fixed, and pre-3.11 behavior maintained.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3.11 only security fixes stdlib Python modules in the Lib dir
Projects
None yet
Development

No branches or pull requests

4 participants