Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bpo-15999: Accept arbitrary values for boolean parameters. #15609

Merged
merged 10 commits into from
Dec 3, 2022

Conversation

serhiy-storchaka
Copy link
Member

@serhiy-storchaka serhiy-storchaka commented Aug 30, 2019

Copy link
Member

@gpshead gpshead left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall this looks good. Though I do wonder if PyObject_IsTrue is the best behavior for users? The behavior of all objects being True by default is understandable but could lead to some confusion if someone passes the wrong parameter in the wrong place and no longer gets an exception.

We don't have, and probably do not want, a PyObject_IsExplicitlyTrueOrRaise API (if it existed I'd just want it to change the default case of True to raise a TypeError). So lets go with this as it is consistent with how most pure Python code is written.

@tiran tiran removed their request for review April 18, 2021 09:50
@rhettinger rhettinger removed their request for review May 3, 2022 06:51
Copy link
Member

@iritkatriel iritkatriel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This has merge conflicts now.

@bedevere-bot
Copy link

When you're done making the requested changes, leave the comment: I have made the requested changes; please review again.

@gpshead gpshead merged commit a87c46e into python:main Dec 3, 2022
@bedevere-bot
Copy link

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot wasm32-emscripten node (pthreads) 3.x has failed when building commit a87c46e.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/1050/builds/959) and take a look at the build logs.
  4. Check if the failure is related to this commit (a87c46e) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/1050/builds/959

Failed tests:

  • test_capi

Summary of the results of the build (if available):

== Tests result: FAILURE ==

338 tests OK.

10 slowest tests:

  • test_tokenize: 1 min 15 sec
  • test_unparse: 52.6 sec
  • test_lib2to3: 47.8 sec
  • test_io: 26.4 sec
  • test_unicodedata: 22.8 sec
  • test_zipfile: 22.3 sec
  • test_decimal: 19.2 sec
  • test_statistics: 18.5 sec
  • test_pickle: 16.9 sec
  • test_argparse: 13.6 sec

1 test failed:
test_capi

94 tests skipped:
test__xxsubinterpreters test_asyncgen test_asyncio
test_check_c_globals test_clinic test_cmd_line
test_concurrent_futures test_contextlib_async test_ctypes
test_curses test_dbm_gnu test_dbm_ndbm test_devpoll test_doctest
test_docxmlrpc test_dtrace test_embed test_epoll test_faulthandler
test_fcntl test_file_eintr test_fork1 test_ftplib test_gdb
test_grp test_httplib test_httpservers test_idle test_imaplib
test_interpreters test_ioctl test_kqueue test_launcher test_lzma
test_mmap test_msilib test_multiprocessing_fork
test_multiprocessing_forkserver test_multiprocessing_main_handling
test_multiprocessing_spawn test_nis test_openpty test_ossaudiodev
test_pdb test_peg_generator test_perf_profiler test_poll
test_poplib test_pty test_pwd test_readline test_regrtest
test_repl test_resource test_select test_selectors test_smtplib
test_smtpnet test_socket test_socketserver test_spwd test_ssl
test_stable_abi_ctypes test_startfile test_subprocess
test_sys_settrace test_syslog test_tcl test_telnetlib test_tix
test_tkinter test_tools test_ttk test_ttk_textonly test_turtle
test_urllib2 test_urllib2_localnet test_urllib2net test_urllibnet
test_venv test_wait3 test_wait4 test_webbrowser test_winconsoleio
test_winreg test_winsound test_wmi test_wsgiref test_xmlrpc
test_xmlrpc_net test_xxlimited test_zipfile64
test_zipimport_support test_zoneinfo
0:21:33 load avg: 6.83
0:21:33 load avg: 6.83 Re-running failed tests is not supported with --python host runner option.

Total duration: 21 min 33 sec

Click to see traceback logs
remote: Enumerating objects: 150, done.        
remote: Counting objects:   0% (1/150)        
remote: Counting objects:   1% (2/150)        
remote: Counting objects:   2% (3/150)        
remote: Counting objects:   3% (5/150)        
remote: Counting objects:   4% (6/150)        
remote: Counting objects:   5% (8/150)        
remote: Counting objects:   6% (9/150)        
remote: Counting objects:   7% (11/150)        
remote: Counting objects:   8% (12/150)        
remote: Counting objects:   9% (14/150)        
remote: Counting objects:  10% (15/150)        
remote: Counting objects:  11% (17/150)        
remote: Counting objects:  12% (18/150)        
remote: Counting objects:  13% (20/150)        
remote: Counting objects:  14% (21/150)        
remote: Counting objects:  15% (23/150)        
remote: Counting objects:  16% (24/150)        
remote: Counting objects:  17% (26/150)        
remote: Counting objects:  18% (27/150)        
remote: Counting objects:  19% (29/150)        
remote: Counting objects:  20% (30/150)        
remote: Counting objects:  21% (32/150)        
remote: Counting objects:  22% (33/150)        
remote: Counting objects:  23% (35/150)        
remote: Counting objects:  24% (36/150)        
remote: Counting objects:  25% (38/150)        
remote: Counting objects:  26% (39/150)        
remote: Counting objects:  27% (41/150)        
remote: Counting objects:  28% (42/150)        
remote: Counting objects:  29% (44/150)        
remote: Counting objects:  30% (45/150)        
remote: Counting objects:  31% (47/150)        
remote: Counting objects:  32% (48/150)        
remote: Counting objects:  33% (50/150)        
remote: Counting objects:  34% (51/150)        
remote: Counting objects:  35% (53/150)        
remote: Counting objects:  36% (54/150)        
remote: Counting objects:  37% (56/150)        
remote: Counting objects:  38% (57/150)        
remote: Counting objects:  39% (59/150)        
remote: Counting objects:  40% (60/150)        
remote: Counting objects:  41% (62/150)        
remote: Counting objects:  42% (63/150)        
remote: Counting objects:  43% (65/150)        
remote: Counting objects:  44% (66/150)        
remote: Counting objects:  45% (68/150)        
remote: Counting objects:  46% (69/150)        
remote: Counting objects:  47% (71/150)        
remote: Counting objects:  48% (72/150)        
remote: Counting objects:  49% (74/150)        
remote: Counting objects:  50% (75/150)        
remote: Counting objects:  51% (77/150)        
remote: Counting objects:  52% (78/150)        
remote: Counting objects:  53% (80/150)        
remote: Counting objects:  54% (81/150)        
remote: Counting objects:  55% (83/150)        
remote: Counting objects:  56% (84/150)        
remote: Counting objects:  57% (86/150)        
remote: Counting objects:  58% (87/150)        
remote: Counting objects:  59% (89/150)        
remote: Counting objects:  60% (90/150)        
remote: Counting objects:  61% (92/150)        
remote: Counting objects:  62% (93/150)        
remote: Counting objects:  63% (95/150)        
remote: Counting objects:  64% (96/150)        
remote: Counting objects:  65% (98/150)        
remote: Counting objects:  66% (99/150)        
remote: Counting objects:  67% (101/150)        
remote: Counting objects:  68% (102/150)        
remote: Counting objects:  69% (104/150)        
remote: Counting objects:  70% (105/150)        
remote: Counting objects:  71% (107/150)        
remote: Counting objects:  72% (108/150)        
remote: Counting objects:  73% (110/150)        
remote: Counting objects:  74% (111/150)        
remote: Counting objects:  75% (113/150)        
remote: Counting objects:  76% (114/150)        
remote: Counting objects:  77% (116/150)        
remote: Counting objects:  78% (117/150)        
remote: Counting objects:  79% (119/150)        
remote: Counting objects:  80% (120/150)        
remote: Counting objects:  81% (122/150)        
remote: Counting objects:  82% (123/150)        
remote: Counting objects:  83% (125/150)        
remote: Counting objects:  84% (126/150)        
remote: Counting objects:  85% (128/150)        
remote: Counting objects:  86% (129/150)        
remote: Counting objects:  87% (131/150)        
remote: Counting objects:  88% (132/150)        
remote: Counting objects:  89% (134/150)        
remote: Counting objects:  90% (135/150)        
remote: Counting objects:  91% (137/150)        
remote: Counting objects:  92% (138/150)        
remote: Counting objects:  93% (140/150)        
remote: Counting objects:  94% (141/150)        
remote: Counting objects:  95% (143/150)        
remote: Counting objects:  96% (144/150)        
remote: Counting objects:  97% (146/150)        
remote: Counting objects:  98% (147/150)        
remote: Counting objects:  99% (149/150)        
remote: Counting objects: 100% (150/150)        
remote: Counting objects: 100% (150/150), done.        
remote: Compressing objects:   3% (1/29)        
remote: Compressing objects:   6% (2/29)        
remote: Compressing objects:  10% (3/29)        
remote: Compressing objects:  13% (4/29)        
remote: Compressing objects:  17% (5/29)        
remote: Compressing objects:  20% (6/29)        
remote: Compressing objects:  24% (7/29)        
remote: Compressing objects:  27% (8/29)        
remote: Compressing objects:  31% (9/29)        
remote: Compressing objects:  34% (10/29)        
remote: Compressing objects:  37% (11/29)        
remote: Compressing objects:  41% (12/29)        
remote: Compressing objects:  44% (13/29)        
remote: Compressing objects:  48% (14/29)        
remote: Compressing objects:  51% (15/29)        
remote: Compressing objects:  55% (16/29)        
remote: Compressing objects:  58% (17/29)        
remote: Compressing objects:  62% (18/29)        
remote: Compressing objects:  65% (19/29)        
remote: Compressing objects:  68% (20/29)        
remote: Compressing objects:  72% (21/29)        
remote: Compressing objects:  75% (22/29)        
remote: Compressing objects:  79% (23/29)        
remote: Compressing objects:  82% (24/29)        
remote: Compressing objects:  86% (25/29)        
remote: Compressing objects:  89% (26/29)        
remote: Compressing objects:  93% (27/29)        
remote: Compressing objects:  96% (28/29)        
remote: Compressing objects: 100% (29/29)        
remote: Compressing objects: 100% (29/29), done.        
remote: Total 76 (delta 73), reused 48 (delta 46), pack-reused 0        
From https://github.com/python/cpython
 * branch                  main       -> FETCH_HEAD
Note: switching to 'a87c46eab3c306b1c5b8a072b7b30ac2c50651c0'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at a87c46eab3 bpo-15999: Accept arbitrary values for boolean parameters. (#15609)
Switched to and reset branch 'main'

../../Objects/obmalloc.c:776:1: warning: always_inline function might not be inlinable [-Wattributes]
  776 | arena_map_get(pymem_block *p, int create)
      | ^~~~~~~~~~~~~

configure: ../../configure --prefix $(PWD)/target/host --with-pydebug --without-pydebug --with-emscripten-target=node --disable-wasm-dynamic-linking --enable-wasm-pthreads --build=x86_64-pc-linux-gnu --host=wasm32-unknown-emscripten --with-build-python=../build/python
configure: WARNING: using cross tools not prefixed with host triplet
mcc: error: no input files

make: make -j2 all
../../Python/initconfig.c:2347:27: warning: format specifies type 'wint_t' (aka 'int') but the argument has type 'wint_t' (aka 'unsigned int') [-Wformat]
    printf(usage_envvars, (wint_t)DELIM, (wint_t)DELIM, PYTHONHOMEHELP);
           ~~~~~~~~~~~~~  ^~~~~~~~~~~~~
../../Python/initconfig.c:138:18: note: format string is defined here
"PYTHONPATH   : '%lc'-separated list of directories prefixed to the\n"
                 ^~~
                 %c
../../Python/initconfig.c:2347:42: warning: format specifies type 'wint_t' (aka 'int') but the argument has type 'wint_t' (aka 'unsigned int') [-Wformat]
    printf(usage_envvars, (wint_t)DELIM, (wint_t)DELIM, PYTHONHOMEHELP);
           ~~~~~~~~~~~~~                 ^~~~~~~~~~~~~
../../Python/initconfig.c:141:58: note: format string is defined here
"PYTHONHOME   : alternate <prefix> directory (or <prefix>%lc<exec_prefix>).\n"
                                                         ^~~
                                                         %c
2 warnings generated.
../../Python/pytime.c:297:10: warning: implicit conversion from 'long long' to 'double' changes value from 9223372036854775807 to 9223372036854775808 [-Wimplicit-const-int-float-conversion]
    if (!_Py_InIntegralTypeRange(time_t, intpart)) {
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../../Include/internal/pycore_pymath.h:72:45: note: expanded from macro '_Py_InIntegralTypeRange'
    (_Py_IntegralTypeMin(type) <= v && v <= _Py_IntegralTypeMax(type))
                                         ~~ ^~~~~~~~~~~~~~~~~~~~~~~~~
../../Include/internal/pycore_pymath.h:61:88: note: expanded from macro '_Py_IntegralTypeMax'
    (_Py_IS_TYPE_SIGNED(type) ? (((((type)1 << (sizeof(type)*CHAR_BIT - 2)) - 1) << 1) + 1) : ~(type)0)
                                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~
../../Python/pytime.c:352:14: warning: implicit conversion from 'long long' to 'double' changes value from 9223372036854775807 to 9223372036854775808 [-Wimplicit-const-int-float-conversion]
        if (!_Py_InIntegralTypeRange(time_t, intpart)) {
             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../../Include/internal/pycore_pymath.h:72:45: note: expanded from macro '_Py_InIntegralTypeRange'
    (_Py_IntegralTypeMin(type) <= v && v <= _Py_IntegralTypeMax(type))
                                         ~~ ^~~~~~~~~~~~~~~~~~~~~~~~~
../../Include/internal/pycore_pymath.h:61:88: note: expanded from macro '_Py_IntegralTypeMax'
    (_Py_IS_TYPE_SIGNED(type) ? (((((type)1 << (sizeof(type)*CHAR_BIT - 2)) - 1) << 1) + 1) : ~(type)0)
                                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~
../../Python/pytime.c:518:10: warning: implicit conversion from 'long long' to 'double' changes value from 9223372036854775807 to 9223372036854775808 [-Wimplicit-const-int-float-conversion]
    if (!_Py_InIntegralTypeRange(_PyTime_t, d)) {
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../../Include/internal/pycore_pymath.h:72:45: note: expanded from macro '_Py_InIntegralTypeRange'
    (_Py_IntegralTypeMin(type) <= v && v <= _Py_IntegralTypeMax(type))
                                         ~~ ^~~~~~~~~~~~~~~~~~~~~~~~~
../../Include/internal/pycore_pymath.h:61:88: note: expanded from macro '_Py_IntegralTypeMax'
    (_Py_IS_TYPE_SIGNED(type) ? (((((type)1 << (sizeof(type)*CHAR_BIT - 2)) - 1) << 1) + 1) : ~(type)0)
                                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~
3 warnings generated.
../../Modules/expat/xmlparse.c:3116:9: warning: code will never be executed [-Wunreachable-code]
        parser->m_characterDataHandler(parser->m_handlerArg, parser->m_dataBuf,
        ^~~~~~
../../Modules/expat/xmlparse.c:3115:16: note: silence by adding parentheses to mark code as explicitly dead
      else if (0 && parser->m_characterDataHandler)
               ^
               /* DISABLES CODE */ ( )
../../Modules/expat/xmlparse.c:4059:9: warning: code will never be executed [-Wunreachable-code]
        parser->m_characterDataHandler(parser->m_handlerArg, parser->m_dataBuf,
        ^~~~~~
../../Modules/expat/xmlparse.c:4058:16: note: silence by adding parentheses to mark code as explicitly dead
      else if (0 && parser->m_characterDataHandler)
               ^
               /* DISABLES CODE */ ( )
../../Modules/expat/xmlparse.c:7703:11: warning: format specifies type 'int' but the argument has type 'ptrdiff_t' (aka 'long') [-Wformat]
          bytesMore, (account == XML_ACCOUNT_DIRECT) ? "DIR" : "EXP",
          ^~~~~~~~~
3 warnings generated.
../../Modules/socketmodule.c:4031:33: warning: comparison of integers of different signs: 'unsigned long' and 'long' [-Wsign-compare]
         cmsgh != NULL; cmsgh = CMSG_NXTHDR(&msg, cmsgh)) {
                                ^~~~~~~~~~~~~~~~~~~~~~~~
/opt/buildbot/.emscripten_cache/sysroot/include/sys/socket.h:356:44: note: expanded from macro 'CMSG_NXTHDR'
        __CMSG_LEN(cmsg) + sizeof(struct cmsghdr) >= __MHDR_END(mhdr) - (unsigned char *)(cmsg) \
        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ^  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../../Modules/socketmodule.c:4084:33: warning: comparison of integers of different signs: 'unsigned long' and 'long' [-Wsign-compare]
         cmsgh != NULL; cmsgh = CMSG_NXTHDR(&msg, cmsgh)) {
                                ^~~~~~~~~~~~~~~~~~~~~~~~
/opt/buildbot/.emscripten_cache/sysroot/include/sys/socket.h:356:44: note: expanded from macro 'CMSG_NXTHDR'
        __CMSG_LEN(cmsg) + sizeof(struct cmsghdr) >= __MHDR_END(mhdr) - (unsigned char *)(cmsg) \
        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ^  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../../Modules/socketmodule.c:4708:54: warning: comparison of integers of different signs: 'unsigned long' and 'long' [-Wsign-compare]
            cmsgh = (i == 0) ? CMSG_FIRSTHDR(&msg) : CMSG_NXTHDR(&msg, cmsgh);
                                                     ^~~~~~~~~~~~~~~~~~~~~~~~
/opt/buildbot/.emscripten_cache/sysroot/include/sys/socket.h:356:44: note: expanded from macro 'CMSG_NXTHDR'
        __CMSG_LEN(cmsg) + sizeof(struct cmsghdr) >= __MHDR_END(mhdr) - (unsigned char *)(cmsg) \
        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ^  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
3 warnings generated.
../../Modules/_sqlite/connection.c:2198:19: warning: result of comparison of constant 9223372036854775807 with expression of type 'Py_ssize_t' (aka 'long') is always false [-Wtautological-constant-out-of-range-compare]
    if (data->len > 9223372036854775807) {  // (1 << 63) - 1
        ~~~~~~~~~ ^ ~~~~~~~~~~~~~~~~~~~
1 warning generated.
mcc: warning: USE_PTHREADS + ALLOW_MEMORY_GROWTH may run non-wasm code slowly, see https://github.com/WebAssembly/design/issues/1271 [-Wpthreads-mem-growth]

make: *** [Makefile:1860: buildbottest] Error 2

Cannot open file '/opt/buildbot/bcannon-wasm/3.x.bcannon-wasm.emscripten-node-pthreads/build/build/build_oot/host/test-results.xml' for upload

@serhiy-storchaka
Copy link
Member Author

Oh. The truth is that this PR was not intended to be merged. This is just a demonstration to see how much needs to be changed to achieve this and which tests will be broken in result. I also worked on the opposite change -- to make only True and False (and maybe 1 and 0 for compatibility) be accepted as boolean argument.

@serhiy-storchaka serhiy-storchaka deleted the accept-bool branch December 3, 2022 21:01
@gpshead
Copy link
Member

gpshead commented Dec 3, 2022

Want me to revert it? I was wondering about the utility, but the comment I apparently left here in 2019 still made sense to me: Accepting anything for bool is still more consistent with how pure Python code behaves.

@serhiy-storchaka
Copy link
Member Author

No need to rush, it will only increase the code churn. I will first try to finish the opposite patch.

Yes, accepting arbitrary Python object is more consistent with the pure Python code, but on other hand, it is more error prone, especially for positional arguments. Even for keyword arguments, there are cases in which silently accepting arbitrary Python object as boolean can hide an error. For example if you pass func instead of func(), or x.attr instead of x.attr, or check for emptyness of a generator instead of a sequence, or don't replace some placeholder with an actual value.

@merwok
Copy link
Member

merwok commented Dec 5, 2022

Process note: the DO-NOT-MERGE label should probably not be removed before checking with the person who added it.

@serhiy-storchaka
Copy link
Member Author

Next time I would use the draft mode. This PR was created before introducing of that feature.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type-feature A feature request or enhancement
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants