WinError(): Python int too large to convert to C long

[KelvinYou]

BPO	28474
Nosy	@pfmoore, @amauryfa, @abalkin, @tjguk, @meadori, @zware, @eryksun, @zooba, @MojoVampire, @ameily
PRs	gh-72660: Handle unsigned long win32 error codes #27959

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2016-10-19.07:28:00.932>
labels = ['type-bug', '3.8', '3.9', '3.10', 'ctypes', 'OS-windows']
title = 'WinError(): Python int too large to convert to C long'
updated_at = <Date 2021-08-26.01:28:22.054>
user = 'https://bugs.python.org/KelvinYou'

bugs.python.org fields:

activity = <Date 2021-08-26.01:28:22.054>
actor = 'meilyadam'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Windows', 'ctypes']
creation = <Date 2016-10-19.07:28:00.932>
creator = 'Kelvin You'
dependencies = []
files = []
hgrepos = []
issue_num = 28474
keywords = ['patch']
message_count = 8.0
messages = ['278963', '279005', '279008', '279009', '279033', '279047', '387744', '400134']
nosy_count = 11.0
nosy_names = ['paul.moore', 'amaury.forgeotdarc', 'belopolsky', 'tim.golden', 'meador.inge', 'zach.ware', 'eryksun', 'steve.dower', 'josh.r', 'meilyadam', 'Kelvin You']
pr_nums = ['27959']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue28474'
versions = ['Python 3.8', 'Python 3.9', 'Python 3.10']

[KelvinYou]

// callproc.c
static PyObject *format_error(PyObject *self, PyObject *args)
{
    PyObject *result;
    wchar_t *lpMsgBuf;
    DWORD code = 0;
    if (!PyArg_ParseTuple(args, "|i:FormatError", &code))  
                                  ^ Here the format string should be "|I:FormatError"
        return NULL;
    if (code == 0)
        code = GetLastError();
    lpMsgBuf = FormatError(code);
    if (lpMsgBuf) {
        result = PyUnicode_FromWideChar(lpMsgBuf, wcslen(lpMsgBuf));
        LocalFree(lpMsgBuf);
    } else {
        result = PyUnicode_FromString("<no description>");
    }
    return result;
}

[MojoVampire]

You can't use I as a format code safely; it silently ignores/wraps overflow on the conversion, where i raises on overflow. The unsigned converters are basically useless for resilient code in 99% of cases.

I *think* I remember some private utility functions for doing this using O& though, not sure if they're available in callproc.c...

[KelvinYou]

I report this issue because the function WlanScan(https://msdn.microsoft.com/zh-cn/library/windows/desktop/ms706783(v=vs.85).aspx) returns a error code 0x80342002 when the WLAN is disabled on Windows 10. ctypes.WinError() raise an exception of 'Python int too large to convert to C long' when handle this error code.

[KelvinYou]

Here is the full list of windows error code:
https://msdn.microsoft.com/en-us/library/cc231196.aspx
You can see a lot of error codes is above 0x80000000.

[zooba]

All HRESULT values are, since the topmost bit indicates that it's an error, but all the others should be 16-bit positive integers IIRC. I don't think this function is meant to work with HRESULTs, but I could be wrong - fairly sure it's meant for Win32 error codes, though I'd have to dig into the format function it calls.

What function are you calling that produces an HRESULT from GetLastError?

[eryksun]

Kelvin is calling WlanScan 1, which returns an error code that apparently can include HRESULT (signed) values cast as DWORD (unsigned) values, including 0x80342002 (ERROR_NDIS_DOT11_POWER_STATE_INVALID).

ctypes.FormatError calls FormatMessage 2 with the flag FORMAT_MESSAGE_FROM_SYSTEM. About half of the system error codes defined in winerror.h consist of COM HRESULT codes. But, to clarify Kelvin's link 3, this does not include NTSTATUS values from kernel-mode system calls. NTSTATUS values require the ntdll.dll message table.

A workaround in this case is to use the default c_int restype to return the error as a signed integer:

    >>> ctypes.FormatError(0x80342002 - 2**32)
    "The wireless local area network interface is powered down and doesn't
    support the requested operation."

A similar workaround works for setting the exit code to an HRESULT or NTSTATUS error code:

    >>> hex(subprocess.call('python -c "import os; os._exit(0xC0000001)"'))
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
    OverflowError: Python int too large to convert to C long
    '0x1'
    >>> hex(subprocess.call('python -c "import sys; sys.exit(0xC0000001)"'))
    '0xffffffff'

    >>> hex(subprocess.call('python -c "import os; os._exit(0xC0000001 - 2**32)"'))
    '0xc0000001'
    >>> hex(subprocess.call('python -c "import sys; sys.exit(0xC0000001 - 2**32)"'))
    '0xc0000001'

[eryksun]

format_error() can use format "L" (long long) and then check for a value in the accepted range value >= LONG_MIN && value <= ULONG_MAX. If the value is out of range, raise OverflowError. Otherwise assign the value to DWORD code.

[ameily]

I can potentially take a stab at writing up a PR for this. I've also seen this affecting other locations that eventually call FormatMessage, including:

• ctypes.format_error() - this original issue
• os.strerror()
• OSError(winerror=X)

I will most likely look into fixing all three.