Tests on Windows fail with [WinError 6] The handle is invalid

[Vuizur]

Contributing guidelines

• I understand the contributing guidelines

Documentation

• My problem is not addressed by the documentation or examples

Existing issues

• My problem does not appear in an existing issue

What operating system and Python version are you using?

Windows 11/Python 3.11

What OSMnx version are you using?

cloned from git today

Environment packages and versions

(from pip list)

attrs              23.1.0
branca             0.6.0
certifi            2023.7.22
charset-normalizer 3.2.0
click              8.1.7
click-plugins      1.1.1
cligj              0.7.2
colorama           0.4.6
contourpy          1.1.1
cycler             0.11.0
Fiona              1.9.4.post1
folium             0.14.0
fonttools          4.42.1
geopandas          0.14.0
idna               3.4
iniconfig          2.0.0
Jinja2             3.1.2
kiwisolver         1.4.5
libcst             1.0.1
MarkupSafe         2.1.3
matplotlib         3.8.0
MonkeyType         23.3.0
mypy-extensions    1.0.0
networkx           3.1
numpy              1.26.0
osmnx              1.6.0        D:\Programs\osmnx
packaging          23.1
pandas             2.1.1
Pillow             10.0.1
pip                23.2.1
pluggy             1.3.0
pyparsing          3.1.1
pyproj             3.6.1
pytest             7.4.2
pytest-monkeytype  1.1.0
python-dateutil    2.8.2
pytz               2023.3.post1
PyYAML             6.0.1
requests           2.31.0
setuptools         68.1.2
shapely            2.0.1
six                1.16.0
typing_extensions  4.8.0
typing-inspect     0.9.0
tzdata             2023.3
urllib3            2.0.5
wheel              0.41.2

How did you install OSMnx?

Pip

Problem description

When I run any test (I clone the repo, execute hatch shell and the pytest), it fails with the following error:

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _  

message = "2023-09-26 11:41:17 Projected GeoDataFrame to 'EPSG:32630 / WGS 84 / UTM zone 30N'", level = 20, name = 'OSMnx', filename = 'osmnx'

    def log(message, level=None, name=None, filename=None):
        """
        Write a message to the logger.

        This logs to file and/or prints to the console (terminal), depending on
        the current configuration of settings.log_file and settings.log_console.

        Parameters
        ----------
        message : string
            the message to log
        level : int
            one of Python's logger.level constants
        name : string
            name of the logger
        filename : string
            name of the log file, without file extension

        Returns
        -------
        None
        """
        if level is None:
            level = settings.log_level
        if name is None:
            name = settings.log_name
        if filename is None:
            filename = settings.log_filename

        # if logging to file is turned on
        if settings.log_file:
            # get the current logger (or create a new one, if none), then log
            # message at requested level
            logger = _get_logger(level=level, name=name, filename=filename)
            if level == lg.DEBUG:
                logger.debug(message)
            elif level == lg.INFO:
                logger.info(message)
            elif level == lg.WARNING:
                logger.warning(message)
            elif level == lg.ERROR:
                logger.error(message)

        # if logging to console (terminal window) is turned on
        if settings.log_console:
            # prepend timestamp
            message = f"{ts()} {message}"

            # convert to ascii so it doesn't break windows terminals
            message = (
                unicodedata.normalize("NFKD", str(message)).encode("ascii", errors="replace").decode()
            )

            # print explicitly to terminal in case jupyter notebook is the stdout
            if getattr(sys.stdout, "_original_stdstream_copy", None) is not None:
                # redirect captured pipe back to original
                os.dup2(sys.stdout._original_stdstream_copy, sys.__stdout__.fileno())
                sys.stdout._original_stdstream_copy = None
            with redirect_stdout(sys.__stdout__):
>               print(message, file=sys.__stdout__, flush=True)
E               OSError: [WinError 6] The handle is invalid

osmnx\utils.py:314: OSError

I don't know the reason, only that it is caused by this block (called when there is a log function call in the test code):

if getattr(sys.stdout, "_original_stdstream_copy", None) is not None:
    # redirect captured pipe back to original
    os.dup2(sys.stdout._original_stdstream_copy, sys.__stdout__.fileno())
    sys.stdout._original_stdstream_copy = None
with redirect_stdout(sys.__stdout__):
    print(message, file=sys.__stdout__, flush=True)

(So replacing it with print(message) causes the tests to run correctly.)

Complete minimal reproducible example

-

[gboeing]

I just tested this on Windows 10 and am unable to reproduce. Can anyone else test and troubleshoot on Windows 11?

[gboeing]

I was finally able to test this on a Windows 11 machine. It does not produce an error in the Python interpreter, IPython, or JupyterLab. But it does generate the error @Vuizur documented when running the test suite via pytest. I have no further explanation for why the error occurs. Given the inconsistency across interpreters, I'm inclined to chalk this up to an issue with pytest since OSMnx continues to run fine via Python, IPython, and JupyterLab.

@Vuizur you may want to open an issue with pytest instead.

[gboeing]

The relevant pytest issue appears to be here: pytest-dev/pytest#8257, though this may possibly also be relevant: pytest-dev/pytest#2251

In turn this seems to affect other downstream pytest users (e.g., facebook/Ax#1781, Delgan/loguru#499, yijiangh/pybullet_planning#13).

I'm going to close this issue here as it seems to be a pytest issue with writing stdout to the terminal on Windows.

[Vuizur]

Thanks a lot for the investigation! Dne út 10. 10. 2023 19:16 uživatel Geoff Boeing ***@***.***> napsal:

…

The relevant pytest issue appears to be here: pytest-dev/pytest#8257 <pytest-dev/pytest#8257>, though this may possibly also be relevant: pytest-dev/pytest#2251 <pytest-dev/pytest#2251> In turn this seems to affect other downstream pytest users (e.g., facebook/Ax#1781 <facebook/Ax#1781>, Delgan/loguru#499 <Delgan/loguru#499>, yijiangh/pybullet_planning#13 <yijiangh/pybullet_planning#13>). I'm going to close this issue here as it seems to be a pytest issue with writing stdout to the terminal on Windows. — Reply to this email directly, view it on GitHub <#1061 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AG66XKKX7XLOFYUNA6M4RUTX6V7FHAVCNFSM6AAAAAA5HOBOVCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONJVHEYDANZQGY> . You are receiving this because you were mentioned.Message ID: ***@***.***>