Intermittent error when testing "jupytext --execute" on Windows

[mwouts]

From time to time some of the tests that run jupytext --execute fail on Windows. An instance when this occured is this run.

The corresponding error message is below:

------------------------------ Captured log call ------------------------------
WARNING  traitlets:execute.py:599 Timeout waiting for IOPub output
________________________________ test_execute _________________________________

tmpdir = local('C:\\Users\\runneradmin\\AppData\\Local\\Temp\\pytest-of-runneradmin\\pytest-0\\test_execute0')

    @requires_nbconvert
    def test_execute(tmpdir):
        tmp_ipynb = str(tmpdir.join('notebook.ipynb'))
        tmp_py = str(tmpdir.join('notebook.py'))
    
        with open(tmp_py, 'w') as fp:
            fp.write("""1 + 2
    """)
    
        jupytext(args=[tmp_py, '--to', 'ipynb', '--execute'])
    
        nb = read(tmp_ipynb)
        assert len(nb.cells) == 1
>       assert nb.cells[0].outputs[0]['data'] == {'text/plain': '3'}
E       IndexError: list index out of range

[mwouts]

A similar issue was documented at nteract/papermill#426, and a fix is being prepared at jupyter/nbconvert#1183.

[mwouts]

In nbconvert's documentation there is this paragraph:

NotebookClient.iopub_timeout : Int

Default: 4

The time to wait (in seconds) for IOPub output. This generally doesn’t need to be set, but on some slow networks (such as CI systems) the default timeout might not be long enough to get all messages.

[mwouts]

An alternative is to use this other option:

NotebookClient.raise_on_iopub_timeout : Bool

Default: False

If False (default), then the kernel will continue waiting for iopub messages until it receives a kernel idle message, or until a timeout occurs, at which point the currently executing cell will be skipped. If True, then an error will be raised after the first timeout. This option generally does not need to be used, but may be useful in contexts where there is the possibility of executing notebooks with memory-consuming infinite loops.

[mwouts]

I am still seeing the same error with raise_on_iopub_timeout=True, so I won't change anything in jupytext, and just skip the test if we see the timeout warning.

[mwouts]

There is another occurrence at https://github.com/mwouts/jupytext/runs/584893307

    def test_utf8_out_331(capsys):
        py = u"from IPython.core.display import HTML; HTML(u'\xd7')"
    
        with mock.patch("sys.stdin", StringIO(py)):
            jupytext(["--to", "ipynb", "--execute", "-"])
    
        out, err = capsys.readouterr()
        assert err == ""
        nb = reads(out, "ipynb")
        assert len(nb.cells) == 1
        print(nb.cells[0].outputs)
>       assert nb.cells[0].outputs[0]["data"]["text/html"] == u"\xd7"
E       IndexError: list index out of range

D:\a\jupytext\jupytext\tests\test_cli.py:927: IndexError
---------------------------- Captured stdout call -----------------------------
[]
------------------------------ Captured log call ------------------------------
WARNING  traitlets:execute.py:599 Timeout waiting for IOPub output

[mwouts]

I am seeing another occurence in this run.

        if "Timeout" in caplog.text:
            pytest.skip(caplog.text)  # Issue 489
    
        nb = read(tmp_ipynb)
        assert len(nb.cells) == 1
>       assert nb.cells[0].outputs[0]["data"] == {"text/plain": "3"}
E       IndexError: list index out of range

D:\a\jupytext\jupytext\tests\test_execute.py:65: IndexError
---------------------------- Captured stdout call -----------------------------
[jupytext] Reading C:\Users\runneradmin\AppData\Local\Temp\pytest-of-runneradmin\pytest-0\test_pipe_nbconvert_execute_sy0\notebook.py
[jupytext] Updating notebook metadata with '{"jupytext": {"formats": "py,ipynb"}}'
[jupytext] Executing jupyter nbconvert --stdin --stdout --to notebook --execute
[jupytext] Updating 'C:\Users\runneradmin\AppData\Local\Temp\pytest-of-runneradmin\pytest-0\test_pipe_nbconvert_execute_sy0\notebook.ipynb'
[jupytext] Updating 'C:\Users\runneradmin\AppData\Local\Temp\pytest-of-runneradmin\pytest-0\test_pipe_nbconvert_execute_sy0\notebook.py'
---------------------------- Captured stderr call -----------------------------
[NbConvertApp] Converting notebook into notebook

[NbConvertApp] Executing notebook with kernel: python

[NbConvertApp] WARNING | Timeout waiting for IOPub output

[mwouts]

I am seeing yet another error in this run, but this time it is a nested command that has the timeout...

_________________________ test_pipe_nbconvert_execute _________________________

tmpdir = local('C:\\Users\\runneradmin\\AppData\\Local\\Temp\\pytest-of-runneradmin\\pytest-0\\test_pipe_nbconvert_execute0')
caplog = <_pytest.logging.LogCaptureFixture object at 0x0000029303C74F88>
capsys = <_pytest.capture.CaptureFixture object at 0x0000029303C742C8>

    @requires_nbconvert
    def test_pipe_nbconvert_execute(tmpdir, caplog, capsys):
        tmp_ipynb = str(tmpdir.join("notebook.ipynb"))
        tmp_py = str(tmpdir.join("notebook.py"))
    
        with open(tmp_py, "w") as fp:
            fp.write(
                """1 + 2
    """
            )
    
        jupytext(
            args=[
                tmp_py,
                "--to",
                "ipynb",
                "--pipe-fmt",
                "ipynb",
                "--pipe",
                "jupyter nbconvert --stdin --stdout --to notebook --execute",
            ]
        )
        skip_if_timeout(caplog, capsys)
    
        nb = read(tmp_ipynb)
        assert len(nb.cells) == 1
>       assert nb.cells[0].outputs[0]["data"] == {"text/plain": "3"}
E       IndexError: list index out of range

D:\a\jupytext\jupytext\tests\test_execute.py:43: IndexError
---------------------------- Captured stderr call -----------------------------
[NbConvertApp] Converting notebook into notebook

[NbConvertApp] Executing notebook with kernel: python

[NbConvertApp] WARNING | Timeout waiting for IOPub output

[mwouts]

This is still occuring randomly, the latest instance being ca577da.

Probably a better fix will be to skip these tests on Windows.