Add Xonsh support

[jsquaredosquared]

Summary

Hi @mwouts, I have tried following the instructions in the documentation to add support for Xonsh.

File extension: .xsh
Comment: "#"
Kernel: xonsh-jupyter

However, on my machine, 2 tests are still failing (even though I did not touch the code):

================================================================================= short test summary info ==================================================================================
FAILED tests/integration/cli/test_execute.py::test_execute_r - AttributeError: outputs
FAILED tests/functional/others/test_pytest.py::test_run_pytest_ok - SystemExit: 4
============================================================ 2 failed, 3123 passed, 88 skipped, 6 warnings in 563.72s (0:09:23) ============================================================

Can you provide any guidance on this?

Thanks,
Jeff

---

Full test failures:

______________________________________________________________________________________ test_execute_r ______________________________________________________________________________________
[gw1] linux -- Python 3.11.7 /home/jeffr/miniconda3/envs/jupytext-dev/bin/python

tmpdir = local('/tmp/pytest-of-jeffr/pytest-9/popen-gw1/test_execute_r0'), caplog = <_pytest.logging.LogCaptureFixture object at 0x7f7c2e31ef10>
capsys = <_pytest.capture.CaptureFixture object at 0x7f7c2e31c3d0>

    @pytest.mark.requires_nbconvert
    @pytest.mark.requires_ir_kernel
    @pytest.mark.skip_on_windows
    def test_execute_r(tmpdir, caplog, capsys):  # pragma: no cover
        tmp_ipynb = str(tmpdir.join("notebook.ipynb"))
        tmp_md = str(tmpdir.join("notebook.md"))

        with open(tmp_md, "w") as fp:
            fp.write(
                """```r
    1 + 2 + 3
    ```
    """
            )

        jupytext(args=[tmp_md, "--to", "ipynb", "--execute"])

        nb = read(tmp_ipynb)
        assert len(nb.cells) == 1
>       assert nb.cells[0].outputs[0]["data"]["text/markdown"] == "6"

tests/integration/cli/test_execute.py:176:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = {'cell_type': 'markdown', 'id': 'cell-1208', 'metadata': {}, 'source': '```r\n1 + 2 + 3\n```'}, key = 'outputs'

    def __getattr__(self, key):
        """Get an attr by calling :meth:`dict.__getitem__`.

        Like :meth:`__setattr__`, this method converts :exc:`KeyError` to
        :exc:`AttributeError`.

        Examples
        --------
        >>> s = Struct(a=10)
        >>> s.a
        10
        >>> type(s.get)
        <... 'builtin_function_or_method'>
        >>> try:
        ...     s.b
        ... except AttributeError:
        ...     print("I don't have that key")
        ...
        I don't have that key
        """
        try:
            result = self[key]
        except KeyError:
>           raise AttributeError(key) from None
E           AttributeError: outputs

../miniconda3/envs/jupytext-dev/lib/python3.11/site-packages/nbformat/_struct.py:125: AttributeError
----------------------------------------------------------------------------------- Captured stdout call -----------------------------------------------------------------------------------
[jupytext] Reading /tmp/pytest-of-jeffr/pytest-9/popen-gw1/test_execute_r0/notebook.md in format md
[jupytext] Setting kernel python3
[jupytext] Updating notebook metadata with '{"kernelspec": {"name": "python3", "language": "python", "display_name": "Python 3 (ipykernel)"}}'
[jupytext] Executing notebook with kernel python3
[jupytext] Writing /tmp/pytest-of-jeffr/pytest-9/popen-gw1/test_execute_r0/notebook.ipynb
----------------------------------------------------------------------------------- Captured stderr call -----------------------------------------------------------------------------------
0.00s - Debugger warning: It seems that frozen modules are being used, which may
0.00s - make the debugger miss breakpoints. Please pass -Xfrozen_modules=off
0.00s - to python to disable frozen modules.
0.00s - Note: Debugging will proceed. Set PYDEVD_DISABLE_FILE_VALIDATION=1 to disable this validation.
____________________________________________________________________________________ test_run_pytest_ok ____________________________________________________________________________________
[gw2] linux -- Python 3.11.7 /home/jeffr/miniconda3/envs/jupytext-dev/bin/python

tmpdir = local('/tmp/pytest-of-jeffr/pytest-9/popen-gw2/test_run_pytest_ok0')
text = 'This is a notebook with a test that is supposed to pass\n```python\ndef test_ok():\n    assert True\n```\n'

    def test_run_pytest_ok(
        tmpdir,
        text="""This is a notebook with a test that is supposed to pass
    ```python
    def test_ok():
        assert True
    ```
    """,
    ):
        tmp_md = str(tmpdir.join("notebook.md"))
        with open(tmp_md, "w") as fp:
            fp.write(text)
>       jupytext([tmp_md, "--check", "pytest"])

tests/functional/others/test_pytest.py:18:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
src/jupytext/cli.py:497: in jupytext
    exit_code += jupytext_single_file(nb_file, args, log)
src/jupytext/cli.py:670: in jupytext_single_file
    pipe_notebook(
src/jupytext/cli.py:1182: in pipe_notebook
    exec_command(
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

command = ['pytest', '/tmp/notebook_oo_56at8.py'], input = None, capture = False, warn_only = False

    def exec_command(command, input=None, capture=False, warn_only=False):
        """Execute the desired command, and pipe the given input into it"""
        assert isinstance(command, list)
        sys.stdout.write("[jupytext] Executing {}\n".format(" ".join(command)))
        process = subprocess.Popen(
            command,
            **(
                dict(stdout=subprocess.PIPE, stdin=subprocess.PIPE)
                if input is not None
                else {}
            ),
        )
        out, err = process.communicate(input=input)
        if out and not capture:
            sys.stdout.write(out.decode("utf-8"))
        if err:
            sys.stderr.write(err.decode("utf-8"))

        if process.returncode:
            msg = f"The command '{' '.join(command)}' exited with code {process.returncode}"
            hint = (
                "" if warn_only else " (use --warn-only to turn this error into a warning)"
            )
            sys.stderr.write(
                f"[jupytext] {'Warning' if warn_only else 'Error'}: {msg}{hint}\n"
            )
            if not warn_only:
>               raise SystemExit(process.returncode)
E               SystemExit: 4

src/jupytext/cli.py:1140: SystemExit
----------------------------------------------------------------------------------- Captured stdout call -----------------------------------------------------------------------------------
[jupytext] Reading /tmp/pytest-of-jeffr/pytest-9/popen-gw2/test_run_pytest_ok0/notebook.md in format md
[jupytext] Executing pytest /tmp/notebook_oo_56at8.py
============================= test session starts ==============================
platform linux -- Python 3.11.7, pytest-8.0.0, pluggy-1.4.0
Using --randomly-seed=419236798
rootdir: /tmp
plugins: xdist-3.5.0, cov-4.1.0, anyio-4.2.0, randomly-3.15.0, xonsh-0.14.4
collected 0 items / 1 error

==================================== ERRORS ====================================
______________________________ ERROR collecting . ______________________________
../miniconda3/envs/jupytext-dev/lib/python3.11/site-packages/pluggy/_hooks.py:501: in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
../miniconda3/envs/jupytext-dev/lib/python3.11/site-packages/pluggy/_manager.py:119: in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
../miniconda3/envs/jupytext-dev/lib/python3.11/site-packages/_pytest/python.py:210: in pytest_collect_directory
    if pkginit.is_file():
../miniconda3/envs/jupytext-dev/lib/python3.11/pathlib.py:1267: in is_file
    return S_ISREG(self.stat().st_mode)
../miniconda3/envs/jupytext-dev/lib/python3.11/pathlib.py:1013: in stat
    return os.stat(self, follow_symlinks=follow_symlinks)
E   PermissionError: [Errno 13] Permission denied: '/tmp/systemd-private-0c4b06cedf23405bad0e7365a61e84d6-systemd-logind.service-bDyMgv/__init__.py'
=========================== short test summary info ============================
ERROR ../../../tmp - PermissionError: [Errno 13] Permission denied: '/tmp/sys...
=============================== 1 error in 0.74s ===============================
----------------------------------------------------------------------------------- Captured stderr call -----------------------------------------------------------------------------------
ERROR: found no collectors for /tmp/notebook_oo_56at8.py

[jupytext] Error: The command 'pytest /tmp/notebook_oo_56at8.py' exited with code 4 (use --warn-only to turn this error into a warning)
===================================================================================== warnings summary =====================================================================================
../miniconda3/envs/jupytext-dev/lib/python3.11/site-packages/sphinx_gallery/sphinx_compatibility.py:71
../miniconda3/envs/jupytext-dev/lib/python3.11/site-packages/sphinx_gallery/sphinx_compatibility.py:71
../miniconda3/envs/jupytext-dev/lib/python3.11/site-packages/sphinx_gallery/sphinx_compatibility.py:71
../miniconda3/envs/jupytext-dev/lib/python3.11/site-packages/sphinx_gallery/sphinx_compatibility.py:71
../miniconda3/envs/jupytext-dev/lib/python3.11/site-packages/sphinx_gallery/sphinx_compatibility.py:71
../miniconda3/envs/jupytext-dev/lib/python3.11/site-packages/sphinx_gallery/sphinx_compatibility.py:71
  /home/jeffr/miniconda3/envs/jupytext-dev/lib/python3.11/site-packages/sphinx_gallery/sphinx_compatibility.py:71: RemovedInSphinx80Warning: The alias 'sphinx.util.status_iterator' is deprecated, use 'sphinx.util.display.status_iterator' instead. Check CHANGES for Sphinx API modifications.
    status_iterator = sphinx.util.status_iterator

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================================================================= short test summary info ==================================================================================
FAILED tests/integration/cli/test_execute.py::test_execute_r - AttributeError: outputs
FAILED tests/functional/others/test_pytest.py::test_run_pytest_ok - SystemExit: 4
============================================================ 2 failed, 3123 passed, 88 skipped, 6 warnings in 563.72s (0:09:23) ============================================================

[github-actions]

Thank you for making this pull request.

Did you know? You can try it on Binder: or .

Also, the version of Jupytext developed in this PR can be installed with pip:

pip install git+https://github.com/jsquaredosquared/jupytext.git@main

(this requires nodejs, see more at Developing Jupytext)

[mwouts]

Hey @jsquaredosquared , thank you for this!

Oh I see that the CI is not passing neither, possibly a pytest update. It may take me a few days to fix that, can I come back to you next week? Long story short, this does not seem related at all to your change. Meanwhile feel free to share the command above if anyone wants to test this development version.

Re you PR, can you revert the change on yarn.lock? Thanks!

[jsquaredosquared]

Thanks Marc!

I have reverted the change as requested.

Looking forward to your fix :D
Jeff

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 97.75%. Comparing base (fe2ba5e) to head (1997365).

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1213      +/-   ##
==========================================
+ Coverage   97.69%   97.75%   +0.06%     
==========================================
  Files          29       29              
  Lines        4464     4464              
==========================================
+ Hits         4361     4364       +3     
+ Misses        103      100       -3     

Flag	Coverage Δ
external	75.08% <ø> (ø)
functional	88.51% <ø> (ø)
integration	77.28% <ø> (ø)
unit	66.56% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[mwouts]

Thank you @jsquaredosquared for your PR! And sorry for the delay in taking it. I will release a new version shortly.

[jsquaredosquared]

Thanks for accepting it! 😀
And thanks for your work on jupytext