jupytext.write doesn't error if fmt is unknown, but the output file has a known extension

[choldgraf]

Just playing around with MyST -> ipynb -> MyST right now. The ipynb syncing works beautifully! I love it

I am having a hard time getting MyST code cells to render as ipynb code cells.

E.g., this markdown:

---
jupytext:
  formats: ipynb,md:myst
  text_representation:
    extension: .md
    format_name: myst
    format_version: '0.6'
    jupytext_version: 1.4.1
kernelspec:
  display_name: Python 3
  language: python
  name: python3
---

```{code-cell} python3
print('hi')
```


```{code-cell} python3
print('hi')
```

has its ```{code-cell} ``` read in as markdown instead of being converted to Python code cells.

I tried adding double-spaces between them but that didn't do the trick. Anything I am missing?

(I also noticed a weird thing where the YAML header of the MyST markdown file would be read into the notebook as a raw cell at the top, rather than in the notebook metadata...is that expected behavior?)

[mwouts]

Hello @choldgraf , @chrisjsewell , well, I'd suggest adding a test based on your document to either test_ipynb_to_myst.py, or to a new test file test_read_simple_myst.py based on test_read_simple_markdown.py.

As far as I can tell, it's just that the file gets classified as a plain markdown file, not a md:myst one. If we confirm this, the fix should be easy, and lie in guess_format or inner functions.

Meanwhile, @choldgraf , you could give the full format name when doing conversions, e.g.

jupytext --from md:myst --to ipynb notebook.md

[chrisjsewell]

I tried adding to test_ipynb_to_myst.py, and didn't find a problem:

@requires_myst
def test_matches_mystnb_full():
    text = dedent(
        """\
        ---
        jupytext:
            formats: ipynb,md:myst
            text_representation:
                extension: .md
                format_name: myst
                format_version: '0.6'
                jupytext_version: 1.4.1
        kernelspec:
            display_name: Python 3
            language: python
            name: python3
        ---

        ```{code-cell} python3
        print('hi')
        ```


        ```{code-cell} python3
        print('hi')
        ```
        """
    )
    print(myst_to_notebook(text))
    assert matches_mystnb(text) is True

try running myst_to_notebook(text) and see what you get @choldgraf

[chrisjsewell]

I also noticed a weird thing where the YAML header of the MyST markdown file would be read into the notebook as a raw cell at the top

Are you sure it doesn't start on the 2nd line or something?

The only other thing I can think of is that you are running in windows right? Perhaps it could be something to do with line ending in \r rather than \n

[chrisjsewell]

Oh I've just noticed you are using myst-parser v0.6, you should be using >= 0.7.1
This in the requirements:

jupytext/setup.py

Line 39 in a0676d7

extras_require={"myst": ["myst-parser~=0.7.1; python_version >= '3.6'"]},

but no specific check is done for a lowest version within jupytext (just that you are using the same minor version).

[choldgraf]

ahhh yes, I believe that myst-parser was a key issue here. I've upgraded it and it now works much better! Amazing :-)

One thing that I think exacerbated my problem, if I write the notebook to markdown and give a fmt= value that jupytext doesn't have, then jupytext still writes the format just fine. E.g., this works:

I think what it is doing is inferring the output type from the .md extension of teh output, because if I remove the .md then it throws an error. However, I think jupytext should throw a warning if it doesn't understand the fmt= value, and instead is going to infer from the file extension. This was confusing me, because I was writing jupytext.write(ntbk, "out.md", fmt="myst"), when I should have been writing jupytext.write(ntbk, "out.md", fmt="md:myst"), but because no error or warning was thrown, it was just defaulting to "regular" markdown.

[mwouts]

Thanks @choldgraf , that is a good point... I agree that jupytext.write should raise an error in that case. I'll see how to add a test and fix that.