Convert all notebooks

[jamesmyatt]

I'd like to be able to convert all notebooks in a directory using jupytext, e.g.

jupytext --to py *.ipynb

Am I correct to say that this is not currently possible? Is it planned to be added? I don't think it's as simple as adding a glob near

jupytext/jupytext/cli.py

Line 205 in 3b2f232

for nb_file in args.notebooks:

[phaustin]

probably best left to the shell:

for i in *ipynb;do
  jupytext --to py $i
done

[mwouts]

Hello @jamesmyatt , well this is implemented. I've used it many times, as this is especially useful for testing the round trip conversion on a large collection of notebooks (see #99). Just cd to a directory where you have a few notebooks, and try it, e.g.

(python3) marc@ubuntu:~/jupytext/tests/notebooks/ipynb_py$ jupytext --version                                                  1.0.3
1.0.3
 
(python3) marc@ubuntu:~/jupytext/tests/notebooks/ipynb_py$ jupytext --to py *.ipynb
[jupytext] Reading Notebook with function and cell metadata 164.ipynb
[jupytext] Writing Notebook with function and cell metadata 164.py
[jupytext] Reading Notebook with html and latex cells.ipynb
[jupytext] Writing Notebook with html and latex cells.py
[jupytext] Reading Notebook with many hash signs.ipynb
[jupytext] Writing Notebook with many hash signs.py
[jupytext] Reading Notebook_with_R_magic.ipynb
[jupytext] Writing Notebook_with_R_magic.py
[jupytext] Reading World population.ipynb
[jupytext] Writing World population.py
[jupytext] Reading convert_to_py_then_test_with_update83.ipynb
[jupytext] Writing convert_to_py_then_test_with_update83.py
[jupytext] Reading frozen_cell.ipynb
[jupytext] Writing frozen_cell.py
[jupytext] Reading jupyter.ipynb
[jupytext] Writing jupyter.py
[jupytext] Reading jupyter_again.ipynb
[jupytext] Writing jupyter_again.py
[jupytext] Reading jupyter_with_raw_cell_in_body.ipynb
[jupytext] Writing jupyter_with_raw_cell_in_body.py
[jupytext] Reading jupyter_with_raw_cell_on_top.ipynb
[jupytext] Writing jupyter_with_raw_cell_on_top.py
[jupytext] Reading notebook_with_complex_metadata.ipynb
[jupytext] Writing notebook_with_complex_metadata.py
[jupytext] Reading nteract_with_parameter.ipynb
[jupytext] Writing nteract_with_parameter.py
[jupytext] Reading sample_rise_notebook_66.ipynb
[jupytext] Writing sample_rise_notebook_66.py

[jamesmyatt]

That's odd and I agree it is useful. I'm also using v1.0.3 but on Windows using Python 3.7.1, and I get:

>jupytext --to py *.ipynb
[jupytext] Reading *.ipynb
Traceback (most recent call last):
  File "C:\Apps\miniconda3\envs\p9380-dev\Scripts\jupytext-script.py", line 9, in <module>
    sys.exit(jupytext())
  File "C:\Apps\miniconda3\envs\p9380-dev\lib\site-packages\jupytext\cli.py", line 227, in jupytext
    notebook = readf(nb_file, fmt)
  File "C:\Apps\miniconda3\envs\p9380-dev\lib\site-packages\jupytext\jupytext.py", line 204, in readf
    with io.open(nb_file, encoding='utf-8') as stream:
OSError: [Errno 22] Invalid argument: '*.ipynb'

I don't see anywhere where it would expand a wildcard in jupytext/cli.py or any mentions of glob anywhere in the code. What am I missing?

[mwouts]

Thanks @jamesmyatt for the clarification. It seems then that the windows shell is behaving differently than the linux one. In linux the wildcard expansion is done by the shell, and jupytext does not get a *.ipynb argument, instead it receives the list of notebooks.

I confirm what you report: jupytext --to py *.ipynb fails on windows. Apparently the wildcard extension there is left to the application. Some commands do implemented it, e.g. dir *.ipynb does work.

So to conclude: you're right, we should add the wildcard extension to jupytext's CLI, at least on Windows.

[mwouts]

This is available in version 1.0.4. @jamesmyatt , can you confirm that it now works as expected? Thanks

[jamesmyatt]

That's great! Thanks for fixing and releasing it so quickly!

[mwouts]

You're welcome - thanks for reporting the issue!

[tysun]

Incase anyone else needs it, I went the other way .py to .ipynb using jupytext --to notebook *.py.

Works a treat.