Side-by-side editing of notebook and markdown in JupyterLab

[gnestor]

Hi! Great work on jupytext! I'm wondering if it's possible or how difficult it would be to implement the following for JupyterLab:

• Open a notebook
• Open the same notebook as markdown
• Split the panels so that they're side-by-side
• Save one and the other updates

I'm imagining something similar to JupyterLab's Markdown Preview extension, but instead of rendering markdown we're using jupytext to update the notebook when the markdown changes or update the markdown when the notebook changes.

[dsblank]

Is this the same as #86 ?

[mwouts]

Hello @gnestor , thanks for your feedback.

Well I just started learning Javascript (the language used for coding JupyterLab extensions), so I think it would not be reasonable for me to plan offering a complex extension anytime soon.

Yet... I think what you ask for is already available! Once the two documents are open, edit one of them, save it (Ctrl+S) and simply refresh Jupyter (Ctrl+R). I was surprised to see that even the positions in the documents are preserved in the operation...

[gnestor]

Ok, when I get some time I will give this a shot. I'm imagining the UX being very similar to the Markdown Preview extension: right-click a notebook, open as Markdown (or Python, R, etc.), and as the notebook or markdown file changes, it will use jupytext to update the other (without requiring the page to be refreshed). In order to use jupytext, I believe that the notebook will need to be paired, so the extension can take care of that.

[mwouts]

Great! I'll be happy to follow. Actually, maybe there could be a simple implementation for this - when we observe that one document is saved, we automatically refresh the views on the other paired files.

In the long term, we could consider implementing Jupytext in javascript and use that to maintain the two documents up-to-date with each other, without even needing to save them. Such a javascript implementation would also be useful in other contexts like Hydrogen for Atom, or VScode (#143)

[choldgraf]

this would be awesome <3

[nathancarter]

Like others, I wanted this exact same thing. My goal workflow was:

• edit .md file in Atom in left pane
• when you want to recompile, hit some keystroke
• Atom runs whole .md as an .ipynb and previews result as .html in right pane

I found that this goal can be 95% achieved with existing Atom packages and a Makefile, which I hacked up today. If anyone wants it, it's in this gist.

[mwouts]

Well done! Thanks @nathancarter for sharing this.

Recently I've been thinking if and how Jupytext should execute the notebook. Your use case is interesting... Maybe you would you like Jupytext to watch the .md file, and execute only the cells that are modified? And also update the .html? If you like we can discuss that further on at #231

[nathancarter]

Actually, the fact that it was just a 95% solution bothered me, so here's a better one:
https://github.com/nathancarter/md-jupyter-html

I'll also take a look at #231.

[mwouts]

Hi @nathancarter , that's great, thanks again for sharing!

I am not aware of any other alternative for Markdown notebooks (other than RStudio - did you gave a try to Rmd notebooks in Python?).

Still there are a series of IDE that have adopted the idea of notebooks as scripts. Have you seen them? Are there any idea there that you would like to consider for your plugin?

• PyCharm Professional 2019 opens .ipynb files as python scripts with #%% cells. You can execute these cells and see the notebook being updated. You can set breakpoints. The drawbacks I have seen are:
  • cell metadata may be lost in the operation, the updated .ipynb file is not formatted in the same way as the original .ipynb file
  • javascript outputs (like plotly) are not supported
  • only works for Python kernels
  • and, you need a license for PyCharm Professional
• The Hydrogen plugin for Atom. Files with # %% cells are treated as notebooks. Works for every Jupyter language/kernel. Even with JS outputs. I am not sure you can save the local outputs the the ipynb file, but still you can import and export to ipynb
• VScode can now import/export Jupyter notebooks as # %% Python scripts, see https://code.visualstudio.com/docs/python/jupyter-support
• Spyder has treated #%% files as notebooks for a long time already

[nathancarter]

I did know that knitr supports many languages besides just R, including Python and Julia. So in some sense, if your needs are limited to those three, then the mature and excellent RStudio solution is almost certainly the best option.

The other solutions that you mentioned, however, are of a different ilk. They all treat the notebook file (.ipynb) as the primary source, and give you ways to interact with it as if it were a script. But a notebook as the primary source irritates people who like the benefits of a fully-functioning IDE. And interacting with it as a script de-emphasizes the literate/explanatory/storytelling features that the notebook supports, which are so important to a computational narrative. So none of those solutions is ideal.

I really like the fact that writing a .md file with code in it is telling a code story, preferably with at least as much Markdown as Python. This is especially true in education, which is the industry I'm in, but applies elsewhere as well. It also means that with the .md file as the primary source, it's easy to read diffs in version control.

These are all the reasons I like things like .Rmd/knitr, and the imitation of it in the build plugin I posted above.