Increasing exploratory adoption

[psychemedia]

Hi

I noticed in https://discourse.jupyter.org/t/jupyter-and-github-alternative-file-formant/4972/13 a comment that you have "received very little feedback up to now" on this format.

Do you know if @mwouts' jupytext has plugin support for adding new formats (I haven't checked).

If it does, a Jupytext plugin for pairing notebooks with a jupyter-format representation would allow folk who are already exploring / making use of alternative text formats to use the format for no additional overhead, and compare it to other formats they're using.

[mgeier]

Thanks for the suggestion!

I don't know whether Jupytext has plugin support, from a quick glance at the source code it looks like all supported formats are hard-coded.

But even without plugin support, it looks like it would be easy to add support for a new format to Jupytext.

I'm a bit hesitant to do that, though.

The problem is that I don't really want people to start using my proposed format for their real-life notebooks. I would like them to try my format, experiment with it and give feedback.

I'm sure there are many things that can be improved, and I would like to be able to make breaking changes once people give feedback.

After a period of feedback and breaking changes, I would like to stabilize the format and only at that point I would like to start caring about backwards compatibility.

[psychemedia]

Sure; stabilizing format is the key thing... I also note the related discussion on the Jupyter discourse forum around the Jupyter and GitHub - alternative file format continues... Interesting times...

Just on matter of trying rather than using - that's partly why I wondered if there's an easy way to extend Jupytext; for me, trying is often a quick play in a demo Binder, then a proper try is "does it work if I try I try to use it...".

I'd also want to be able to try it in context of my normal workflow, not have create a new workflow or do much work before I can even start the trial; yes, trying it does "just" mean adding the content manager, and then maybe removing the content manager after the trial, but that's another to have to (remember to) do and another bit of friction against actually trying it?

[mgeier]

Yeah, installing the content manager causes a bit of friction.

I think Jupytext uses a newer API to register itself to Jupyter, which wasn't available (or I wasn't aware of it) when I worked on jupyter_format initially.

I'll have to check that out, as it will potentially make for a smoother installation process.

[mwouts]

Hi @mgeier , thanks for taking the initiative of jupyter-format!

I personally see the JSON/.ipynb format as computer-friendly (easy to parse, safe, but not easy to edit manually), Markdown and Jupytext formats as user-friendly (easy to edit and write manually but much harder to parse). Is it correct that with the YAML format you want to achieve a format that is both computer and user friendly?

Also, if you wanted to add your format to Jupytext, even locally for testing, that should not be very hard indeed - see for instance my recommendations on how to add the md:myst format, and the subsequent PR.

[mgeier]

Is it correct that with the YAML format ...

It's not actually YAML (see https://jupyter-format.readthedocs.io/en/latest/motivation.html#YAML-as-Almost-Solution), but it is certainly inspired by YAML (and using YAML was the original idea, which several people had before me).

It's much simpler than YAML, and it doesn't need a YAML library.

It just needs nbformat, no other third party libraries.

And it needs json and re from the standard library, no further dependencies.

... you want to achieve a format that is both computer and user friendly?

Yes. And I guess such a goal will always be a balancing act. In case of doubt, I'm leaning towards "user friendly".

The main goal is "diff friendly".
And then it should be "human readable" (which is probably related to "diff friendly"?).
And to some extent, but much less importantly, it should be "somewhat human editable".

I guess most Jupytext formats put more emphasis on the "human editable" part.

From a user's standpoint, I don't really care whether it is "computer friendly", but I think it makes sense to make it as easy to parse as possible, in order to facilitate adoption.

For example in my format all "data" is indented by 4 spaces, so it should be really simple to split a notebook into its "components". The line before each component determines what kind of "data" it is.

That's basically it.

Also, if you wanted to add your format to Jupytext, even locally for testing, that should not be very hard indeed - see for instance my recommendations on how to add the md:myst format, and the subsequent PR.

Thanks!

I've already had a quick look at the code, and it indeed looks very straightforward to add it.

As mentioned above, I don't really want to "officially publish" the format yet, but I think I could make a temporary fork to try out the integration with Jupytext.