Use jupytext with jupyterlab-gitlab or jupyterlab-github

[beenje]

The jupyterlab-github and jupyterlab-gitlab allow to access notebooks from GitHub or GitLab repositories. Notebooks can be run directly from the remote repo.

I installed jupytext and local .md and .py files can be opened as notebooks by using the "Open With -> Notebook" menu.
But when using the jupyterlab-gitlab extension, those files are not recognised as notebooks and can only be opened in the editor.

Do you know if there is a way to make those extensions play well with jupytext so that .md file can be opened directly as notebook?

[mwouts]

Hello @beenje , well this is an interesting question. So, if I understand correctly, the extension allows you to browse github or gitlab repo (cf. your screenshot there), but you don't have the context menu there? Also, can you tell us whether Markdown files have a text, or a notebook icon?

To give some more context:

• The classification of text files as notebooks is done by Jupytext's content manager
• Jupyter notebook opens all notebook files as notebooks
• But, currently, Jupyter lab only open ipynb notebooks as notebooks. So for the other notebooks (.md, etc) we have to use the context menu.

Note that in the future I'd like to allow Jupyter Lab to open the text files which are explicitly notebooks as notebooks by default (I mean, without the right click), see #271 and #340.

[beenje]

The icon of the Markdown files in the GitLab repository is not changed. It still appears as text icon (not notebook).
The GitLab and GitHub extensions defines a Drive that implements Contents.IDrive. I guess this doesn't use Jupytext's content manager.
Not sure how we can plug it there.

Maybe @ian-r-rose who developed the GitHub extension (that I just copied) has an idea?

[mwouts]

Oh good, so this issue may be independent of the other issues I linked above. Sure, we need to understand how the idrive interface works.

According to the documentation, The interface for a network drive that can be mounted in the contents manager, I would be tempted to think that the file is still read by the contents manager, but maybe model['type'] is overwritten in IDrive at some point?

[beenje]

There is a gitLabContentsToJupyterContents function to convert the content from the repo to Jupyter contents. In that function we force the type to "file". But it still works for notebooks (they have the proper icon). So I probably don't understand how everything works together...

[ian-r-rose]

The IDrive interface is purely client-side. The jupyterlab-github extension does have a small server-side component, but it is currently just used to attache authentication information to the GitHub requests. The same appears to be the case for jupyterlab-gitlab (and thanks for making that @beenje!).

For that reason, I would think that using the jupytext contents manager would be pretty challenging. The GitHub/Lab extensions as it stands do not ever hit the Python contents manager.

I don't know much about the implementation of jupytext: how does it communicate that a markdown file is intended to be opened as a notebook?

[mwouts]

Hello @ian-r-rose , thanks for joining the conversation!

For that reason, I would think that using the jupytext contents manager would be pretty challenging. The GitHub/Lab extensions as it stands do not ever hit the Python contents manager.

I agree. But that's a nice challenge. The most difficult part is certainly to find out how to connect to the (Python) contents manager. Currently I have no pointer - I don't even know how to detect in TypeScript that the contents manager is Jupytext's one (#350)... maybe that could be a simpler target to start with?

I don't know much about the implementation of jupytext: how does it communicate that a markdown file is intended to be opened as a notebook?

Jupytext's contents manager open the files as notebooks when get is called with a path that has a Jupytext extension (.md, .Rmd, .py, ....). To do this, we temporarily replace (mock) nbformat.reads, with jupytext.reads, and call the parent method _notebook_model, see this extract:

jupytext/jupytext/contentsmanager.py

Lines 285 to 301 in a04f548

	def get(self, path, content=True, type=None, format=None, load_alternative_format=True):
	""" Takes a path for an entity and returns its model"""
	os_path = self._get_os_path(path.strip('/'))
	ext = os.path.splitext(path)[1]
	# Not a notebook?
	if (not self.exists(path) or os.path.isdir(os_path) or
	(type != 'notebook' if type else ext not in self.all_nb_extensions())):
	return super(JupytextContentsManager, self).get(path, content, type, format)
	fmt = preferred_format(ext, self.preferred_jupytext_formats_read)
	if ext == '.ipynb':
	model = self._notebook_model(path, content=content)
	else:
	self.set_default_format_options(fmt, read=True)
	with mock.patch('nbformat.reads', _jupytext_reads(fmt)):
	model = self._notebook_model(path, content=content)

[mwouts]

@ian-r-rose , @beenje , as you explained the Python contents manager is not called when we open a document with either the github or the gitlab extension. Now, the only thing that we have to do to convert a text document to a notebook is to call e.g. jupytext.reads(text, fmt='md') where text is the file content. Do you think we could use a server extension for that?