HTML Display in .ipynb using R kernel

[Naveen-Annam-CMA]

Issue

I'm trying to display some HTML table generated by the DT library, and save a notebook that uses an R kernel. I'm using the workaround detailed in this SO question, which uses display_html to display the HTML generated by DT.

JupyterLab does not allow to save the .ipynb file, giving the following error message

File Save Error for Untitled.ipynb
HTTP 500: Internal Server Error (Unexpected error while saving file: user/naveen/Untitled.Rmd Additional properties are not allowed ('source' was unexpected) Failed validating 'additionalProperties' in display_data: On instance['cells'][1]['outputs'][0]: {'data': {'text/html': '<iframe src="m.html" width=100%% ' 'height=450></iframe>'}, 'metadata': {}, 'output_type': 'display_data', 'source': 'R display func'})

Also, this message keeps popping up

File Changed
The file has changed on disk since the last time it was opened or saved. Do you want to overwrite the file on disk with the version open here, or load the version on disk (revert)?

If I try to close and reopen the .ipynb notebook, I get the following message:

File Load Error for Untitled.ipynb
user/naveen/Untitled.ipynb (last modified 2019-05-22 15:38:14.867000+00:00) seems more recent than user/naveen/Untitled.Rmd (last modified 2019-05-22 15:32:24.342000+00:00) Please either: - open user/naveen/Untitled.Rmd in a text editor, make sure it is up to date, and save it, - or delete user/naveen/Untitled.Rmd if not up to date, - or increase check margin by adding, say, c.ContentsManager.outdated_text_notebook_margin = 5 # in seconds # or float("inf") to your .jupyter/jupyter_notebook_config.py file

Despite everything, the .Rmd file looks to be correct.

Steps to reproduce
Using jupytext 1.1.1 and c.ContentsManager.default_jupytext_formats = "ipynb,Rmd" in notebook config.

1. Create a new notebook with an R kernel.
2. Paste the following code

library(DT)
mtcars2 = mtcars[, c(1:5, 9)]
mtcars2$am = factor(mtcars$am, c(0, 1), c('automatic', 'manual'))
# search for Ma or Me
m <- datatable(
  mtcars2, colnames = c('model' = 1),
  filter = list(position = 'top', clear = FALSE),
  options = list(
    search = list(regex = TRUE, caseInsensitive = FALSE, search = 'M[ae]'),
    pageLength = 5
  )
)

library(IRdisplay)
htmlwidgets::saveWidget(m, "m.html")
display_html('<iframe src="m.html" width=100% height=450></iframe>')

3. Wait a few seconds for the error message 'File Changed' to appear

4. Try saving the .ipynb file for the 'File Save Error' message to appear

5. Close and reopen the .ipynb for the 'Load Error' message to appear

[Naveen-Annam-CMA]

Even simpler code to reproduce the error (replaces point 2 above):

library("ggplot2")
ggplot(mtcars, aes(mpg)) + stat_ecdf()

[Sorry but getting errors uploading screenshot.]

[mwouts]

Hello @Naveen-Annam-CMA , thanks for reporting this.
I do not have a R kernel right now for testing this, but I will certainly have a look at this in the next days.

By the way, an user with the same config as yours reported an issue with the chunk language (#227), you may want to update to version 1.1.2.

[Naveen-Annam-CMA]

Hello @Naveen-Annam-CMA , thanks for reporting this.
I do not have a R kernel right now for testing this, but I will certainly have a look at this in the next days.

By the way, an user with the same config as yours reported an issue with the chunk language (#227), you may want to update to version 1.1.2.

Sorry, we are actually using version 1.1.3.

[mwouts]

Good. I was able to reproduce the issue.

Jupytext fails to copy the notebook and save it as a Rmd file because the notebook cannot be validated. I do not know exactly what that means, but I will have a look into that. The stacktrace is

      File "/home/marc/.conda/envs/python3/lib/python3.7/site-packages/jupytext/jupytext.py", line 110, in writes
        nb = new_notebook(cells=nb.cells, metadata=deepcopy(metadata or nb.metadata))
 (...)
    nbformat.validator.NotebookValidationError: Additional properties are not allowed ('source' was unexpected)

Do we agree that we get a similar warning (not fatal, though) when you only save the ipynb, without pairing to the R Markdown file? In mean, in that case I see that the console says:

[W 22:45:16.850 LabApp] HTTP 500: Internal Server Error (Unexpected error while saving file: Untitled.Rmd Additional properties are not allowed ('source' was unexpected)

    Failed validating 'additionalProperties' in display_data:

    On instance['cells'][0]['outputs'][1]:
    {'data': {'image/png': 'iVBORw0KGgoAAAANSUhEUgAAA0gAAANICAMAAADKOT/pAAAC2VBMVEUAAAABAQEC...',
              'text/plain': 'plot without title'},
     'metadata': {},
     'output_type': 'display_data',
     'source': 'R display func'})

[mwouts]

This seems to be related to IRkernel/IRkernel#551.

In my tests I have used IRkernel from conda, and it comes in version 0.8.12. Yet I see that IRkernel also exists in version 1.0. Which version are you using? Can you get rid of the validation error/warning (with Jupytext off) with the latest version of IRkernel?

[Naveen-Annam-CMA]

Thank you for your response. Updating the IRkernel package to version 1.0 seems to have resolved the issue.

[mwouts]

Great news! Well that was an interesting issue. On Jupytext's side I will see if I can make it robust to non-fully valid notebooks, I mean, since Jupyter does not fail, Jupytext should not either...