Persist Cell Language in Notebook Cell Metadata

[ChrisCompton]

Environment data

Version: 1.54.0-insider
Commit: 93f705ab40b37aade9d3b5165ed09114a8c87ac9
Date: 2021-02-11T05:43:26.672Z (8 hrs ago)
Electron: 11.2.3
Chrome: 87.0.4280.141
Node.js: 12.18.3
V8: 8.7.220.31-electron.0
OS: Darwin x64 20.3.0

Expected behaviour

In a notebook, when I select the language for a cell, it should retain that setting whenever the notebook is opened. This should be maintained even if the kernel is changed because the language highlighting in the cell depends on this selection.

The example shows expected behavior when python and ruby cells are included in a notebook.

Actual behaviour

When I close the notebook file, and reopen it in the same session, the selections seem to be maintained; however, when I close the editor completely, reopen the folder, then open the file, the language selections are reset.

The example shows the ruby kernel selected, but all cells are marked as "Python." Also, notice that the ruby cells are marked with syntax errors based on python.

Steps to reproduce:

The example images show a very basic notebook format that will reproduce the problem.

1. Open a notebook with mixed cells.
2. Make sure that the language selections are accurate.
3. Close the notebook.
4. Close the editor.
5. Start the editor.
6. Open the notebook.
7. All cell language settings will be based on one language.

I'm not sure exactly what determines the language automatically chosen. In the example above, the iruby kernel was selected at close, and was selected when I started the editor, yet all cells were marked as Python.

Lastly, to repeat the behavior stated above, as long as I haven't closed the editor itself, I can open and close notebooks, and switch kernels without impacting the cell language selections.

[DonJayamanne]

Thanks for reporting this issue, this is a known issue.
Currently we try to persis the language once you have executed a cell.
Please could you confirm this works as expected once you have executed a cell, saved the notebook & then reopened it.

Does the Ruby kernel allow you to run Python code as well?

[DonJayamanne]

Related issues:

• Persist languages of cells when re-opening a notebook dotnet/interactive#1038
• Using other kernels that support multiple languages (Jupyter SOS)

[ChrisCompton]

In the samples provided above, the cells had been saved after executing. It did persist when closing the notebook and reopening -- provided you never close the editor. Once you close VS Code, start it, and then open the notebook, the cells appeared as depicted in the second screenshot.

When the ruby kernel is selected, I can run both Ruby and Python in the same notebook as shown. I will say that I can't pass output to the next cell if the languages are different, but other than that, I can run both languages in one notebook.

[greazer]

Check with John Lam about state of polyglot notebook investigation.

[DonJayamanne]

Moving back to triage, as Johns spec doesn't address this, that's a whole different spec (related to execution, not persisting information in cells).

[Nordes]

I think this should be addressed. It's a pity that every time I re-open VsCode I lose all my previous settings for each cells.

Only the markdown seems to be persisted. As the duplicated #4934 mentionned, cells always reset back to python while they were shell script, json, html, etc.

I understand that Jupyter Notebook, by default keep all as "python" then we have to add the %%bash, but it would be very nice to keep track in the metadata of such details.

[tennox]

I also used to have this issue, but for me it is fixed.
Please check again.

[greazer]

The original bug still repro's in vscode insiders 1.61 and an associated jupyter extension.

[DonJayamanne]

Concerns:

• Opening & saving a notebook will now end up with a lot of changes
• Should this be done in vscode or jupyter extension
  • If we have a kernel such as Julia, whats the benefit of saving language, 0 benefit, as we only support julia languages in cells
    • Hence adding this metadata for notebooks with julia kernels is just going to add more noise
  • The opposite for .NET notebooks (they save the language info in the cells, or so i think)
  • However looking at the above screen sample, it seems we have python & ruby cells, looks like we're switching kernels.
  • Hence, i think we should store languages only when a Jupyter kernel is used (kernels/controllers contributed by the Jupyter extension) - upon running.
• If users have cell magics like %%shell, %%javascript, etc, then no point saving the language in the metadata, we can already get that info from the magics
  • Suggestion - store the language in metadata only when necessary
  • For these we can automatically detect the language from the magics (this can be done in vscode core extension)

[storrer]

I think that I am having this issue right now. I create a blank notebook with a powershell cell. I don't select a language kernel. I save the notebook, close it, and open it again. The language of the cell is changed to python. I can't get the language of a cell to save in the notebook editor.

[amunger]

mostly fixed in vs code insiders, though a blank cell will still lose language selection when closing and re-opening the .ipynb file or selecting a different kernel. So more of an edge case now

[claudiaregio]

Hey @amunger, I am still seeing this in stable when working with .net interactive. If I open up a notebook (I've attached it) that has a C# cell and an F# cell, they both open up as C# despite the correct metadata being saved by our extension but I can't reopen microsoft/vscode#140673 as it's limited to collaborators.

default.NETtest.zip

[amunger]

The notebook you attached only has the language specified in the "dotnet_interactive" metadata, which vscode doesn't seem to use to set the cell language, but if I open that file with the latest .net interactive extension installed, it immediately adds the vscode metadata and seems to have the correct language set in the cell (without the .net extension, the cell language is C# for both).

   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "dotnet_interactive": {
     "language": "fsharp"
    },
    "vscode": {
     "languageId": "dotnet-interactive.fsharp"
    }
   },

So I'm not sure how you are managing to save a notebook without that vscode metadata. Is the kernel getting selected correctly?

[jonsequitur]

Right. The issue is that on first open, the Jupyter extension rather than the .NET Interactive extension opens the notebook, and the cell languages are shown incorrectly. The metadata is set as expected but the wrong extension is opening the notebook.

[amunger]

ah, I got that to repro once out of around 10 tries. But how was this file created in the first place where it doesn't have vscode.languageId in the metadata?

[jonsequitur]

It could be an older file from before that metadata was implemented or it could be an .ipynb that was created or edited using an editor other than VS Code.

[DonJayamanne]

@jonsequitur is this still an issue in the latest version of VS Code?

[jonsequitur]

No, this is no longer an issue.

[DonJayamanne]

Thanks, closing this issue for now.

[johnmbarrett]

This issue appears to have regressed as of VS Code v1.74.2. The attached notebook has the language specified as Matlab everywhere possible, but when I open it in VS Code the language indicator in the bottom right briefly reads MATLAB before switching to Python (along with a tonne of red squiggles, because the code is very clearly Matlab and not Python).

Environment info:-

Version: 1.74.2 (user setup)
Commit: e8a3071ea4344d9d48ef8a4df2c097372b0c5161
Date: 2022-12-20T10:29:14.590Z
Electron: 19.1.8
Chromium: 102.0.5005.167
Node.js: 16.14.2
V8: 10.2.154.15-electron.0
OS: Windows_NT x64 10.0.19044
Sandboxed: No

Relevant extentions and their versions:-

Jupyter v2022.11.1003412109
Matlab v2.3.1
Python v2022.20.1