Unable to remove lines_to_next_cell metadata

[erdnaavlis]

If I have the following global configs:

c.NotebookApp.contents_manager_class = "jupytext.TextFileContentsManager"
c.ContentsManager.freeze_metadata = True

When I open a .py script that has the following content:

import os


print('hello1')



print('hello2')

Then hit the button "Save and Checkpoint" in the browser interface and open the script again in a text editor, the new content of the script is:

# + {"lines_to_next_cell": 2}
import os
# -


print('hello1')



# + {"lines_to_next_cell": 2}
print('hello2')

I've also tried to have in the global configs the following:

c.NotebookApp.contents_manager_class = "jupytext.TextFileContentsManager"
c.ContentsManager.freeze_metadata = True
c.ContentsManager.default_cell_metadata_filter = "-lines_to_next_cell"

But the behavior is still the same.

Shouldn't the c.ContentsManager.freeze_metadata option save no metadata at all when the original files have no metadata?

If not, how can I remove the lines_to_next_cell metadata in this scenario?

Thanks in advance!

[mwouts]

Hello Andre,
Thanks for reporting this - this is clearly not intended !

For the next release I have been thinking of removing the freeze_metadata option and instead provide a better handling by default of scripts with no metadata (and, as you expect, add no metadata in that case - see also #124).

I am not in a position to test your example right now, but I will later on. Meanwhile, you may want to edit the metadata filter manually (in the notebook metadata), or give a try to the following configuration,

c.NotebookApp.contents_manager_class = "jupytext.TextFileContentsManager"
c.ContentsManager.default_notebook_metadata_filter = "-all"
c.ContentsManager.default_cell_metadata_filter = "-all"

[mwouts]

Hello Andre, I confirm that I have been able to reproduce the issue. I have added a test, and provided a fix in the branch 0.9.0. I will let you know (well, I think GitHub will automatically close this issue) when the new version becomes available - this may well take a few more days.

[erdnaavlis]

Thanks for the great support @mwouts !

[mwouts]

Hello Andre, it has been a while... The new release candidate should solve this. Can you test and confirm? It is available with

pip install jupytext --pre --upgrade

Note that you will have to change your configuration, as the freeze_metadata option was suppressed.

Thanks!

[erdnaavlis]

Hi @mwouts ,

Sorry for the delay!

I've tried to reproduce the behavior I described above and I can confirm that the lines_to_next_cell metadata is no longer present. That's great!

But, I get the ExecuteTime metadata.

For reference, here is the full content:

# + {"ExecuteTime": {"start_time": "2019-02-04T14:32:07.977074Z", "end_time": "2019-02-04T14:32:07.980676Z"}}
import os


# + {"ExecuteTime": {"start_time": "2019-02-04T14:32:08.178820Z", "end_time": "2019-02-04T14:32:08.184917Z"}}
print('hello1')
# -



# + {"ExecuteTime": {"start_time": "2019-02-04T14:32:08.964653Z", "end_time": "2019-02-04T14:32:08.969993Z"}}
print('hello3')
# -

Note that you will have to change your configuration, as the freeze_metadata option was suppressed.

What do I need to do in the config then, to make sure no metadata is saved at all?

[mwouts]

Hello @andrethrill ... well, I am a bit surprised!

My expectations are:

• In the absence of any configuration, ExecuteTime metadata is filtered from the notebook.
• However, if you open a Python script that has some metadata, a metadata filter is created that will preserve the current metadata.

May I ask you to have a look at the notebook metadata using the Jupyter notebook metadata editor? I expect that you will find a non-trivial metadata filter there, which probably explain what you observe.

If you want that Jupytext always save your notebooks with no metadata, use "-all" as the default notebook and cell metadata filters in you Jupyter config file, as documented above. And also, make sure you remove the existing metadata filters in your notebooks. If there are many of them, you may want to use the new --update-metadata argument in Jupytext CLI.

[erdnaavlis]

Hi again @mwouts .

Here's what I've done:

Created a new .py file, added the content:

import os


print('hello1')



print('hello2')

And saved it. Then opened it using the notebook interface. Executed the 3 cells:

Checked the Edit-->Edit Notebook Metadata:

Saved the "notebook". And this is its content:

# + {"ExecuteTime": {"start_time": "2019-02-06T11:53:21.208644Z", "end_time": "2019-02-06T11:53:21.213071Z"}}
import os


# + {"ExecuteTime": {"start_time": "2019-02-06T11:53:21.463238Z", "end_time": "2019-02-06T11:53:21.468873Z"}}
print('hello2')
# -



# + {"ExecuteTime": {"start_time": "2019-02-06T11:53:21.870293Z", "end_time": "2019-02-06T11:53:21.875642Z"}}
print('hello2')
# -

I was expecting the contents of the .py to not change. Do you know what's happening?

In the ~/.jupyter/jupyter_notebook_config.py I have:

c.NotebookApp.contents_manager_class = "jupytext.TextFileContentsManager"
c.ContentsManager.freeze_metadata = True
c.ContentsManager.default_cell_metadata_filter = "-lines_to_next_cell"

(I know you said the freeze_metadata option is deprecated. It's just there for historical reasons.)

[mwouts]

That is interesting, thanks @andrethrill for reporting this so carefully. I will have a look.

[mwouts]

Andre, there was indeed an issue with the combination of the user and the default filter. The fix is on its way. What I advice for now is that you remove the default cell metadata filter filter (as, anyway the metadata lines_to_next_cell is not expected to go to the text representation anymore).

[mwouts]

@andrethrill , this should be solved in the latest rc:

pip install jupytext==1.0.0-rc3

Thanks for reporting this, it was a nasty bug!
Could you please confirm the issue is over, and close this? Thanks

[erdnaavlis]

@mwouts I can confirm the bug is fixed! Thanks for your great support.

[mwouts]

Excellent. Thank you André.