Error: the git index is outdated for pre-commit with new file and pure VSCode

[yasirroni]

Use case:

I'm trying pre-commit hook on jupytext. The config is like this:

  # jupytext pair to .py files
  - repo: https://github.com/mwouts/jupytext
    rev: v1.13.8
    hooks:
      - id: jupytext
        args: [
          '--sync',
          '--set-formats',
          'ipynb,py:percent',
          '--'
        ]
        files: ^notebooks/

With no pyproject.toml just yet.

---

Adding new notebook from VSCode, did not use jupyter nor jupyterlab (only jupyter from VSCode to do edits, it throws error:

[jupytext] Error: the git index is outdated.
Please add the paired notebook with:
git add notebooks/test.ipynb
[jupytext] Updating the timestamp of notebooks/test.py

Run git add notebooks/test.ipynb did not fix. Anyway, files are already staged anyway.

---

Further investigating, for each push, the pre-commit hook update cell id in metadata:

{
	"language": "markdown",
	"custom": {
		"metadata": {},
		"id": "89e7c1ce"
	}
}

Both pre-commit and pip use jupytext-1.13.8

[yasirroni]

Note I'm using this pre-commit: https://pre-commit.com/

[yasirroni]

Furthermore, jupytext also deletes all notebook cells outputs! Even from file that already synced with jupytext from jupyterlab, that already tracked by git for a long time.

---

I;m trying to add jupytext pre-commit hook in an already estabilished repository.

Previously, I'm relying on

jupytext --sync notebooks/test.ipynb

[mwouts]

Hi @yasirroni , thank you for reporting this. This is very interesting.

I have been able to reproduce what you describe in a test (first commit of main...reproduce_967).

A workaround is to restrict the hook to .ipynb files (second commit in that branch).
Alternatively, another option is to use a config file (this is tested in https://github.com/mwouts/jupytext/blob/main/tests/test_pre_commit_1_sync_with_config.py)

I suspect this reveals another undesirable behavior #969 - I'll confirm and see what I can do about that.

[mwouts]

The root issue for this problem was #969, it has been addressed in jupytext==1.14.0. Would you mind to confirm that the pre-commit works as expected with that version? Thanks

[yasirroni]

The root issue for this problem was #969, it has been addressed in jupytext==1.14.0. Would you mind to confirm that the pre-commit works as expected with that version? Thanks

It works!

[yasirroni]

Anyway, my workflow consisting of jupytext and various pre-commit. What is the best approach to make jupytext reupdate the .ipynb file after the .py file is modified by other pre-commit? The sync arguments sadly didn't reupdate the ipynb file after the the generated .py file modified by other pre-commit.

I've read pipe arguments, but it seems not abstract and can't accept arbitrary pre-commit hook?

Does running jupytext twice (the first and the last) is the best approach (haven't tested this yet)? Thank you.