nb2xls==0.1.6 is not work with mistune==3.0.1

[ivlucky]

Problem

I have to use mkdocs to build pages in gitlab.

When I execute mkdocs serve -w docs I failed with following error:

INFO - DeprecationWarning: Jupyter is migrating its paths to use standard platformdirs
given by the platformdirs library. To remove this warning and
see the appropriate new directories, set the environment variable
JUPYTER_PLATFORM_DIRS=1 and then run jupyter --paths.
The use of platformdirs will be the default in jupyter_core v6
File "/home/i_migal/.local/lib/python3.9/site-packages/jupyter_core/utils/init.py", line 89, in
deprecation
warnings.warn(message, DeprecationWarning, stacklevel=stacklevel + 1)
File "/home/i_migal/.local/lib/python3.9/site-packages/notebook/nbextensions.py", line 15, in
from jupyter_core.paths import (
Traceback (most recent call last):
File "/home/i_migal/.local/bin/mkdocs", line 8, in
sys.exit(cli())
File "/home/i_migal/.local/lib/python3.9/site-packages/click/core.py", line 1130, in call
return self.main(*args, **kwargs)
File "/home/i_migal/.local/lib/python3.9/site-packages/click/core.py", line 1055, in main
rv = self.invoke(ctx)
File "/home/i_migal/.local/lib/python3.9/site-packages/click/core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/i_migal/.local/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/i_migal/.local/lib/python3.9/site-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "/home/i_migal/.local/lib/python3.9/site-packages/mkdocs/main.py", line 272, in serve_command
serve.serve(**kwargs)
File "/home/i_migal/.local/lib/python3.9/site-packages/mkdocs/commands/serve.py", line 54, in serve
config = get_config()
File "/home/i_migal/.local/lib/python3.9/site-packages/mkdocs/commands/serve.py", line 43, in get_config
config = load_config(
File "/home/i_migal/.local/lib/python3.9/site-packages/mkdocs/config/base.py", line 374, in load_config
errors, warnings = cfg.validate()
File "/home/i_migal/.local/lib/python3.9/site-packages/mkdocs/config/base.py", line 231, in validate
run_failed, run_warnings = self._validate()
File "/home/i_migal/.local/lib/python3.9/site-packages/mkdocs/config/base.py", line 188, in _validate
self[key] = config_option.validate(value)
File "/home/i_migal/.local/lib/python3.9/site-packages/mkdocs/config/config_options.py", line 186, in validate
return self.run_validation(value)
File "/home/i_migal/.local/lib/python3.9/site-packages/mkdocs/config/config_options.py", line 1065, in run_validation
self.load_plugin_with_namespace(name, cfg)
File "/home/i_migal/.local/lib/python3.9/site-packages/mkdocs/config/config_options.py", line 1103, in load_plugin_with_namespace
return (name, self.load_plugin(name, config))
File "/home/i_migal/.local/lib/python3.9/site-packages/mkdocs/config/config_options.py", line 1121, in load_plugin
plugin_cls = self.installed_plugins[name].load()
File "/home/i_migal/.local/lib/python3.9/site-packages/importlib_metadata/init.py", line 209, in load
module = import_module(match.group('module'))
File "/usr/local/lib/python3.9/importlib/init.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1030, in _gcd_import
File "", line 1007, in _find_and_load
File "", line 986, in _find_and_load_unlocked
File "", line 680, in _load_unlocked
File "", line 850, in exec_module
File "", line 228, in _call_with_frames_removed
File "/home/i_migal/.local/lib/python3.9/site-packages/mkdocs_jupyter/plugin.py", line 13, in
from . import convert
File "/home/i_migal/.local/lib/python3.9/site-packages/mkdocs_jupyter/convert.py", line 50, in
from .nbconvert2 import *
File "/home/i_migal/.local/lib/python3.9/site-packages/mkdocs_jupyter/nbconvert2.py", line 20, in
from nbconvert.nbconvertapp import NbConvertApp
File "/home/i_migal/.local/lib/python3.9/site-packages/nbconvert/nbconvertapp.py", line 193, in
class NbConvertApp(JupyterApp):
File "/home/i_migal/.local/lib/python3.9/site-packages/nbconvert/nbconvertapp.py", line 252, in NbConvertApp
Options include {get_export_names()}.
File "/home/i_migal/.local/lib/python3.9/site-packages/nbconvert/exporters/base.py", line 145, in get_export_names
e = get_exporter(exporter_name)(config=config)
File "/home/i_migal/.local/lib/python3.9/site-packages/nbconvert/exporters/base.py", line 106, in get_exporter
exporter = items[0].load()
File "/home/i_migal/.local/lib/python3.9/site-packages/importlib_metadata/init.py", line 209, in load
module = import_module(match.group('module'))
File "/usr/local/lib/python3.9/importlib/init.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "/home/i_migal/.local/lib/python3.9/site-packages/nb2xls/init.py", line 1, in
from .exporter import XLSExporter
File "/home/i_migal/.local/lib/python3.9/site-packages/nb2xls/exporter.py", line 18, in
from .mdrenderer import Md2XLSRenderer,
File "/home/i_migal/.local/lib/python3.9/site-packages/nb2xls/mdrenderer.py", line 3, in
from mistune import Renderer, escape, escape_link
ImportError: cannot import name 'Renderer' from 'mistune' (/home/i_migal/.local/lib/python3.9/site-packages/mistune/init.py)

After search I replaced in site-packages following strings in /home/i_migal/.local/lib/python3.9/site-packages/nb2xls/mdrenderer.py:
Renderer -> BaseRenderer
escape_link -> escape_url

And it is worked!

Can you update mistune version in requirements.txt to 3.0.1 or latest version and release new version of nb2xls?

[danlester]

Thanks for your interest!

However, it doesn't quite make sense to update mistune to 3.0.1 in requirements - that would normally be installed anyway on a fresh installation of nb2xls.

The problem is that the code of nb2xls is written assuming mistune version 0.8.4... So, if anything we would need to enforce that version rather than the latest version of mistune.

Or, better would be to update the nb2xls code so that it works with the latest version of mistune.

If taken a quick look at the latter possibility. Unfortunately, it seems the API of mistune has changed quite a bit and I'm not sure I'll have time to adjust the code to match it.

If anyone else is able to do so, ideally so the pytests still pass, then I can accept a PR and publish a new version.

[ivlucky]

Cool!

Can I make some MR into your repository and release 0.1.7?

[ivlucky]

I have updated requirements.txt
requirements.txt
pip freeze result in the following:
freeze.txt
I follow the contribution guide, updated code and run pytest .. Short info is here:

=================================================== short test summary info ===================================================
FAILED tests/test_nb2xls.py::TestsExcelExporter::test_export_compare[ExcelTest4.ipynb-42193] - assert (42090 >= (42193 - 10))
FAILED tests/test_nb2xls.py::TestsExcelExporter::test_export_compare[NestedMarkdown1.ipynb-6225] - assert (5581 >= (6225 - 10))
FAILED tests/test_nb2xls.py::TestsExcelExporter::test_export_compare[ExcelTest.ipynb-5455] - assert (5431 >= (5455 - 10))
FAILED tests/test_nb2xls.py::TestsExcelExporter::test_export_compare[PandasNA.ipynb-6038] - assert (5537 >= (6038 - 10))
FAILED tests/test_nb2xls.py::TestsExcelExporter::test_export_compare[MarkdownReprDisplay.ipynb-5574] - assert (5563 >= (5574 - 10))
FAILED tests/test_nb2xls.py::TestsExcelExporter::test_export_compare[MultipleOutputs.ipynb-6630] - assert (6614 >= (6630 - 10))
FAILED tests/test_nb2xls.py::TestsExcelExporter::test_export_compare[PandasTables.ipynb-7711] - assert (7181 >= (7711 - 10))
=========================================== 7 failed, 2 passed, 1 warning in 3.25s ============================================

Can I fix expected_size in test_nb2xls.py?

[danlester]

Thanks, but please see my last comment... It doesn't make sense to have mistune 3 without also changing the code to match! (Which it sounds like you have done through search/replace somehow in the installed package, but we need to do this actually in the GitHub code.)

If you do want to work on this, please fork this repo and then submit a pull request as described here, or maybe you can find a detailed guide on the internet.

We might end up needing to change the file sizes, but we would first need to prove the results are as desired - i.e. visually inspect that they make sense based on the source notebooks.

[ivlucky]

Yes, you're right. Here is method list to implement in repo.
I'll do it, but need more time...

[danlester]

Yes, it looks like mistune changed a bit between v0.8.4 and v2, and then again to v3. To be honest, I imagine that nb2xls hasn't really worked for most people over many years now... Hopefully you can make it produce the same output as before, but if not quite then as long as it looks similar enough...