passing environment variables to the kernel

[keewis]

xref pydata/xarray#3567

I'm trying to pass PYTHONPATH from conf.py to the kernels in order to make sure the notebook is executed using the source checkout instead of a version pulled by dependencies.

I tried to modify $PYTHONPATH (as recommended in the related #96) by editing os.environ in conf.py, which seems to work:

root = pathlib.Path(__file__).absolute().parent.parent
sys.path.insert(0, str(root))
os.environ["PYTHONPATH"] = str(root)

is this something I can rely on? If so, maybe you should add that to your documentation?

[mgeier]

TBH, I don't know.

nbsphinx is simply calling nbconvert.preprocessors.ExecutePreprocessor in whatever context Sphinx extensions (source parsers, to be more specific) are running.

I don't know whether ExecutePreprocessor provides a way to explicitly provide environment variables. Or whether there are any guarantees regarding os.environ.
You would have to ask the nbconvert team about that.

There is a related option https://nbsphinx.readthedocs.io/en/latest/usage.html#nbsphinx_execute_arguments, but I guess this doesn't help in your case?

[keewis]

I looked into ExecutePreprocessor, which uses jupyter_client.KernelManager to start the kernels. KernelManager.start_kernel passes the contents of os.environ to the kernel (after removing PYTHONEXECUTABLE): this is how the kernels receive the environment variables. So I guess unless the source parser context changes os.environ that's a yes?

[mgeier]

I guess so.

It's really hard to give any guarantees, because there are so many moving parts.

But I'd say it seems to be currently a reasonable way to achieve what you want.
Whether it will work in the future, nobody knows.

If so, maybe you should add that to your documentation?

You should add that to the documentation!

There is already a section "Controlling Notebook Execution" where this would fit.

[keewis]

I'm trying to figure out where to add this: would that be a new subsection (i.e. linking to a new notebook in executing-notebooks.ipynb)?

[mgeier]

Yes, I think a new notebook would be nice for that.

This new notebook could probably also mention nbsphinx_execute_arguments (or not?).