Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move backend mapping to Matplotlib #14311

Closed
ianthomas23 opened this issue Jan 30, 2024 · 4 comments · Fixed by #14371
Closed

Move backend mapping to Matplotlib #14311

ianthomas23 opened this issue Jan 30, 2024 · 4 comments · Fixed by #14371

Comments

@ianthomas23
Copy link
Collaborator

I wanted to draw your attention to matplotlib/matplotlib#27663, about moving the Matplotlib backend mappings out of IPython and into Matplotlib.

The primary use case it so support Matplotlib widgets (ipympl and matplotlib-inline) registering themselves as Matplotlib backends without requiring additional code in IPython and/or Matplotlib. The secondary use case is to support backends in IPython using Matplotlib's module://name.of.the.backend syntax, e.g.

%matplotlib module://mplcairo.backend

which one can already do using matplotlib.use(...) but not directly via the %matplotlib magic.

Whilst doing this it seems sensible to bring all of the backend registering and mapping together in one place, and that should be Matplotlib rather than IPython. I am not sure how easy (or even possible!) it will be to remove all the related hard-coded stuff in IPython, but I am willing to start and see how it goes.

@ivanov
Copy link
Member

ivanov commented Jan 30, 2024

Thanks for opening this up for awareness, @ianthomas23 ! This sounds like a good plan to me. I think the only modification we'd want is to keep the current hardcoded functionality to support older versions of Matplotlib that will not have the list_backends, as there are users who update some packages while pinning others, so we would not want to break them by assuming that when they get a newer IPython they will also get a recent Matplotlib.

@Carreau
Copy link
Member

Carreau commented Jan 31, 2024

+1, I have no issue hardcoding a matplotlib version check in IPython to either fall to the hardcoded path or assuming things get registered.

@ianthomas23
Copy link
Collaborator Author

Thanks both. Yes, I'll put in a check on matplotlib.__version__ and use the existing code for "old" Matplotlib.

I have a concern about backend2gui. It won't be needed in future (apart from the above fallback) but I wonder if other downstream libraries might be using it such that it cannot just be removed. Maybe I need to add a shim so that use of backend2gui does the right thing by calling Matplotlib as necessary.

@Carreau
Copy link
Member

Carreau commented Jan 31, 2024

We can add a getattr to the module level that warns about pending deprecation on access.

Carreau added a commit that referenced this issue Apr 12, 2024
This is WIP to move IPython's backend mapping into Matplotlib and
extends it to allow backends to register themselves via [entry
points](https://setuptools.pypa.io/en/latest/userguide/entry_point.html#entry-points-for-plugins).
It is a coordinated effort across Matplotlib, IPython, matplotlib-inline
and ipympl. Closes #14311. Most of the work is in Matplotlib
(matplotlib/matplotlib#27948) but I will repeat the relevant points
here.

When using a Matplotlib magic command of the form `%matplotlib
something` the identification of the Matplotlib backend and GUI loop are
now performed in Matplotlib not IPython. This supports:

1. Matplotlib backends that IPython already supports such as `qtagg` and
`tkagg`.
2. Other built-in Matplotlib backends such as `qtcairo`.
3. Backends that use `module://something` syntax such as
`module://mplcairo.qt`.
4. Backends that self-register using entry points, currently `inline`
and `widget`/`ipympl`.

Implementation details:

1. The magic command is now explicitly `%matplotlib gui_or_backend`
rather than `%matplotlib gui`. This is already effectively the case as
e.g. `%matplotlib ipympl` is really a backend not GUI name. Within
Matplotlib the `gui_or_backend` is checked first to see if it is a GUI
name and then falls back to checking if it is a backend name.
2. If you select the `inline` backend the corresponding GUI is now
`None` not `inline`. All backends which do not have a GUI loop return
`None`, otherwise we have to keep explicit checks within IPython for
particular backends.
3. `backends` and `backend2gui` are now deprecated but are still
exposed, with a deprecation warning, if required. If using Matplotlib,
ipympl, etc releases that include the corresponding changes to this PR
then they are not needed as Matplotlib deals with it all. But for
backward compatibility they must still be available for a while along
with the remainder of the existing backend-handling code.
4. I haven't yet updated the documentation but we need to keep
information about valid GUI frameworks and I propose that we should
remove all lists of valid Matplotlib backends, replacing them with
instructions on how to obtain the current list (pointing to the
Matplotlib docs and using `%matplotlib --list`). If we keep any lists
then they will inevitably become out of date. This extends to the
`backend_keys` in IPython/core/shellapp.py.

Because the four related projects are loosely coupled without direct
dependencies on each other (except for `ipython` and
`matplotlib-inline`), backward compatibility requires all possible
combinations of projects before and after the new functionality (I will
call these "old" and "new" from now on) to continue to work. I have
tested these all locally, and the CI of this PR will test new IPython
against old Matplotlib for example, but I need to add one or more new
temporary CI runs to test new IPython against new Matplotlib etc. The
identification of new versus old depends on version checks on the other
libraries, so here is a table that I will update showing the current
status of progress in the 4 projects:

| Project | Relevant PRs | Possible release version |
| --- | --- | --- |
| matplotlib-inline | ipython/matplotlib-inline#34,
ipython/matplotlib-inline#35 | 0.1.7 |
| ipympl | matplotlib/ipympl#549 | 0.9.4 |
| Matplotlib | matplotlib/matplotlib#27948 | 3.9.1 |
| IPython | #14371 (this) | 8.24.0 |

The two widget projects can be released soon, once we are happy with the
entry point approach. The other two projects' PRs will have to be
synchronised as each includes version checks on each other.

To do

- [ ] Add CI runs against the new PR branches of the other projects.
- [ ] Add comments for conditions required for backward-compatibility
code blocks to be removed.
- [ ] Update documentation, including removal of lists of valid
backends.
- [ ] Update version checks before merging.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants