Tests hanging with serialisation failure in test_save_aggregate_slice

[dhomeier]

Describe the bug
Almost all tests on Python > 3.8 (except the dev ones) have been hanging and timed out after 6 hours for several weeks (basically since 1.12.0, it seems) in test_data_viewer, e.g.
https://github.com/glue-viz/glue/actions/runs/5393566950/job/14600937184

Running locally I found a failure in

__________________________________________________________ TestImageViewer.test_save_aggregate_slice ___________________________________________________________

self = <glue.viewers.image.qt.tests.test_data_viewer.TestImageViewer object at 0x15f5238d0>
tmpdir = local('/private/var/folders/bb/f_z2vsjd21d_p0jpqxzhhwhh00027w/T/pytest-of-derek/pytest-5/test_save_aggregate_slice0')

    def test_save_aggregate_slice(self, tmpdir):
    
        # Regression test to make sure that image viewers that include
        # aggregate slice objects in the slices can be saved/restored
    
        self.viewer.add_data(self.hypercube)
        self.viewer.state.slices = AggregateSlice(slice(1, 3), 10, np.sum), 3, 0, 0
    
        filename = tmpdir.join('session.glu').strpath
    
        self.application.save_session(filename)
        self.application.close()
    
>       app2 = GlueApplication.restore_session(filename)

glue/viewers/image/qt/tests/test_data_viewer.py:679: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
glue/app/qt/application.py:1135: in restore_session
    ga = Application.restore_session(path)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

path = '/private/var/folders/bb/f_z2vsjd21d_p0jpqxzhhwhh00027w/T/pytest-of-derek/pytest-5/test_save_aggregate_slice0/session.glu'

    @staticmethod
    def restore_session(path):
        """
        Reload a previously-saved session
    
        Parameters
        ----------
        path : `str`
            Path to the file to load.
    
        Returns
        -------
        app : :class:`Application`
            The loaded application.
        """
        from glue.core.state import GlueUnSerializer
    
        # In case relative paths are needed in the session file, we do the
        # loading while setting the current directory to the directory
        # in which the session file is so that relative paths are interpreted
        # as relative to the session file.
        start_dir = os.path.abspath('.')
        session_dir = os.path.dirname(path) or '.'
        session_file = os.path.basename(path)
    
        try:
            os.chdir(session_dir)
>           with open(session_file) as infile:
E           FileNotFoundError: [Errno 2] No such file or directory: 'session.glu'

glue/core/application_base.py:155: FileNotFoundError

together with a pop-up error window (on which the CI probably hangs)

Failed to save session
Don't know how to serialize <function sum at 0x104ff6b30> of type <class 'numpy._ArrayFunctionDispatcher'>

Traceback (most recent call last):
  File "/Users/derek/git/glue/glue/core/application_base.py", line 23, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/derek/git/glue/glue/core/application_base.py", line 121, in save_session
    state = gs.dumps(indent=2)
            ^^^^^^^^^^^^^^^^^^
  File "/Users/derek/git/glue/glue/core/state.py", line 421, in dumps
    result = self.dumpo()
             ^^^^^^^^^^^^
  File "/Users/derek/git/glue/glue/core/state.py", line 400, in dumpo
    return self.do_all()
           ^^^^^^^^^^^^^
  File "/Users/derek/git/glue/glue/core/state.py", line 325, in do_all
    result = [(oid, self.do(obj)) for oid, obj in list(self._objs.items())]
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/derek/git/glue/glue/core/state.py", line 325, in <listcomp>
    result = [(oid, self.do(obj)) for oid, obj in list(self._objs.items())]
                    ^^^^^^^^^^^^
  File "/Users/derek/git/glue/glue/core/state.py", line 353, in do
    result = fun(obj, self)
             ^^^^^^^^^^^^^^
  File "/Users/derek/git/glue/glue/core/state.py", line 583, in _save_tuple
    return dict(contents=[context.do(item) for item in state])
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/derek/git/glue/glue/core/state.py", line 583, in <listcomp>
    return dict(contents=[context.do(item) for item in state])
                          ^^^^^^^^^^^^^^^^
  File "/Users/derek/git/glue/glue/core/state.py", line 353, in do
    result = fun(obj, self)
             ^^^^^^^^^^^^^^
  File "/Users/derek/git/glue/glue/viewers/image/state.py", line 59, in __gluestate__
    function=context.do(self.function))
             ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/derek/git/glue/glue/core/state.py", line 351, in do
    fun, version = self._dispatch(obj)
                   ^^^^^^^^^^^^^^^^^^^
  File "/Users/derek/git/glue/glue/core/state.py", line 380, in _dispatch
    raise GlueSerializeError("Don't know how to serialize"
glue.core.state.GlueSerializeError: Don't know how to serialize <function sum at 0x104ff6b30> of type <class 'numpy._ArrayFunctionDispatcher'>

py38 is still on Numpy 1.24.4 while the newer envs have 1.25.0, so that is probably behind it; but the dev jobs passing indicates that there is some other dependency that needs to be updated.

[dhomeier]

Ah, the reason is that dev had been already pinned to
git+https://github.com/numpy/numpy@9b6a7b4f8
since this already came up in #2353 , so the dev jobs are behind numpy 1.25 by now!

So really time to fix this, or possibly pin the other jobs to numpy < 1.25 until we've found a solution.

[astrofrog]

We shouldn't really be trying to serialize np.sum - since I think it's only a few possible functions that need to get saved to the session file we could always map them to strings such as 'numpy.sum' and then restore them back as functions.