Interpolation to/from sub-meshes causes parallel deadlock

[garth-wells]

PR #3114 causes deadlock in parallel, see https://github.com/FEniCS/dolfinx/actions/runs/8838261829.

Also see #3114 (comment).

The logic in

dolfinx/python/dolfinx/fem/function.py

Line 415 in 59449a0

def interpolate(

and in

dolfinx/python/dolfinx/fem/function.py

Line 468 in 59449a0

def _(expr: Expression, cells: typing.Optional[np.ndarray] = None):

is muddled and conflates None and len(foo) == 0.

[jorgensd]

Issue is that since we do not associate any relationships between meshes in the objects, it gets problematic to distinguish between non-matching meshes and a sub-mesh parent mesh relation when one has no cells on a process for:

1. Have parent cells, no submesh cells

2. No parent cells, no submesh cells (no cells for either nonmatching grid).

3. Is easy to fix, 2, however, is challenging to fix without adding a flag to interpolate stating if we want to call non-matching interpolation (as this uses mpi communication).

[garth-wells]

Issue is that since we do not associate any relationships between meshes in the objects, it gets problematic to distinguish between non-matching meshes and a sub-mesh parent mesh relation when one has no cells on a process for:

1. Have parent cells, no submesh cells
2. No parent cells, no submesh cells (no cells for either nonmatching grid).
3. Is easy to fix, 2, however, is challenging to fix without adding a flag to interpolate stating if we want to call non-matching interpolation (as this uses mpi communication).

The first problem seems to be that the code conflates None and len(foo) == 0, which breaks when a sub-mesh has no cells on a rank. This is a failure in logic.

The second is a design failure - one function doing too many different things. And it's not clear from the argument descriptions what is going on.

Unless there is an easy and good fix, the change should probably be wound back until it can done properly.

[jorgensd]

Working on a fix where a user that wants non-matching interpolation has to pass in an enum indicating this. Should not affect usage of normal interpolation