Mismatched send receives in parallel lazy

[anderson2981]

It's possible to generate non-working parallel lazy code within the coupling. The issue appears to be related to not using data on one side of the coupled domain that is exchanged with the other side. For example, if I need the temperature_gradient on the fluid side, then I need to exchange temperature on the wall side. If the function doesn't actually use the wall-side temperature, then it's not available for exchange and I get mis-matched send/receives.

A function that gets around this issue can be found here:

https://github.com/illinois-ceesd/drivers_y3-prediction/blob/8e9fb061bbdb21354933df31d526a0014cc5fbb8/y3prediction/prediction.py#L2321

This function grabs the necessary data, in this case the gradients, and returns them to the caller for viz. It also uses a call to the coupled Navier-Stokes operator and does a dummy update of the conserved variables cv, wv. These are returned, but never used outside this compiled function.

[inducer]

That's indeed a difference between sends and receives: Receives disappear if unused, but sends do not. It would of course be best to not communicate data that isn't used, but if that's not practical, we can probably think of a robust way to address this. I've filed inducer/pytato#441 in pytato to discuss this.

cc @majosm

[anderson2981]

I'm not sure this quite addresses the current situation. From my perspective, I need the data on the fluid side and setup a data exchange (send/receive) to get it. So I need both the send and the receive. The issue appears to be that if data is not used on the side that needs to do the sending, then it doesn't exist in the DAG and can't be be sent at all.