Fix adjoint for streamwise periodic massflow + General handling of adjoints of additional solution variables

[TobiKattmann]

Dear reader,

Let me give a little bit background first:

When a massflow is prescribed for periodic flow (KIND_STREAMWISE_PERIODIC= MASSFLOW), then a pressure drop is iterated(/adapted in each iteration) to fulfill that massflow requirement. In the Volume Source term only this pressure drop is used. The initial value at solver start is the value given in STREAMWISE_PERIODIC_PRESSURE_DROP= ?? and for restarted/adjoint simulations the STREAMWISE_PERIODIC_PRESSURE_DROP from flow.meta is taken.

Now this PR:

Until now, the update of the pressure drop, based on the difference between current and prescribed massflow, was done in the Flow solver Preprocessing. That causes some trouble as that routine is called multiple times during normal starts, restarts and even twice(!) for every normal iteration. So do not put any iterated quantities in those like I did. I now did put the update right in front of where it is used -> in the Source_Residual. Like so there shoulnd't be that much trouble of correct taping in that one iteration for the adjoint and it makes the logic of Inner_iter/outer_iter==0 etc unnecessary.

I am not sure the changes up until now are necessary but I am not sure whether they are enough tbh. I will update this description accordingly:

Update: So to give you an idea of the current state of gradient validation. My setup is the good'ol 2D periodic Pin case (fluid-only) with the Design variables being FFD-parameters only around the middle pin (so no Periodic Interface shenanigans up to now) and I have 4 OF: Avg_Temp and Drag on the middle-Pin Surface as well as the Pressure-drop between In/outlet plus massflow through in- OR outlet (which is the same in value but maybe not for the gradient 🤔 ):

1. For prescribed pressure drop I get a very good agreement for Avg_Temp and Drag (below 0.1% relative diff across the board).
   
   For Massflow and pressure drop the gradients are off by quite a bit but not too shabby (a good optimizer certainly could make good use of those gradients). Of course the Pressure-drop gradient should be 0 but there are prob some numerical gains to be made. The Pressure-drop gradient is O(1e-5) where the others are O(1 - 1e4) so I consider that to be ok
2. For prescribed massflow (and I differentiate here between the code before this PR and ...well this PR): The gradient for Avg_Temp and drag are okay-ish but far from being validated (just like for massflow and dp for prescribed dp)
   
   For massflow OF I get wrongly reported gradients by the adjoints, both before and after this PR
   
   and pressure drop OF
   

Two other aspects.

1. the PRESSURE_DROP OF is computed in streamwise periodic flow using the Recovered_Pressure (as the "standard" Pressure is the same on both faces). There I need to make sure that the recovered values are computed as well after the iteration (but with the Flow Preprocessing after the MultiGridCycle that seems to be the case)
2. As pressure drop is iterated to get the correct flow solution, is it necessary to treat is specially in the adjoint like a solution variable? <- turns out, that was the case and also is the majority of work done in this PR

Le fin

Related Work

#773 initial implementation
#1527 first step in getting a consistent restart for massflow prescribed flow by storing the pressure drop in massflow-meta file

PR Checklist

• I am submitting my contribution to the develop branch.
• My contribution generates no new compiler warnings (try with the '-Wall -Wextra -Wno-unused-parameter -Wno-empty-body' compiler flags, or simply --warnlevel=2 when using meson).
• My contribution is commented and consistent with SU2 style.
• I have added a test case that demonstrates my contribution, if necessary.
• I have updated appropriate documentation (Tutorials, Docs Page, config_template.cpp) , if necessary.

[pcarruscag]

Grabs 🍿

The fixed-Cl mode is not active during adjoints, the AOA is just taken from the metadata.
If something is determined iteratively, then I think it is a solution variable, but I never validated this. I always made my life simpler by constraining Cl instead of using fixed Cl 😊
In theory mass flow outlets and your periodic mass flow fall in the same category of fixed-Cl.

[TobiKattmann]

The fixed-Cl mode is not active during adjoints, the AOA is just taken from the metadata.

I.e. you would propose to try to disable the delta-pressure udpdate during the adjoint for prescribed massflow? In my understanding that would result in an inconsistent gradient -> which might work good enough for optimizations but in a gradient validation I would expect significant deviations.

If something is determined iteratively, then I think it is a solution variable, [...]

So it is settled then -> I will try to give pressure-drop the same treatment as regular solution variables as mentioned in point 2. of the PR description.

[TobiKattmann]

In theory mass flow outlets and your periodic mass flow fall in the same category of fixed-Cl

I thought about that as well. If I get the streamwise periodic massflow to work that should be equivalent for massflow outlets. After all, quite a few ideas were taken from that implementation.

Maybe a question to @economon in that regard (I surely asked that already but cannot remember 🧠 🔨 ): Did you make a gradient validation for massflow-outlets back in the day? If yes, do you remember whether that went well, or did you see some imperfection that wasn't there for pressure-outlets?

[pcarruscag]

Yes it is inconsistent that's why it would need validation, the worst-case scenario is when your design variable is the thing the primal solver is manipulating to implement the fixed mode.
For example I'm pitching the wing (geometric AOA let's call it) the adjoint gives me a value for dCL / dalpha, but in practice this value is 0 because the primal solver does not allow the CL to change.
But doing the above would be dumb right, the validation question is what happens to the derivatives of other outputs? Is dCD / dalpha correct? I imagine this depends on the relation between CD and CL in this case.

[pcarruscag]

This can probably be tested with a dummy fixed point iteration...

[TobiKattmann]

With commit 1836f11 the gradients (for massflow prescribed in the primal) can now be validated really well (tested for AvgTemp and PRESSURE_DROP OF's).

BUT only for serial execution ... for multiple processes the adjoint diverges after a few iterations. I need to give that parallel treatment another thought.
UPDATE: With ce164a2 also the parallel execution works fine. Please read my explanation attempt in the commit message (in case you are interested) and let me know what you think

[TobiKattmann]

So now that the Fluid-only version is reporting correct gradients for prescribed massflow due to the introduced adjoint equation for pressure drop -> the next problem seems to arrive: As of now I did not manage to get validated gradients for a CHT setup. I wonder whether sth additional is necessary there. But as the changes were introduced in the CDiscAdjSolver and CDiscFluidIteration ,always with using proper conditional to assure that the respective code is only excecuted when really needed, I am not sure if the mistake is not somewhere else.

Why cant shit just work 😞

btw: for prescribed pressure drop I get nice validated gradients for avgTemp, Drag, massflow

[TobiKattmann]

Update/Solution: d42692f and 44bc57b add the Intermediate storage of the adjoint solution and the storage for the external(OF) contribution to the pressure drop adjoint which solves the issue described in this post 😊

So for CHT the situation is this:

For prescribed massflow I do get a good gradient agreement (DA vs FD). Below a picture with avgTemp and massflow OF

But if I switch to prescribed massflow and look at the avgTemp and pressureDrop OF then the gradient is again bad

Here an comparison of sensitivities (wrt to mdot and dp OF (top/bottom)) for massflow and pressure drop precsribed flows (red/orange). The expectation is, that for massflow prescription, the sensitivity wrt to the massflow OF would be zero ... but we see that the sensitivities are very similar to the pressure-drop prescribed flow, so the hypothesis now is, for multizone the algorithm for some reason does not incorporate the pressure-drop adjoint which works like a charm for singlezone

[TobiKattmann]

Now I am happy with the results of the code but the question now is: How to code this up nicely!
This concept of an additional iterated variable next to the solution variable is "new" to SU2.

1. Add the Adjoint pressure-drop variables always to the existing arrays (Solution, External, AD_InputIndex, etc.) which would make for ugly allocation routines and one would have to mess with nPoint as one would prob extend the containers by one element. So I consider this direction to be a bit too intrusive as this concept is not needed for most applications anyway.
2. Add an Extended part to all relevant copide methods and variables and keep that general for the different applications

[pcarruscag]

👍 Looks quite simple to me now, what do you think?

[TobiKattmann]

Looks quite simple to me now, what do you think?

I fully agree. The CVariable footprint is much smaller and no more nasty address handling.

(Adding another variable in another solver requires a bit more code though and a little understanding of what is going on than the "Address"-version, but on the other hand this explicit handling of each primal-solver creates a good separation 👍 )

[TobiKattmann]

Thanks for the commit tackling the hardcoded allocation. And of course thanks for all the other suggestions and comments, made the code a lot better 💐 very much appreciated

Once the reg tests pass I'll merge this in 👍