[WIP] add reduced hessian example

[santiagoropb]

Fixes # .

Summary/Motivation:

Implement a basic example that demonstrates how to compute the reduced hessian of a nonlinear equality constrained problem with pynumero. Extensions to general nonlinear optimization problems with bounds can be implemented in pynumero following a very similar approach where the KKT matrix is the augmented system of the barrier subproblem at the optimal point. @carldlaird @blnicho @michaelbynum please refer to A.35 in Here for the computation of the KKT matrix.

The calculation of the reduced hessian can be beneficial for different applications of pyomo models. These include advanced step NMPC (e.g. pyomo.dae @blnicho ) and estimation of covariance matrices in parameter estimation problems (e.g. Parmest @kaklise, KIPET @salvadorgarciamunoz). Note that now that pynumero is supported on windows the implementation of the reduced hessian can be done in pynumero without the need of compiling any additional packages

@carldlaird for PyNumero itself, efficient calculation of the reduced hessian can be very useful for designing new algorithms. An interesting example is presented in here. There information from the reduced hessian is used within the inertia correction.

Changes proposed in this PR:

Legal Acknowledgement

By contributing to this software project, I agree to the following terms and conditions for my contribution:

1. I agree my contributions are submitted under the BSD license.
2. I represent I am authorized to make the contributions and grant the license. If my employer has rights to intellectual property that includes these contributions, I represent that I have received permission to make contributions and grant the required license on behalf of that employer.

[codecov]

Codecov Report

Merging #1282 into master will increase coverage by 0.45%.
The diff coverage is 27.77%.

@@            Coverage Diff             @@
##           master    #1282      +/-   ##
==========================================
+ Coverage   70.61%   71.07%   +0.45%     
==========================================
  Files         479      535      +56     
  Lines       76333    82769    +6436     
==========================================
+ Hits        53906    58824    +4918     
- Misses      22427    23945    +1518

Impacted Files	Coverage Δ
pyomo/contrib/pynumero/examples/reducedHessian.py	0% <0%> (ø)
pyomo/contrib/pynumero/interfaces/utils.py	100% <100%> (ø)	⬆️
pyomo/contrib/pynumero/interfaces/pyomo_nlp.py	100% <100%> (ø)	⬆️
pyomo/contrib/pynumero/extensions/utils.py	66.66% <0%> (-4.17%)	⬇️
pyomo/core/base/indexed_component_slice.py	97.27% <0%> (-0.78%)	⬇️
pyomo/core/kernel/component_set.py	100% <0%> (ø)	⬆️
pyomo/core/kernel/component_map.py	100% <0%> (ø)	⬆️
...omo/core/kernel/piecewise_library/transforms_nd.py	100% <0%> (ø)	⬆️
pyomo/core/kernel/piecewise_library/transforms.py	100% <0%> (ø)	⬆️
pyomo/core/base/set.py	100% <0%> (ø)	⬆️
... and 113 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 59b3664...b493b5f. Read the comment docs.

[adowling2]

@santiagoropb I am looking at this reduced Hessian example to add covariance matrix estimation to parmest. You are welcome to add me as a reviewer to this PR, but I warn I'll ask for some additional comments in the example.

For inequality constraints or bounds, does one just need to assemble the active set and use the corresponding null space calculation? I quickly looked through section 3.2 of V. Zavala's thesis. That specifically discusses Ipopt and the barrier Hessian.

[santiagoropb]

@adowling2 actually we do not need the active set nor the null-space since that info is implicit in the KKT system. We only need the primal and dual variables at the solution point (e.g. solve with ipopt and use x* and y*), evaluate the KKT matrix at (x*,y*) and perform backsolves recursively (similarly as is done in the second method of this example). I will send you an email with the details.

Carl and Michael have been refactoring some of the interfaces in PyNumero. However, the matrix we want to perform backsolves on can be found in line 157 here

stale

[blnicho]

@santiagoropb this has been open as a WIP for a while and I think some of the changes here have been superseded by other changes to PyNumero. I'm going to close this and move the summary to an issue.