Repeatability of optimizations

[guigau17]

Hi everyone,

I use Ipopt with CasADi for geometric optimizations of upper limb exoskeletons.

Concerning the repeatability of optimizations, is it right to assume that I should obtain the same results if I run an optimization multiple times with the same parameters ? By parameters, I am referring to the state variables, optimization variables, initial conditions, lower/upper bounds, variables constraints, etc.

What I observe is the following:

• For optimization no.1, I obtain result A;
• For optimization no.2, I obtain result B;
• For all subsequent optimization (no.3 and above), I obtain result B.

I am trying to find an explanation to this and would like any feedback, either if it is known to anyone or stuff that I should double check that could influence the optimizations.

My setup is Matlab R2020b, Ipopt 3.12.3, CasADi v.3.5.5, MacOS Big Sur (11.6). Using the solver Mumps for resolution of a NLP problem.
Let me know if additional informations are needed. Feel free to point me towards references or material that could help me understand this situation.

Thanks !

[svigerske]

Usually you should have gotten the same result already from the first run on.
But it is difficult to say why that didn't happen.
You could try increasing the print level in Ipopt to a really high value and see whether things differ already in the very beginning. That might then be an indication that the interface to Ipopt is doing something different for the second run.

[guigau17]

Thank you for the confirmation that I should indeed get the same result. I'll continue to investigate my code in order to identify what could differ from the 1st run.

[3plus10i]

Not sure why the subsequent solutions are identical, but there is possible helpful information for you : some calculation programs generate different results when they are working in parallel mode. Because there may be uncertainty in the cooperation of threads.

[zmbc]

If you are using MKL, there are settings you can use to ensure reproducibility even when you are using a (fixed) >1 number of threads: https://www.intel.com/content/www/us/en/developer/articles/technical/introduction-to-the-conditional-numerical-reproducibility-cnr.html. Additionally, there is a 'strict' mode that only works for a smaller set of CPUs but allows the number of threads to vary.

[danieloliveira56]

To make sure the model inputs are the same, I suggest you unload your model to a gdx file using the execute_unload command, preferably right before calling Solve. Then you can use the GDX diff tool in GAMS Studio to check if any set or parameter is different between the gdx files from two runs with different results.

Also, you can obtain different results if the ordering of your parameters and sets are different. This won't be reflected in the GDX diff tool.

If the input is exactly the same, the results should be the same on the same machine.