Interactive optimization

[annulen]

When dealing with geometry optimization (no matter what particular engine is in use) I often find myself wanting to change some settings on the fly. If calculation is cheap it could be done by restarting from scratch with new settings, but when each point requires 30 minutes or more it becomes a huge waste. Breaking ongoing optimization and properly restarting it is always a pain.

But what if we could intervene into running optimization? Possible features include:

• return to the previous step of selected number;
• increase or decrease trust radius;
• freeze trust radius at specific value, or let it change freely again;
• force rebuild of Hessian and/or coordinates;
• "patch" existing Hessian, for example replace negative eigenvalue with zero or positive in minimum search;
• change number of used CPU cores (threads);
• pause optimization gracefully after current gradient calculation finishes so it frees up most of used memory and could be restarted later after some other task;
• change convergence criteria or constraints;
• change logging verbosity;
• dump current Hessian to file.

I can imagine at least a couple of ways how it could be implemented:

• add interactive command line interface to geometric-optimize, allowing interactive input of commands from user controlling terminal;
• listen on unix or tcp socket and allow client to attach, internally it could accept either raw commands like in a previous approach (so user can use anything as simple as nc to enter commands), or use some kind of wire protocol and rely on custom client application to handle user interaction.

[leeping]

That's a very interesting idea, but I think it's also a pretty significant project. We would also need logging in order to have the ability to reproduce the interactive job. If you could provide a minimal working example of having geomeTRIC listen on a socket while running and accept user input in real-time, we could get this started (though I can't guarantee how fast I could contribute things). It would also need to be secure, so geomeTRIC should generate a password that the user needs to provide in order to attach.

[annulen]

It would also need to be secure, so geomeTRIC should generate a password that the user needs to provide in order to attach.

This sounds weird to me. There are a lot of ways to restrict access via infrastructure outside of geomeTRIC's code. It would be quite risky to have control port exposed from geomeTRIC directly to the public Internet.

I imagined it like that. If we listen to Unix port, it's by definition limited to the local OS, and access can be controlled via access rights on the socket file. If it's TCP port it could be handy for a local network, but it should listen on LAN IP address, not a public one, so that connections from outside are not possible. Firewall should prevent accidental exposure of port outside of LAN (on Linux by default all ports are blocked). If you need to attach from outside of LAN, you can SSH into computer inside LAN and access TCP port from there, or use some kind of TCP proxy with access control.

Compare to gdbserver program. It does quite a similar thing (remotely controlled debugger), but it has no access control at all — because it's not needed.

[annulen]

Also, authorization with passwords immediately requires use of SSL to avoid sending passwords in clear text which can be easily intercepted with a sniffer like Wireshark. And use of SSL in geomeTRIC creates a whole new layer of complications.