[Bug] Cryostat should not always assume Kubernetes ports numbered 9091 are JMX

[andrewazores]

Current Behavior

Since forever, Cryostat's Kubernetes API discovery mechanism has assumed that if it sees a port named jfr-jmx or numbered 9091 that this is a JMX port. More recent Cryostat versions try to proactively connect to discovered target applications to determine their JVM IDs.

In the case that 9091 is exposed for a different service and is NOT a JMX port, this results in Cryostat trying to open a JMX connection to a non-JMX port as soon as it sees such a target application. This connection attempt will fail, but it takes time for that to happen. If enough of these attempts happen simultaneously (ie there is a large number of non-JMX 9091 applications discovered), it can lead to thread starvation, which degrades Cryostat performance - possibly to the point that the container platform kills and restarts Cryostat.

The jfr-jmx/9091 pattern has been in Cryostat for so long that it is best not to change it now, at least not as the default setting. However, it is about time that we make this configurable. These values can be kept as defaults but they should be overrideable by environment variables, and these variables should also be configurable by Operator CRD fields.

Expected Behavior

No response

Steps To Reproduce

No response

Environment

No response

Anything else?

No response