Solr 9 SSL incompatibilities

[sarkaramrit2]

Solr 9.2.1 facing two issues:

• except solr home dir, read/write permissions are needed and hence fails

                    "solrClientTLS": {
                        "mountedTLSDir": {
                            "path": "/abv-client-tls",  # see `spec.customSolrKubeOptions.podOptions.volumes` below
                            "path": "/opt/solr-9.2.1/abc-client-tls",

• SNI 400s

2023-07-25 17:19:34.649 WARN  (qtp912747546-98) [] o.e.j.s.HttpChannel handleException /solr/admin/info/system org.eclipse.jetty.http.BadMessageException: 400: Invalid SNI
2023-07-25 17:19:34.656 WARN  (qtp912747546-35) [] o.e.j.s.HttpChannel handleException /solr/admin/info/system org.eclipse.jetty.http.BadMessageException: 400: Invalid SNI

[HoustonPutman]

The first one is SOLR-16934, and will be fixed in Solr 9.4.

The second one is an issue with how the Solr Operator calls the Solr API. Basically we don't always use a "correct" hostname, and instead use localhost. There is a new option in Solr 9.2.1 (SOLR-16735), -Dsolr.jetty.ssl.sniHostCheck, that acts in the same way that verifyHostName does. So we need to set both options according to what the user wants regarding verifyHostName. If they want to verifyHostName, then we need to make sure that the correct hostname is used when executing the liveness/readiness probes.

[HoustonPutman]

This has been fixed by #610