Distribution tests failing due to unknown setting [xpack.security.ssl.diagnose.trust]

[davidkyle]

process was found dead while waiting for ports files, node{:distribution:archives:integ-test-zip:integTest-0}

Caused by:

ERROR][o.e.b.ElasticsearchUncaughtExceptionHandler] [integTest-0] uncaught exception in thread [main] |  
-- | --
  | »  org.elasticsearch.bootstrap.StartupException: java.lang.IllegalArgumentException: unknown setting [xpack.security.ssl.diagnose.trust] please check that any required plugins are installed, or check the breaking changes documentation for removed settings |  
  | »  	at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:174) ~[elasticsearch-7.6.0-SNAPSHOT.jar:7.6.0-SNAPSHOT] |  
  | »  	at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:161) ~[elasticsearch-7.6.0-SNAPSHOT.jar:7.6.0-SNAPSHOT] |  
  | »  	at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) ~[elasticsearch-7.6.0-SNAPSHOT.jar:7.6.0-SNAPSHOT] |  
  | »  	at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:125) ~[elasticsearch-cli-7.6.0-SNAPSHOT.jar:7.6.0-SNAPSHOT] |  
  | »  	at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-cli-7.6.0-SNAPSHOT.jar:7.6.0-SNAPSHOT] |  
  | »  	at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:126) ~[elasticsearch-7.6.0-SNAPSHOT.jar:7.6.0-SNAPSHOT] |  
  | »  	at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92) ~[elasticsearch-7.6.0-SNAPSHOT.jar:7.6.0-SNAPSHOT] |  
  | »  Caused by: java.lang.IllegalArgumentException: unknown setting [xpack.security.ssl.diagnose.trust] please check that any required plugins are installed, or check the breaking changes documentation for removed settings |  
  | »  	at org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:530) ~[elasticsearch-7.6.0-SNAPSHOT.jar:7.6.0-SNAPSHOT] |  
  | »  	at org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:475) ~[elasticsearch-7.6.0-SNAPSHOT.jar:7.6.0-SNAPSHOT] |  
  | »  	at org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:446) ~[elasticsearch-7.6.0-SNAPSHOT.jar:7.6.0-SNAPSHOT] |  
  | »  	at org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:417) ~[elasticsearch-7.6.0-SNAPSHOT.jar:7.6.0-SNAPSHOT] |  
  | »  	at org.elasticsearch.common.settings.SettingsModule.<init>(SettingsModule.java:149) ~[elasticsearch-7.6.0-SNAPSHOT.jar:7.6.0-SNAPSHOT] |  
  | »  	at org.elasticsearch.node.Node.<init>(Node.java:352) ~[elasticsearch-7.6.0-SNAPSHOT.jar:7.6.0-SNAPSHOT] |  
  | »  	at org.elasticsearch.node.Node.<init>(Node.java:253) ~[elasticsearch-7.6.0-SNAPSHOT.jar:7.6.0-SNAPSHOT] |  
  | »  	at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:221) ~[elasticsearch-7.6.0-SNAPSHOT.jar:7.6.0-SNAPSHOT] |  
  | »  	at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:221) ~[elasticsearch-7.6.0-SNAPSHOT.jar:7.6.0-SNAPSHOT] |  
  | »  	at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:349) ~[elasticsearch-7.6.0-SNAPSHOT.jar:7.6.0-SNAPSHOT] |  
  | »  	at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:170) ~[elasticsearch-7.6.0-SNAPSHOT.jar:7.6.0-SNAPSHOT] |  
  | »  	... 6 more

https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+7.x+matrix-java-periodic/ES_BUILD_JAVA=openjdk13,ES_RUNTIME_JAVA=java8fips,nodes=general-purpose/397/console
https://gradle-enterprise.elastic.co/s/rghajdqd44lle

Related to #49900?

[elasticmachine]

Pinging @elastic/es-security (:Security/Security)

[jkakavas]

Related to #49900?

Yes..

 cluster.setting 'xpack.security.ssl.diagnose.trust', 'false'

is applied to all clusters and this fails on oss.

[jkakavas]

I think that @mark-vieira might have alluded to this in that comment with

Is this safe to do all the time, even when not running in FIPS mode?

but this wasn't obvious in that moment.

There are two things here:

1. We are running OSS distribution in a FIPS 140 JVM and we shouldn't as this is not supported. Is this easy to exclude @mark-vieira
2. If the above is not actionable, could we conditionally apply the setting based on distribution type in BuildPlugin ? From the looks of it we can't.

[mark-vieira]

We are running OSS distribution in a FIPS 140 JVM and we shouldn't as this is not supported. Is this easy to exclude @mark-vieira

Sounds like we should just force the use of the default distro when running FIPS tests. Does this sound reasonable?

[jkakavas]

. Does this sound reasonable?

Yes @mark-vieira 🙏