[CI] Failure in :x-pack:qa:rolling-upgrade:v7.14.0#twoThirdsUpgradedTest on master

[cbuescher]

Build scan:

https://gradle-enterprise.elastic.co/s/5wzpqiepvm2mw

Repro line:

No particular test failure

Reproduces locally?:

No

Applicable branches:

master

Failure history:

seen once today

Failure excerpt:

Log contains suspicious:

[2021-04-28T06:19:56,312][WARN ][o.e.t.TcpTransport       ] [v7.14.0-0] exception caught on transport layer [Netty4TcpChannel{localAddress=0.0.0.0/0.0.0.0:63412, remoteAddress=null, profile=default}], closing connection
»  io.netty.handler.codec.DecoderException: javax.net.ssl.SSLException: No PSK available. Unable to resume.
»  	at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:471) ~[netty-codec-4.1.49.Final.jar:4.1.49.Final]
»  	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276) ~[netty-codec-4.1.49.Final.jar:4.1.49.Final]
»  	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
»  	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
»  	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
»  	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
»  	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
»  	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
»  	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
»  	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
»  	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
»  	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
»  	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
»  	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) [netty-transport-4.1.49.Final.jar:4.1.49.Final]
»  	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) [netty-common-4.1.49.Final.jar:4.1.49.Final]
»  	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [netty-common-4.1.49.Final.jar:4.1.49.Final]
»  	at java.lang.Thread.run(Thread.java:834) [?:?]
»  Caused by: javax.net.ssl.SSLException: No PSK available. Unable to resume.
»  	at sun.security.ssl.Alert.createSSLException(Alert.java:129) ~[?:?]
»  	at sun.security.ssl.Alert.createSSLException(Alert.java:117) ~[?:?]
»  	at sun.security.ssl.TransportContext.fatal(TransportContext.java:308) ~[?:?]
»  	at sun.security.ssl.TransportContext.fatal(TransportContext.java:264) ~[?:?]
»  	at sun.security.ssl.TransportContext.fatal(TransportContext.java:255) ~[?:?]
»  	at sun.security.ssl.ServerHello$T13ServerHelloConsumer.consume(ServerHello.java:1224) ~[?:?]
»  	at sun.security.ssl.ServerHello$ServerHelloConsumer.onServerHello(ServerHello.java:984) ~[?:?]
»  	at sun.security.ssl.ServerHello$ServerHelloConsumer.consume(ServerHello.java:872) ~[?:?]
»  	at sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:392) ~[?:?]
»  	at sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:444) ~[?:?]
»  	at sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(SSLEngineImpl.java:1065) ~[?:?]
»  	at sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(SSLEngineImpl.java:1052) ~[?:?]
»  	at java.security.AccessController.doPrivileged(Native Method) ~[?:?]
»  	at sun.security.ssl.SSLEngineImpl$DelegatedTask.run(SSLEngineImpl.java:999) ~[?:?]
»  	at io.netty.handler.ssl.SslHandler.runAllDelegatedTasks(SslHandler.java:1542) ~[netty-handler-4.1.49.Final.jar:4.1.49.Final]
»  	at io.netty.handler.ssl.SslHandler.runDelegatedTasks(SslHandler.java:1556) ~[netty-handler-4.1.49.Final.jar:4.1.49.Final]
»  	at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1440) ~[netty-handler-4.1.49.Final.jar:4.1.49.Final]
»  	at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1267) ~[netty-handler-4.1.49.Final.jar:4.1.49.Final]
»  	at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1314) ~[netty-handler-4.1.49.Final.jar:4.1.49.Final]
»  	at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:501) ~[netty-codec-4.1.49.Final.jar:4.1.49.Final]
»  	at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:440) ~[netty-codec-4.1.49.Final.jar:4.1.49.Final]
»  	... 16 more

[elasticmachine]

Pinging @elastic/es-security (Team:Security)

[elasticmachine]

Pinging @elastic/es-delivery (Team:Delivery)

[iverase]

Another similar failure today: https://gradle-enterprise.elastic.co/s/uthp6fzynplk6. Logs are full of stack-traces like the one above. It does not reproduce locally.

[slobodanadamovic]

Last failure I found is from Jun 17th 2021: https://gradle-enterprise.elastic.co/s/jfechvhd7rhk6/console-log#L31398
Looking at the history, this test only failed on darwin CI builds which have been removed. Hence, I will close this issue.