Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

TLS setup fails to a master reported by sentinel #1209

Closed
ae6rt opened this issue Jan 15, 2020 · 3 comments
Closed

TLS setup fails to a master reported by sentinel #1209

ae6rt opened this issue Jan 15, 2020 · 3 comments
Labels
type: bug A general bug
Milestone
5.2.2

Comments

@ae6rt
Copy link

ae6rt commented Jan 15, 2020 •
edited
Loading

Bug Report

Lettuce Core cannot establish TLS with a master reported by sentinel, even though Lettuce Core can establish TLS with sentinel itself.

Current Behavior

Test code throws this exception when setting up TLS to the master


io.lettuce.core.RedisConnectionException: Unable to connect to 192.168.99.1:6381

	at io.lettuce.core.RedisConnectionException.create(RedisConnectionException.java:78)
	at io.lettuce.core.RedisConnectionException.create(RedisConnectionException.java:56)
	at io.lettuce.core.AbstractRedisClient.getConnection(AbstractRedisClient.java:234)
	at io.lettuce.core.RedisClient.connect(RedisClient.java:207)
	at io.lettuce.core.RedisClient.connect(RedisClient.java:192)
	at com.xoom.inf.TestApp.testLettuceSentinel(TestApp.java:29)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:567)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:160)
	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
	at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:33)
	at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:230)
	at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:58)
Caused by: javax.net.ssl.SSLHandshakeException: Hostname or IP address is undefined.
	at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:131)
	at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:324)
	at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:267)
	at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:262)
	at java.base/sun.security.ssl.CertificateMessage$T12CertificateConsumer.checkServerCerts(CertificateMessage.java:641)
	at java.base/sun.security.ssl.CertificateMessage$T12CertificateConsumer.onCertificate(CertificateMessage.java:460)
	at java.base/sun.security.ssl.CertificateMessage$T12CertificateConsumer.consume(CertificateMessage.java:360)
	at java.base/sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:396)
	at java.base/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:444)
	at java.base/sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(SSLEngineImpl.java:1260)
	at java.base/sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(SSLEngineImpl.java:1247)
	at java.base/java.security.AccessController.doPrivileged(AccessController.java:691)
	at java.base/sun.security.ssl.SSLEngineImpl$DelegatedTask.run(SSLEngineImpl.java:1192)
	at io.netty.handler.ssl.SslHandler.runAllDelegatedTasks(SslHandler.java:1502)
	at io.netty.handler.ssl.SslHandler.runDelegatedTasks(SslHandler.java:1516)
	at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1400)
	at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1227)
	at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1274)
	at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:503)
	at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:442)
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:281)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1422)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:931)
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:700)
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:635)
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:552)
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:514)
	at io.netty.util.concurrent.SingleThreadEventExecutor$6.run(SingleThreadEventExecutor.java:1050)
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.base/java.lang.Thread.run(Thread.java:830)
Caused by: java.security.cert.CertificateException: Hostname or IP address is undefined.
	at java.base/sun.security.util.HostnameChecker.match(HostnameChecker.java:96)
	at java.base/sun.security.ssl.X509TrustManagerImpl.checkIdentity(X509TrustManagerImpl.java:452)
	at java.base/sun.security.ssl.X509TrustManagerImpl.checkIdentity(X509TrustManagerImpl.java:426)
	at java.base/sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:292)
	at java.base/sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:144)
	at java.base/sun.security.ssl.CertificateMessage$T12CertificateConsumer.checkServerCerts(CertificateMessage.java:619)
	... 32 more

Sentinel and master are running at IP:26380 and IP:6380, respectively. In front of sentinel is a Layer 4 proxy terminating TLS at IP:26381, which forwards TCP to port 26380. In front of the master is another Layer 4 proxy terminating TLS at IP:6381, which forwards TCP to port 6380.

Lettuce Core succeeds in establishing TLS with the sentinel. From sentinel, it learns the master IP:6380, then with a MappingSocketAddressResolver, maps the master to IP:6381. TLS setup at the master proxy IP:6381 fails per the stack trace.

Input Code

public class TestApp {
    @Test
    public void testLettuceSentinel() {
        MappingSocketAddressResolver resolver = MappingSocketAddressResolver.create(DnsResolvers.UNRESOLVED,
                                                                                    hostAndPort -> {
                                                                                        System.out.println("mapping: " + hostAndPort.toString());
                                                                                        int port = hostAndPort.getPort();
                                                                                        if (hostAndPort.getPort() == 6380) {
                                                                                            port = port + 1;
                                                                                        }
                                                                                        return HostAndPort.of(hostAndPort.getHostText(), port);
                                                                                    });

        ClientResources clientResources = ClientResources.builder().socketAddressResolver(resolver).build();
        RedisClient redisClient = RedisClient.create(clientResources, "rediss-sentinel://192.168.99.1:26381/0#mymaster");
        final StatefulRedisConnection<String, String> connection = redisClient.connect();
        System.out.println("connected");
        connection.close();
        redisClient.shutdown();
    }

Expected behavior/code

Master TLS setup should succeed as Sentinel TLS setup succeeds.

Environment

  • Lettuce version(s): 5.2.1.RELEASE
  • Redis version: Redis server v=5.0.3 sha=00000000:0 malloc=jemalloc-5.1.0 bits=64 build=5d899ba8b1c64f5c
  • JDK: JDK13

Possible Solution

Unknown

Additional context

N/A
@ae6rt ae6rt added the type: bug A general bug label Jan 15, 2020
@ae6rt ae6rt mentioned this issue Jan 15, 2020
Closed
@mp911de mp911de added this to the 5.2.2 milestone Jan 15, 2020
mp911de added a commit that referenced this issue Jan 15, 2020
@mp911de
Use hostname and port from SocketAddress for peer verification #1209
7fb2af5 
Lettuce now uses the InetSocketAddress.hostString to verify the SSL host and for SNI instead of using the RedisURI. When using Redis Sentinel, the URI host was null and the port was zero which caused failures during the SSL handshake.
mp911de added a commit that referenced this issue Jan 15, 2020
@mp911de
Polishing #1209
5787811 
Simplify SSL tests. Refactor SSLEngine initialization into its own method.
mp911de added a commit that referenced this issue Jan 15, 2020
@mp911de
Polishing #1209
12b6894 
Fix cluster slots setup.
mp911de added a commit that referenced this issue Jan 15, 2020
@mp911de
Use hostname and port from SocketAddress for peer verification #1209
e4e4303 
Lettuce now uses the InetSocketAddress.hostString to verify the SSL host and for SNI instead of using the RedisURI. When using Redis Sentinel, the URI host was null and the port was zero which caused failures during the SSL handshake.
mp911de added a commit that referenced this issue Jan 15, 2020
@mp911de
Polishing #1209
120d57d 
Simplify SSL tests. Refactor SSLEngine initialization into its own method.
mp911de added a commit that referenced this issue Jan 15, 2020
@mp911de
Use hostname and port from SocketAddress for peer verification #1209
8a1a22e 
Lettuce now uses the InetSocketAddress.hostString to verify the SSL host and for SNI instead of using the RedisURI. When using Redis Sentinel, the URI host was null and the port was zero which caused failures during the SSL handshake.
mp911de added a commit that referenced this issue Jan 15, 2020
@mp911de
Polishing #1209
f02c353 
Simplify SSL tests. Refactor SSLEngine initialization into its own method.
mp911de added a commit that referenced this issue Jan 15, 2020
@mp911de
Polishing #1209
7ddef9c 
Fix cluster slots setup.
@mp911de
Copy link
Collaborator

mp911de commented Jan 15, 2020

That's fixed and available now in the most recent snapshot build for 5.2.2.BUILD-SNAPSHOT.

@mp911de mp911de closed this as completed Jan 15, 2020
@ae6rt
Copy link
Author

ae6rt commented Jan 15, 2020

Thank you for the fast turnaround. Amazing.

@ae6rt
Copy link
Author

ae6rt commented Jan 15, 2020

Looks good

$ mvn -q clean test

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running com.xoom.inf.TestApp
Jan 15, 2020 2:35:12 PM io.lettuce.core.EpollProvider <clinit>
INFO: Starting without optional epoll library
Jan 15, 2020 2:35:12 PM io.lettuce.core.KqueueProvider <clinit>
INFO: Starting without optional kqueue library
URI: rediss://192.168.99.1:26381
mapping: 192.168.99.1:26381
URI: redis://192.168.99.1:6380
mapping: 192.168.99.1:6380
connected
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.999 sec

Results :

Tests run: 1, Failures: 0, Errors: 0, Skipped: 0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
type: bug A general bug
Projects
None yet
Milestone
5.2.2
Development

No branches or pull requests
2 participants
@ae6rt @mp911de