HBASE-28110 Align TestShadeSaslAuthenticationProvider between differe… #5434
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
NihalJain
commented
Sep 24, 2023
| rootCause.printStackTrace(new PrintWriter(writer)); | ||
| String text = writer.toString(); | ||
| assertTrue("Message did not contain expected text", | ||
| text.contains("Connection reset by peer")); |
There was a problem hiding this comment.
This assertion is different between master and branch-2.
For branch-2 we get:
2023-09-25T00:45:03,375 INFO [Listener at localhost/61441 {}] example.TestShadeSaslAuthenticationProvider(298): Root cause was
java.io.IOException: Connection reset by peer
at sun.nio.ch.FileDispatcherImpl.read0(Native Method) ~[?:JAVA_8_VERSION]
at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) ~[?:JAVA_8_VERSION]
at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223) ~[?:JAVA_8_VERSION]
at sun.nio.ch.IOUtil.read(IOUtil.java:192) ~[?:JAVA_8_VERSION]
at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:379) ~[?:JAVA_8_VERSION]
at org.apache.hbase.thirdparty.io.netty.buffer.PooledByteBuf.setBytes(PooledByteBuf.java:259) ~[hbase-shaded-netty-4.1.4.jar:?]
at org.apache.hbase.thirdparty.io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:1132) ~[hbase-shaded-netty-4.1.4.jar:?]
at org.apache.hbase.thirdparty.io.netty.channel.socket.nio.NioSocketChannel.doReadBytes(NioSocketChannel.java:357) ~[hbase-shaded-netty-4.1.4.jar:?]
at org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:151) ~[hbase-shaded-netty-4.1.4.jar:?]
at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) ~[hbase-shaded-netty-4.1.4.jar:?]
at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) ~[hbase-shaded-netty-4.1.4.jar:?]
at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) ~[hbase-shaded-netty-4.1.4.jar:?]
at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) ~[hbase-shaded-netty-4.1.4.jar:?]
at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.4.jar:?]
at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.4.jar:?]
at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.4.jar:?]
at java.lang.Thread.run(Thread.java:748) ~[?:JAVA_8_VERSION]
For master we get:
2023-09-24T23:45:02,333 INFO [Listener at localhost/50641 {}] example.TestShadeSaslAuthenticationProvider(306): Root cause was
org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed contacting masters after 1 attempts.
Exceptions:
java.io.IOException: Call to address=IP_ADDRESS:PORT failed on local exception: java.io.IOException: org.apache.hbase.thirdparty.io.netty.handler.codec.DecoderException: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): Authentication failed for user1
at org.apache.hadoop.hbase.client.AbstractRpcBasedConnectionRegistry.lambda$groupCall$1(AbstractRpcBasedConnectionRegistry.java:197) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT]
at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:71) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT]
at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) ~[?:?]
at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) ~[?:?]
at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510) ~[?:?]
at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2162) ~[?:?]
at org.apache.hadoop.hbase.client.AbstractRpcBasedConnectionRegistry.lambda$call$0(AbstractRpcBasedConnectionRegistry.java:153) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT]
at org.apache.hbase.thirdparty.com.google.protobuf.RpcUtil$1.run(RpcUtil.java:79) ~[hbase-shaded-protobuf-4.1.4.jar:4.1.4]
at org.apache.hbase.thirdparty.com.google.protobuf.RpcUtil$1.run(RpcUtil.java:70) ~[hbase-shaded-protobuf-4.1.4.jar:4.1.4]
at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:396) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT]
at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT]
at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT]
at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT]
at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT]
at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT]
at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT]
at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.4.jar:?]
at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.4.jar:?]
at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.4.jar:?]
at org.apache.hbase.thirdparty.io.netty.channel.ChannelInboundHandlerAdapter.userEventTriggered(ChannelInboundHandlerAdapter.java:117) ~[hbase-shaded-netty-4.1.4.jar:?]
at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.userEventTriggered(ByteToMessageDecoder.java:387) ~[hbase-shaded-netty-4.1.4.jar:?]
at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:400) ~[hbase-shaded-netty-4.1.4.jar:?]
at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.4.jar:?]
at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.4.jar:?]
at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.4.jar:?]
at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.4.jar:?]
at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.4.jar:?]
at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.4.jar:?]
at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT]
at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT]
at org.apache.hadoop.hbase.ipc.NettyRpcConnection$1.operationComplete(NettyRpcConnection.java:271) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT]
at org.apache.hadoop.hbase.util.NettyFutureUtils.lambda$addListener$0(NettyFutureUtils.java:56) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT]
at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.4.jar:?]
at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) ~[hbase-shaded-netty-4.1.4.jar:?]
at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.4.jar:?]
at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.4.jar:?]
at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.4.jar:?]
at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.4.jar:?]
at org.apache.hadoop.hbase.security.NettyHBaseSaslRpcClientHandler.exceptionCaught(NettyHBaseSaslRpcClientHandler.java:196) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT]
at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:346) ~[hbase-shaded-netty-4.1.4.jar:?]
at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:325) ~[hbase-shaded-netty-4.1.4.jar:?]
at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:317) ~[hbase-shaded-netty-4.1.4.jar:?]
at org.apache.hbase.thirdparty.io.netty.channel.ChannelInboundHandlerAdapter.exceptionCaught(ChannelInboundHandlerAdapter.java:143) ~[hbase-shaded-netty-4.1.4.jar:?]
at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:346) ~[hbase-shaded-netty-4.1.4.jar:?]
at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:447) ~[hbase-shaded-netty-4.1.4.jar:?]
at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[hbase-shaded-netty-4.1.4.jar:?]
at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[hbase-shaded-netty-4.1.4.jar:?]
at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[hbase-shaded-netty-4.1.4.jar:?]
at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) ~[hbase-shaded-netty-4.1.4.jar:?]
at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[hbase-shaded-netty-4.1.4.jar:?]
at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[hbase-shaded-netty-4.1.4.jar:?]
at org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) ~[hbase-shaded-netty-4.1.4.jar:?]
at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) ~[hbase-shaded-netty-4.1.4.jar:?]
at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) ~[hbase-shaded-netty-4.1.4.jar:?]
at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) ~[hbase-shaded-netty-4.1.4.jar:?]
at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) ~[hbase-shaded-netty-4.1.4.jar:?]
at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.4.jar:?]
at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.4.jar:?]
at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.4.jar:?]
at java.lang.Thread.run(Thread.java:833) ~[?:?]
Upon digging found the same issue was found and fixed in HBASE-23881. Can backport as part of HBASE-24179. Or as another commit here. Let me know how we can proceed.
There was a problem hiding this comment.
You can ping @joshelser to see if he still have time to do the backport, if not I think we can do it by ourselves.
There was a problem hiding this comment.
I also have a local backport patch for the HBASE-23881 with which expected error is thrown. But not really sure if the fix is correct/complete. So let's wait and see if he can take up.
|
🎊 +1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
Apache9
reviewed
Sep 25, 2023
| public Void run() throws Exception { | ||
| try (Connection conn = ConnectionFactory.createConnection(clientConf);) { | ||
| conn.getAdmin().listTableDescriptors(); | ||
| fail("Should not successfully authenticate with HBase"); |
There was a problem hiding this comment.
Better use assertThrows here.
There was a problem hiding this comment.
Sure this patch just syncs code changes with master as is. Could we:
-
- Fix the test code improvements as part of another JIRA and merge to all branches, once code is in sync?
-
- Or we could fix here and forward port diff to master as well.
Please, let me know how you want me to proceed.
- Or we could fix here and forward port diff to master as well.
There was a problem hiding this comment.
ping @Apache9, let me know how to proceed
There was a problem hiding this comment.
I suggest we backport HBASE-23881 first?
There was a problem hiding this comment.
I suggest we backport HBASE-23881 first?
Sure I felt it would be better if these tests are in place for a easier test for backport. With backport we could fix the assert to be as expected. But yes let me do it other way around.
There was a problem hiding this comment.
OK. For assertThrows, I think we can file another issue to fix it. Just aligning the tests for all branches first is fine for me.
There was a problem hiding this comment.
Cool, let me create another JIRA to remove usage of assert throws. So that this JIRA can go in 1st and the new one fixing review comments across all branches next.
| @Override | ||
| public Void run() throws Exception { | ||
| try (Connection conn = ConnectionFactory.createConnection(clientConf); | ||
| Table t = conn.getTable(tableName)) { | ||
| t.get(new Get(Bytes.toBytes("r1"))); | ||
| return null; | ||
| fail("Should not successfully authenticate with HBase"); |
| } | ||
| })); | ||
| }); | ||
| } catch (InterruptedException e) { |
There was a problem hiding this comment.
Just throw these exceptions out? We are in a UT...
|
If @joshelser does not respond here, I think you can try to pick up the backport work for HBASE-23881 @NihalJain Thanks. |
Apache9
approved these changes
Oct 21, 2023
|
Plan (in order):
|
NihalJain
added a commit
that referenced
this pull request
Oct 25, 2023
…nt branches (#5434) Signed-off-by: Duo Zhang <[email protected]> (cherry picked from commit 6febf48)
NihalJain
added a commit
that referenced
this pull request
Oct 25, 2023
…nt branches (#5478) (#5434) Signed-off-by: Duo Zhang <[email protected]>
vinayakphegde
pushed a commit
to vinayakphegde/hbase
that referenced
this pull request
Apr 4, 2024
…nt branches (apache#5478) (apache#5434) Signed-off-by: Duo Zhang <[email protected]> (cherry picked from commit 0391eb1) Change-Id: I36aae07f432dec32efcfa088bd20a92f227aa7f9
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
…nt branches
MasterRegistrycase while sync'ingConnection reset by peerfromInvalidToken.class.getName()