HBASE-22547 Align the config keys and add document for offheap read in HBase Book. #301
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
🎊 +1 overall
This message was automatically generated. |
anoopsjohn
reviewed
Jun 13, 2019
anoopsjohn
reviewed
Jun 13, 2019
anoopsjohn
reviewed
Jun 13, 2019
anoopsjohn
reviewed
Jun 13, 2019
anoopsjohn
reviewed
Jun 13, 2019
anoopsjohn
reviewed
Jun 13, 2019
anoopsjohn
reviewed
Jun 13, 2019
anoopsjohn
reviewed
Jun 13, 2019
anoopsjohn
reviewed
Jun 13, 2019
anoopsjohn
reviewed
Jun 13, 2019
anoopsjohn
reviewed
Jun 13, 2019
anoopsjohn
reviewed
Jun 13, 2019
anoopsjohn
reviewed
Jun 13, 2019
anoopsjohn
reviewed
Jun 13, 2019
anoopsjohn
reviewed
Jun 13, 2019
|
🎊 +1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
anoopsjohn
reviewed
Jun 18, 2019
| @@ -114,7 +114,7 @@ | |||
| public static final int DEFAULT_BUFFER_SIZE = 65 * 1024; | |||
|
|
|||
| public static final String MIN_ALLOCATE_SIZE_KEY = | |||
| "hbase.ipc.server.reservoir.minimal.allocating.size"; | |||
| "hbase.server.reservoir.minimal.allocating.size"; | |||
There was a problem hiding this comment.
This is not part of this patch any more right?
There was a problem hiding this comment.
I think I forget remove the ipc of this config in https://issues.apache.org/jira/browse/HBASE-22598 before. So remove here.
anoopsjohn
reviewed
Jun 18, 2019
|
|
||
| Next thing to tune is the ByteBuffer pool on the RPC server side. | ||
| The buffers from this pool will be used to accumulate the cell bytes and create a result cell block to send back to the client side. | ||
| `hbase.ipc.server.reservoir.enabled` can be used to turn this pool ON or OFF. By default this pool is ON and available. HBase will create off heap ByteBuffers |
There was a problem hiding this comment.
Oh we had this config also using ipc. .. We could have deprecated this too the same way. Sorry missed in last comment.
There was a problem hiding this comment.
Em..seems we can refactor this config as hbase.server.allocator.enabled... the hbase.ipc.server.reservoir.enabled is some confusing ...
anoopsjohn
reviewed
Jun 18, 2019
| and pool them. Please make sure not to turn this OFF if you want end-to-end off-heaping in read path. | ||
| If this pool is turned off, the server will create temp buffers on heap to accumulate the cell bytes and make a result cell block. This can impact the GC on a highly read loaded server. | ||
| The user can tune this pool with respect to how many buffers are in the pool and what should be the size of each ByteBuffer. | ||
| Use the config `hbase.ipc.server.reservoir.initial.buffer.size` to tune each of the buffer sizes. Default is 64 KB for HBase2.x, while it will be changed to 65KB by default for HBase3.x |
There was a problem hiding this comment.
In all these new document, can we refer to new config at 1st place. Also say the old one and is deprecated in 3.0
openinx
force-pushed
the
HBASE-22547
branch
2 times, most recently
from
f6ae122 to
5e93acb
Compare
openinx
commented
Jun 18, 2019
|
|
||
| Next thing to tune is the ByteBuffer pool on the RPC server side: | ||
|
|
||
| NOTE: the config keys which starts with prefix hbase.ipc.server.reservoir are deprecated in HBase3.x. If you are still |
There was a problem hiding this comment.
In all these new document, can we refer to new config at 1st place. Also say the old one and is deprecated in 3.0
@anoopsjohn, I've addressed your comment here.
|
💔 -1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
|
Ping @anoopsjohn , any other concerns ? I think it's the last subtask of HBASE-21879 until now. Thanks. |
anoopsjohn
reviewed
Jun 20, 2019
| @@ -68,10 +68,18 @@ | |||
| // default heap allocator, it will just allocate ByteBuffers from heap but wrapped by an ByteBuff. | |||
| public static final ByteBuffAllocator HEAP = ByteBuffAllocator.createOnHeap(); | |||
|
|
|||
| public static final String ALLOCATOR_ENABLED_KEY = "hbase.server.allocator.enabled"; | |||
There was a problem hiding this comment.
We should call it some way which can clear say if this is enabled, it is pooled stuff what we use. Anyways allocator will always be there in RS. It might be HEAP one.
There was a problem hiding this comment.
Sorry for the confusing, my fault. let's define the 4 new config keys as:
1. hbase.server.allocator.enabled -> hbase.server.allocator.pool.enabled
2. hbase.server.reservoir.minimal.allocating.size -> hbase.server.allocator.minimal.allocate.size
3. hbase.server.allocator.max.buffer.count
4. hbase.server.allocator.buffer.size
Will change all plances.
openinx
changed the title
|
💔 -1 overall
This message was automatically generated. |
|
Updated the patch:
Also checked all the places. @anoopsjohn , PTAL. |
|
Update the patch:
@anoopsjohn FYI. |
anoopsjohn
approved these changes
Jun 21, 2019
|
💔 -1 overall
This message was automatically generated. |
openinx
added a commit
to openinx/hbase
that referenced
this pull request
Jun 25, 2019
infraio
pushed a commit
to infraio/hbase
that referenced
this pull request
Aug 17, 2020
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.