-
Notifications
You must be signed in to change notification settings - Fork 25
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: Add user guide for compaction offload #158
Changes from 3 commits
3638daa
ada4582
fd0c99a
b0bb97f
161709d
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -187,9 +187,34 @@ timeout = "5s" | |
server_addrs = ['http://{HoraeMetaAddr}:2379'] | ||
``` | ||
|
||
### Compaction Offload | ||
|
||
Compaction offload is also supported. To enable compaction offload, the corresponding compaction mode with node picker and endpoint should be configured. | ||
|
||
- `node_picker`: There are two types of node picker -- `Local` and `Remote`(WIP). | ||
- When the `Local` is setted, the local compaction task would be offloaded to the specific remote compaction server, which decided by `endpoint`. | ||
- `endpoint`: The endpoint, in the form `addr:port`, indicating the _grpc port_ of the remote compaction server. | ||
|
||
Here is an example for it: | ||
|
||
```toml | ||
[analytic.compaction_mode] | ||
compaction_mode = "Offload" | ||
node_picker = "Local" | ||
endpoint = "{RemoteCompactionServerAddr}:{RemoteCompactionServerGrpcPort}" | ||
``` | ||
|
||
A Compaction Server, responsible for executing the compaction task, is also needed. The configuration of Compaction Server is similar to the HoraeDB Server, except that the node type (default `HoraeDB`) should be set to `CompactionServer`: | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
This is required? In apache/horaedb#1573, you don't configure There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Right. It is not required. It is used to help horaemeta distinguish the horaedb node and compaction node, which is not supported now. |
||
|
||
```toml | ||
[cluster_deployment] | ||
mode = "WithMeta" | ||
node_type = "CompactionServer" | ||
``` | ||
|
||
### Complete Config of HoraeDB | ||
|
||
With all the parts of the configurations mentioned above, a runnable complete config for HoraeDB can be made. In order to make the HoraeDB cluster runnable, we can decide to adopt RocksDB-based WAL and local-disk-based Object Storage: | ||
With all the parts of the configurations mentioned above, a runnable complete config for HoraeDB can be made. In order to make the HoraeDB cluster runnable, we can decide to adopt RocksDB-based WAL and local-disk-based Object Storage without compaction offload: | ||
|
||
```toml | ||
[server] | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The remote node picker is not supported without horaemeta. Currently, we need to use local node picker and specify the
endpoint
.