-
Notifications
You must be signed in to change notification settings - Fork 312
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
docs: update readme and remove xiaomi refs (#609)
- Loading branch information
Wu Tao
authored
Sep 25, 2020
1 parent
95ac849
commit 39d840f
Showing
1 changed file
with
32 additions
and
32 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,4 +1,4 @@ | ||
| [github-release]: https://github.com/XiaoMi/pegasus/releases | ||
| [github-release]: https://github.com/apache/incubator-pegasus/releases | ||
| [PacificA]: https://www.microsoft.com/en-us/research/publication/pacifica-replication-in-log-based-distributed-storage-systems/ | ||
| [pegasus-rocksdb]: https://github.com/xiaomi/pegasus-rocksdb | ||
| [facebook-rocksdb]: https://github.com/facebook/rocksdb | ||
|
|
@@ -7,31 +7,23 @@ | |
|
|
||
|  | ||
|
|
||
| [](https://travis-ci.org/XiaoMi/pegasus) | ||
| [](https://travis-ci.org/apache/incubator-pegasus) | ||
| [](https://www.apache.org/licenses/LICENSE-2.0.html) | ||
| [][github-release] | ||
| [][github-release] | ||
|
|
||
| **Note**: The `master` branch may be in an *unstable or even broken state* during development. | ||
| Please use [releases][github-release] instead of the `master` branch in order to get stable binaries. | ||
|
|
||
| Pegasus is a distributed key-value storage system which is designed to be: | ||
| Apache Pegasus is a distributed key-value storage system which is designed to be: | ||
|
|
||
| - **horizontally scalable** distributed using hash-based partitioning | ||
| - **horizontally scalable**: distributed using hash-based partitioning | ||
| - **strongly consistent**: ensured by [PacificA][PacificA] consensus protocol | ||
| - **high-performance**: using [RocksDB][pegasus-rocksdb] as underlying storage engine | ||
| - **simple**: well-defined, easy-to-use APIs | ||
|
|
||
| Pegasus has been widely-used in XiaoMi and serves millions of requests per second. | ||
| It is a mature, active project. We hope to build a diverse developer and user | ||
| community and attract contributions from more people. | ||
|
|
||
| ## Background | ||
|
|
||
| HBase was recognized as the only large-scale KV store solution in XiaoMi | ||
| until Pegasus came out in 2015 to solve the problem of high latency | ||
| of HBase because of its Java GC and RPC overhead of the underlying distributed filesystem. | ||
|
|
||
| Pegasus targets to fill the gap between Redis and HBase. As the former | ||
| Pegasus targets to fill the gap between Redis and [HBase][hbase]. As the former | ||
| is in-memory, low latency, but does not provide a strong-consistency guarantee. | ||
| And unlike the latter, Pegasus is entirely written in C++ and its write-path | ||
| relies merely on the local filesystem. | ||
|
|
@@ -40,43 +32,51 @@ Apart from the performance requirements, we also need a storage system | |
| to ensure multiple-level data safety and support fast data migration | ||
| between data centers, automatic load balancing, and online partition split. | ||
|
|
||
| After investigating the existing storage systems in the open source world, | ||
| we could hardly find a suitable solution to satisfy all the requirements. | ||
| So the journey of Pegasus begins. | ||
| ## Features | ||
|
|
||
| - **Persistence of data**: Each write is replicated three-way to different ReplicaServers before responding to the client. Using PacificA protocol, Pegasus has the ability for strong consistent replication and membership changes. | ||
|
|
||
| - **Automatic load balancing over ReplicaServers**: Load balancing is a builtin function of MetaServer, which manages the distribution of replicas. When the cluster is in an inbalance state, the administrator can invoke a simple rebalance command that automatically schedules the replica migration. | ||
|
|
||
| - **Cold Backup**: Pegasus supports an extensible backup and restore mechanism to ensure data safety. The location of snapshot could be a distributed filesystem like HDFS or local filesystem. The snapshot storing in the filesystem can be further used for analysis based on [pegasus-spark](https://github.com/pegasus-kv/pegasus-spark). | ||
|
|
||
| - **Eventually-consistent intra-datacenter replication**: This is a feature we called *duplication*. It allows a change made in the local cluster accesible after a short time period by the remote cluster. It help achieving higher availability of your service and gaining better performance by accessing only local cluster. | ||
|
|
||
| ## To start using Pegasus | ||
|
|
||
| See our documentation on [Pegasus Website][website]. | ||
|
|
||
| ## Client drivers | ||
|
|
||
| Pegasus has support for serveral languages: | ||
|
|
||
| - [Java](https://github.com/xiaomi/pegasus-java-client) | ||
| - [C++](https://github.com/apache/incubator-pegasus/blob/master/src/include/pegasus/client.h) | ||
| - [Go](https://github.com/xiaomi/pegasus-go-client) | ||
| - [Python](https://github.com/xiaomi/pegasus-python-client) | ||
| - [Node.js](https://github.com/xiaomi/pegasus-nodejs-client) | ||
| - [Scala](https://github.com/xiaomi/pegasus-scala-client) | ||
|
|
||
| ## Contact us | ||
|
|
||
| - Send mails to Apache Pegasus dev mailing list: [email protected]. This is the place where topics around development, community, and problems are officially discussed. Please remember to subsribe the mail list via [email protected]. | ||
|
|
||
| - Github Issues: submit an issue when you have any idea to improve Pegasus, and when you encountered some bugs or problems. | ||
|
|
||
| ## Related Projects | ||
|
|
||
| Submodules: | ||
|
|
||
| - [rDSN](https://github.com/xiaomi/rdsn) | ||
| - [RocksDB](https://github.com/xiaomi/pegasus-rocksdb) | ||
|
|
||
| Client libs: | ||
|
|
||
| - [Java client](https://github.com/xiaomi/pegasus-java-client) | ||
| - [Python Client](https://github.com/xiaomi/pegasus-python-client) | ||
| - [Go Client](https://github.com/xiaomi/pegasus-go-client) | ||
| - [Node.js Client](https://github.com/xiaomi/pegasus-nodejs-client) | ||
| - [Scala Client](https://github.com/xiaomi/pegasus-scala-client) | ||
|
|
||
| Test tools: | ||
|
|
||
| - [Java YCSB](https://github.com/xiaomi/pegasus-YCSB) | ||
| - [Go YCSB](https://github.com/xiaomi/pegasus-YCSB-go) | ||
|
|
||
| Data import/export tools: | ||
|
|
||
| - [DataX](https://github.com/xiaomi/pegasus-datax) | ||
|
|
||
| ## Contact | ||
|
|
||
| - Gitter: <https://gitter.im/XiaoMi/Pegasus> | ||
| - Issues: <https://github.com/XiaoMi/pegasus/issues> | ||
|
|
||
| ## License | ||
|
|
||
| Copyright 2020 The Apache Software Foundation. Licensed under the Apache License, Version 2.0: | ||
|
|
||