diff --git a/README.md b/README.md index 9ceb3de145..1cd0365aee 100644 --- a/README.md +++ b/README.md @@ -1,4 +1,4 @@ -[github-release]: https://github.com/XiaoMi/pegasus/releases +[github-release]: https://github.com/apache/incubator-pegasus/releases [PacificA]: https://www.microsoft.com/en-us/research/publication/pacifica-replication-in-log-based-distributed-storage-systems/ [pegasus-rocksdb]: https://github.com/xiaomi/pegasus-rocksdb [facebook-rocksdb]: https://github.com/facebook/rocksdb @@ -7,31 +7,23 @@ ![pegasus-logo](docs/media-img/pegasus-logo.png) -[![Build Status](https://travis-ci.org/XiaoMi/pegasus.svg?branch=master)](https://travis-ci.org/XiaoMi/pegasus) +[![Build Status](https://travis-ci.org/apache/incubator-pegasus.svg?branch=master)](https://travis-ci.org/apache/incubator-pegasus) [![License](https://img.shields.io/badge/license-Apache%202-4EB1BA.svg)](https://www.apache.org/licenses/LICENSE-2.0.html) -[![Releases](https://img.shields.io/github/release/xiaomi/pegasus.svg)][github-release] +[![Releases](https://img.shields.io/github/release/apache/incubator-pegasus.svg)][github-release] **Note**: The `master` branch may be in an *unstable or even broken state* during development. Please use [releases][github-release] instead of the `master` branch in order to get stable binaries. -Pegasus is a distributed key-value storage system which is designed to be: +Apache Pegasus is a distributed key-value storage system which is designed to be: -- **horizontally scalable** distributed using hash-based partitioning +- **horizontally scalable**: distributed using hash-based partitioning - **strongly consistent**: ensured by [PacificA][PacificA] consensus protocol - **high-performance**: using [RocksDB][pegasus-rocksdb] as underlying storage engine - **simple**: well-defined, easy-to-use APIs -Pegasus has been widely-used in XiaoMi and serves millions of requests per second. -It is a mature, active project. We hope to build a diverse developer and user -community and attract contributions from more people. - ## Background -HBase was recognized as the only large-scale KV store solution in XiaoMi -until Pegasus came out in 2015 to solve the problem of high latency -of HBase because of its Java GC and RPC overhead of the underlying distributed filesystem. - -Pegasus targets to fill the gap between Redis and HBase. As the former +Pegasus targets to fill the gap between Redis and [HBase][hbase]. As the former is in-memory, low latency, but does not provide a strong-consistency guarantee. And unlike the latter, Pegasus is entirely written in C++ and its write-path relies merely on the local filesystem. @@ -40,43 +32,51 @@ Apart from the performance requirements, we also need a storage system to ensure multiple-level data safety and support fast data migration between data centers, automatic load balancing, and online partition split. -After investigating the existing storage systems in the open source world, -we could hardly find a suitable solution to satisfy all the requirements. -So the journey of Pegasus begins. +## Features + +- **Persistence of data**: Each write is replicated three-way to different ReplicaServers before responding to the client. Using PacificA protocol, Pegasus has the ability for strong consistent replication and membership changes. + +- **Automatic load balancing over ReplicaServers**: Load balancing is a builtin function of MetaServer, which manages the distribution of replicas. When the cluster is in an inbalance state, the administrator can invoke a simple rebalance command that automatically schedules the replica migration. + +- **Cold Backup**: Pegasus supports an extensible backup and restore mechanism to ensure data safety. The location of snapshot could be a distributed filesystem like HDFS or local filesystem. The snapshot storing in the filesystem can be further used for analysis based on [pegasus-spark](https://github.com/pegasus-kv/pegasus-spark). + +- **Eventually-consistent intra-datacenter replication**: This is a feature we called *duplication*. It allows a change made in the local cluster accesible after a short time period by the remote cluster. It help achieving higher availability of your service and gaining better performance by accessing only local cluster. ## To start using Pegasus See our documentation on [Pegasus Website][website]. +## Client drivers + +Pegasus has support for serveral languages: + +- [Java](https://github.com/xiaomi/pegasus-java-client) +- [C++](https://github.com/apache/incubator-pegasus/blob/master/src/include/pegasus/client.h) +- [Go](https://github.com/xiaomi/pegasus-go-client) +- [Python](https://github.com/xiaomi/pegasus-python-client) +- [Node.js](https://github.com/xiaomi/pegasus-nodejs-client) +- [Scala](https://github.com/xiaomi/pegasus-scala-client) + +## Contact us + +- Send mails to Apache Pegasus dev mailing list: dev@pegasus.apache.org. This is the place where topics around development, community, and problems are officially discussed. Please remember to subsribe the mail list via dev-subscribe@pegasus.apache.org. + +- Github Issues: submit an issue when you have any idea to improve Pegasus, and when you encountered some bugs or problems. + ## Related Projects Submodules: - [rDSN](https://github.com/xiaomi/rdsn) -- [RocksDB](https://github.com/xiaomi/pegasus-rocksdb) - -Client libs: - -- [Java client](https://github.com/xiaomi/pegasus-java-client) -- [Python Client](https://github.com/xiaomi/pegasus-python-client) -- [Go Client](https://github.com/xiaomi/pegasus-go-client) -- [Node.js Client](https://github.com/xiaomi/pegasus-nodejs-client) -- [Scala Client](https://github.com/xiaomi/pegasus-scala-client) Test tools: - [Java YCSB](https://github.com/xiaomi/pegasus-YCSB) -- [Go YCSB](https://github.com/xiaomi/pegasus-YCSB-go) Data import/export tools: - [DataX](https://github.com/xiaomi/pegasus-datax) -## Contact - -- Gitter: -- Issues: - ## License Copyright 2020 The Apache Software Foundation. Licensed under the Apache License, Version 2.0: