Skip to content

Commit

Permalink
Merge pull request #185 from imzhenyu/imzhenyu
Browse files Browse the repository at this point in the history
documentation enhancement about scenario configurations
  • Loading branch information
imzhenyu authored Oct 12, 2016
2 parents 1b57d9d + 987d2c8 commit fdf058f
Show file tree
Hide file tree
Showing 12 changed files with 429 additions and 24 deletions.
21 changes: 17 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,10 @@
**Robust Distributed System Nucleus (rDSN)** is a framework for quickly building robust distributed systems. It has a microkernel for pluggable components, including applications, distributed frameworks, devops tools, and local runtime/resource providers, enabling their independent development and seamless integration. The project was originally developed for Microsoft Bing, and now has been adopted in production both inside and outside Microsoft.

* [What are the existing modules I can immediately use?](#existing)
* [What scenaios are enabled by combining these modules differently?](#scenarios)
* [How does rDSN build robustness?](#novel)
* [Related papers](#papers)


### Top Links
* [[Case](https://github.com/imzhenyu/rocksdb)] RocksDB made replicated using rDSN!
* [[Tutorial](https://github.com/Microsoft/rDSN/wiki/Tutorial:-one-box-cluster)] A one-box cluster demo to understand how rDSN helps service registration, deployment, monitoring etc..
Expand All @@ -20,8 +20,9 @@

The core of rDSN is a service kernel with which we can develop (via [Service API](http://imzhenyu.github.io/rDSN/documents/v1/html/group__service-api.html) and [Tool API](http://imzhenyu.github.io/rDSN/documents/v1/html/group__tool-api.html)) and plugin lots of different application, framework, tool, and local runtime modules, so that they can seamlessly benefit each other. Here is an incomplete list of the pluggable modules.

| Pluggable modules | Description | Demo |
| Pluggable modules | Description | Release |
|--------|-------------|------|
| [dsn.core](https://github.com/Microsoft/rDSN/tree/master/src/core) | rDSN service kernel | todo |
| [dsn.dist.service.stateless](https://github.com/imzhenyu/rDSN.dist.service/tree/master/src/app_daemon) | scale-out and fail-over for stateless services (e.g., micro services) | todo |
| [dsn.dist.service.stateful.type1](https://github.com/imzhenyu/rDSN.dist.service/tree/master/src/replica_server) | scale-out, replicate, and fail-over for stateful services (e.g., storage) | todo |
| [dsn.dist.service.meta_server](https://github.com/imzhenyu/rDSN.dist.service/tree/master/src/meta_server) | membership, load balance, and machine pool management for the above service frameworks | todo |
Expand All @@ -33,9 +34,21 @@ The core of rDSN is a service kernel with which we can develop (via [Service API
| [dsn.tools.hpc](https://github.com/imzhenyu/rDSN.tools.hpc) | high performance counterparts for the modules as implemented in tools.common | todo |
| [dsn.tools.explorer](https://github.com/imzhenyu/rDSN.tools.explorer) | extracts task-level dependencies automatically | todo |
| [dsn.tools.log.monitor](https://github.com/imzhenyu/rDSN.tools.log.monitor) | collect critical logs (e.g., log-level >= WARNING) in cluster | todo |
| [dsn.apps.skv](https://github.com/Microsoft/rDSN/tree/master/src/plugins/apps.skv) | an example application module | todo |
| [dsn.app.simple_kv](https://github.com/Microsoft/rDSN/tree/master/src/plugins/apps.skv) | an example application module | todo |

### <a name="scenarios"> Scenarios by different module combination and configuration </a>

rDSN provides flexible configuration so that developers can combine and configure the modules differently to enable different scenarios. All modules are loaded by [dsn.svchost](https://github.com/Microsoft/rDSN/tree/master/src/tools/svchost), a common process runner in rDSN, with the given configuration file. The following table lists some examples (note **dsn.core** is always required therefore omitted in ```Modules``` column).

| Scenarios | Modules | Config | Demo |
|----------|---------------|--------|------|
| logic correctness development | dsn.app.simple_kv + dsn.tools.emulator + dsn.tools.common | [config](https://github.com/Microsoft/rDSN/blob/master/tutorial/simple_kv/config.logic.ini) | todo |
| logic correctness with failure | dsn.app.simple_kv + dsn.tools.emulator + dsn.tools.common | [config](https://github.com/Microsoft/rDSN/blob/master/tutorial/simple_kv/config.logic.failure.ini) | todo |
| performance tuning | dsn.app.simple_kv + dsn.tools.common | [config](https://github.com/Microsoft/rDSN/blob/master/tutorial/simple_kv/config.logic.perf.ini) | todo |
| progressive performance tuning | dsn.app.simple_kv + dsn.tools.common + dsn.tools.emulator | [config](https://github.com/Microsoft/rDSN/blob/master/tutorial/simple_kv/config.logic.perf.prog.ini) | todo |
| Paxos enabled stateful service | dsn.app.simple_kv + dsn.tools.common + dsn.tools.emulator + dsn.dist.uri.resolver + dsn.dist.serivce.meta_server + dsn.dist.service.stateful.type1 | [config](https://github.com/Microsoft/rDSN/blob/master/tutorial/simple_kv/config.stateful.ini) | todo |

rDSN also provides a [web portal](https://github.com/Microsoft/rDSN/tree/master/src/tools/webstudio) that enables quick deployment of the above modules in a cluster, and allows easy operations through simple clicks as well as rich visualization.
There are a lot more possibilities. rDSN provides a [web portal](https://github.com/Microsoft/rDSN/tree/master/src/tools/webstudio) to enable quick deployment of these scenarios in a cluster, and allow easy operations through simple clicks as well as rich visualization. Deployment scenarios are defined [here](https://github.com/Microsoft/rDSN/blob/master/src/tools/webstudio/app_package/static/js/rdsn.envs.js), and developers can add more on demand.

### <a name="novel"> How does rDSN build robustness? </a>

Expand Down
2 changes: 1 addition & 1 deletion bin/config.common.ini
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ arguments = <meta-server1,meta-server2>
start_nfs = true

tool = nativerun
;tool = simulator
;tool = emulator
toollets = tracer
;toollets = tracer,profiler,fault_injector
pause_on_start = false
Expand Down
2 changes: 1 addition & 1 deletion bin/config.onecluster.ini.template
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ arguments = localhost:%meta_port%
start_nfs = true

tool = nativerun
;tool = simulator
;tool = emulator
toollets = tracer
;toollets = tracer,profiler,fault_injector
pause_on_start = false
Expand Down
16 changes: 12 additions & 4 deletions include/dsn/utility/extensible_object.h
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@ class extensible_object : public extensible
memset((void*)_extensions, 0, sizeof(_extensions));
}

~extensible_object()
DSN_API ~extensible_object()
{
int maxId = static_cast<int>(get_extension_count());

Expand All @@ -98,9 +98,17 @@ class extensible_object : public extensible
s_extensionDeletors[i]((void*)_extensions[i]);
}
}

// impossible branch to ensure the used apis are exported:-)
if (maxId > 0x0eadbeef)
{
extensible_object<T, MAX_EXTENSION_COUNT> r;
copy_to(r);
register_extension(nullptr);
}
}

void copy_to(extensible_object<T, MAX_EXTENSION_COUNT>& r)
DSN_API void copy_to(extensible_object<T, MAX_EXTENSION_COUNT>& r)
{
int maxId = static_cast<int>(get_extension_count());

Expand All @@ -113,7 +121,7 @@ class extensible_object : public extensible
}
}

static uint32_t register_extension(extension_deletor deletor = nullptr)
DSN_API static uint32_t register_extension(extension_deletor deletor = nullptr)
{
int idx = s_nextExtensionIndex++;
if (idx < MAX_EXTENSION_COUNT)
Expand All @@ -128,7 +136,7 @@ class extensible_object : public extensible
return idx;
}

static uint32_t get_extension_count()
DSN_API static uint32_t get_extension_count()
{
return s_nextExtensionIndex.load();
}
Expand Down
15 changes: 15 additions & 0 deletions src/core/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,3 +4,18 @@ This directory contains the source code for rDSN service microkernel.
* src - source code, including the unit tests
* dev.cpp.core.use - reference to the dev/cpp module

***dsn.core*** is the service kernel in rDSN. It defines [Service API](http://imzhenyu.github.io/rDSN/documents/v1/html/group__service-api.html) and [Tool API](http://imzhenyu.github.io/rDSN/documents/v1/html/group__tool-api.html), with which users can develop and plugin various modules, including distributed frameworks, development & operation tools, local runtime, and applications (see [examples](https://github.com/Microsoft/rDSN#existing-pluggable-modules-and-growing-) here). dsn.core takes charge of interconnecting these components, and makes sure they can benefit each other transparently (while developed independently).

### Build tools and local runtime libraries

[Tool API](http://imzhenyu.github.io/rDSN/documents/v1/html/group__tool-api.html) is mainly for this purpose, with wich developers can plugin, for example, new network providers (e.g,. a RDMA network provider with better performance, a [virtual network](https://github.com/Microsoft/rDSN/blob/master/src/plugins/tools.emulator/network.sim.h) for emulation), toollets for capturing how the requests are processed in the system (e.g., [tracer](https://github.com/Microsoft/rDSN/blob/master/src/plugins/tools.common/tracer.h)), tools for driving the execution of the whole distributed system (e.g., [emulator](https://github.com/Microsoft/rDSN/tree/master/src/plugins/tools.emulator)).

For all cases, developers use ```dsn::tools::register_component_provider```, ```dsn::tools::register_component_aspect```, ```dsn::tools::register_toollet```, ```dsn::tools::register_tool``` to plugin the modules into **dsn.core**.

### Build frameworks and applications

[Service API](http://imzhenyu.github.io/rDSN/documents/v1/html/group__service-api.html) provides the basic C APIs for building these components, and **dsn.dev.xxx** provides language wrappers atop (e.g., **dsn.dev.cpp**) to ease the development. The frameworks are considered advanced applications, and both are registered into **dsn.core** through ```dsn_register_app```. See [here](http://imzhenyu.github.io/rDSN/documents/v1/html/group__service-api-model.html) for more details.




3 changes: 1 addition & 2 deletions src/plugins/apps.skv/simple_kv.app.example.h
Original file line number Diff line number Diff line change
Expand Up @@ -145,8 +145,7 @@ class simple_kv_perf_test_client_app :
return ::dsn::ERR_INVALID_PARAMETERS;

// argv[1]: e.g., dsn://mycluster/simple-kv.instance0
rpc_address service_addr;
service_addr.assign_uri(dsn_uri_build(argv[1]));
url_host_address service_addr(argv[1]);

_simple_kv_client.reset(new simple_kv_perf_test_client(service_addr));
_simple_kv_client->start_test("simple_kv.simple_kv.perf-test.case", 3);
Expand Down
12 changes: 0 additions & 12 deletions src/tools/webstudio/README.md
Original file line number Diff line number Diff line change
@@ -1,17 +1,5 @@
**rDSN.WebStudio** is a web portal for service registration, deployment, testing, and monitoring, as well as cluster management, atop of the service frameworks and tool modules built with rDSN.

#### Serivce registration and cluster machine view
![registration](app_package/static/img/register.jpg)

#### Service deployment as a stateless service
![deploy](app_package/static/img/deploy.jpg)

#### Service monitoring with built-in profiler toollet
![monitor](app_package/static/img/monitor.jpg)

#### Service deployed as a replicated stateful service
![stateful](app_package/static/img/stateful.jpg)

## Installation

To start rDSN.WebStudio, you should install python 2.7.11+, and run the following commands.
Expand Down
61 changes: 61 additions & 0 deletions tutorial/simple_kv/config.logic.failure.ini
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
[config.args]
;; defined in app config file
;; service_type = <service-type>

service_type = simple_kv

[modules]
dsn.app.simple_kv
dsn.tools.common
dsn.tools.emulator

[apps.server]
type = %service_type%
ports = 54333
pools = THREAD_POOL_DEFAULT

[apps.client]
type = %service_type%.client
arguments = localhost:54333
pools = THREAD_POOL_DEFAULT

[core]
;tool = nativerun
tool = emulator
toollets = tracer,fault_injector
;toollets = tracer,profiler,fault_injector

[tools.emulator]
random_seed = 594750959

[task..default]
rpc_request_data_corrupted_ratio = 0.00
rpc_response_data_corrupted_ratio = 0.00

; data corrupted type: random/header/body
rpc_message_data_corrupted_type = random

rpc_request_drop_ratio = 0.001
rpc_response_drop_ratio = 0.001
rpc_request_delay_ratio = 0.001
rpc_response_delay_ratio = 0.001
disk_read_fail_ratio = 0.001
disk_write_fail_ratio = 0.001

rpc_message_delay_ms_min = 10
rpc_message_delay_ms_max = 10000
disk_io_delay_ms_min = 0
disk_io_delay_ms_max = 0
execution_extra_delay_us_max = 0

[task.RPC_SIMPLE_KV_SIMPLE_KV_READ]
rpc_response_data_corrupted_ratio = 0.01

[task.LPC_AIO_IMMEDIATE_CALLBACK]
is_trace = false
is_profile = false
allow_inline = false
disk_write_fail_ratio = 0.0

[task.LPC_RPC_TIMEOUT]
is_trace = false
35 changes: 35 additions & 0 deletions tutorial/simple_kv/config.logic.ini
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
[config.args]
;; defined in app config file
;; service_type = <service-type>

service_type = simple_kv

[modules]
dsn.app.simple_kv
dsn.tools.common
dsn.tools.emulator

[apps.server]
type = %service_type%
ports = 54333
pools = THREAD_POOL_DEFAULT

[apps.client]
type = %service_type%.client
arguments = localhost:54333
pools = THREAD_POOL_DEFAULT

[core]
;tool = nativerun
tool = emulator
toollets = tracer
;toollets = tracer,profiler,fault_injector

[task.LPC_AIO_IMMEDIATE_CALLBACK]
is_trace = false
is_profile = false
allow_inline = false
disk_write_fail_ratio = 0.0

[task.LPC_RPC_TIMEOUT]
is_trace = false
45 changes: 45 additions & 0 deletions tutorial/simple_kv/config.logic.perf.ini
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
[config.args]
;; defined in app config file
;; service_type = <service-type>

service_type = simple_kv

[modules]
dsn.app.simple_kv
dsn.tools.common

[apps.server]
type = %service_type%
ports = 54333
pools = THREAD_POOL_DEFAULT

[apps.client.perf]
type = %service_type%.client.perf
arguments = localhost:54333
pools = THREAD_POOL_DEFAULT

[simple_kv.simple_kv.perf-test.case.1]
perf_test_seconds = 20
perf_test_key_space_size = 100000
perf_test_concurrency = 10
perf_test_payload_bytes = 10
perf_test_timeouts_ms = 10000
perf_test_hybrid_request_ratio = 1,1,1,

[core]
tool = nativerun
;tool = emulator
;toollets = profiler
;toollets = tracer,profiler,fault_injector

[task.LPC_AIO_IMMEDIATE_CALLBACK]
is_trace = false
is_profile = false
allow_inline = false
disk_write_fail_ratio = 0.0

[task.LPC_RPC_TIMEOUT]
is_trace = false

[threadpool.THREAD_POOL_DEFAULT]
worker_count = 10
73 changes: 73 additions & 0 deletions tutorial/simple_kv/config.logic.perf.prog.ini
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@

;
; progressive system complexity for performance tuning:
;
; . single thread for both client and server, workload concurrency set to 1, use emulated network to avoid network cost (delay set to 0)
; . + concurrency
; . + native network
; . + multi-threaded server/client
;
; the similar approach can be applied for correctness debugging
;

[config.args]
;; defined in app config file
;; service_type = <service-type>

service_type = simple_kv

[modules]
dsn.app.simple_kv
dsn.tools.common
dsn.tools.emulator

[apps..default]

;network.client.RPC_CHANNEL_TCP = dsn::tools::asio_network_provider, 65536
network.client.RPC_CHANNEL_TCP = dsn::tools::sim_network_provider, 65536
network.client.RPC_CHANNEL_UDP = dsn::tools::sim_network_provider, 65536

;network.server.0.RPC_CHANNEL_TCP = dsn::tools::asio_network_provider, 65536
; 0 for any server port
network.server.0.RPC_CHANNEL_TCP = dsn::tools::sim_network_provider, 65536
network.server.0.RPC_CHANNEL_UDP = dsn::tools::sim_network_provider, 65536

[apps.server]
type = %service_type%
ports = 54333
pools = THREAD_POOL_DEFAULT

[apps.client.perf]
type = %service_type%.client.perf
arguments = localhost:54333
pools = THREAD_POOL_DEFAULT

[simple_kv.simple_kv.perf-test.case.1]
perf_test_seconds = 20
perf_test_key_space_size = 100000
perf_test_concurrency = 1
perf_test_payload_bytes = 10
perf_test_timeouts_ms = 10000
perf_test_hybrid_request_ratio = 1,1,1,

[core]
tool = nativerun
;tool = emulator
;toollets = profiler
;toollets = tracer,profiler,fault_injector

[tools.emulator]
min_message_delay_microseconds = 0
max_message_delay_microseconds = 0

[task.LPC_AIO_IMMEDIATE_CALLBACK]
is_trace = false
is_profile = false
allow_inline = false
disk_write_fail_ratio = 0.0

[task.LPC_RPC_TIMEOUT]
is_trace = false

[threadpool.THREAD_POOL_DEFAULT]
worker_count = 1
Loading

0 comments on commit fdf058f

Please sign in to comment.