-
Notifications
You must be signed in to change notification settings - Fork 0
/
README
240 lines (174 loc) · 11.2 KB
/
README
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
Caution: I am not the author of ZHT. I only use ZHT API for testing.
THe origin ZHT project: https://bitbucket.org/xiaobingo/iit.datasys.zht-mpi.git
The following README is witten by xiaobingo on bitbucket.
-------------------ABOUT ZHT-------------------
ZHT is a zero-hop distributed hash table(see also the paper, http://datasys.cs.iit.edu/projects/ZHT/ZHT-CRC-PID2666213-Final.pdf).
-------------------SOFTWARE REQUIREMENTS-------------------
Google protocol buffers c binding, VERSION 0.15 or later
http://code.google.com/p/protobuf-c/downloads/list
Google protocol buffers c++ binding, VERSION 2.4.1
http://code.google.com/p/protobuf/downloads/list
-------------------ABOUT ZHT API, SPECIAL ATTENTION-------------------
ZHT supports the API set such as insert/lookup/remove/append/compare_swap.
When you are using append, BE CAREFUL THAT your key and value CAN NOT contain the colon(:), which is used
internally to delimite key/value pairs.
-------------------COMPILE AND INSTALL-------------------
FIRSTLY install Google protocol buffers c++ binding,
AND THEN install Google protocol buffers c binding since the latter dependes on the former.
Assuming that you installed them to default directory, possibly that is, /usr/local
1. vim/gedit ~/.bashrc
2. append the following lines to the end,
export LD_LIBRARY_PATH=/usr/local/lib/:$LD_LIBRARY_PATH
3. source ~/.bashrc
4. cd to ZHT src directory
5. TO COMPILE ZHT, RUN
make
ORTHERWISE, assuming that you will install them to /home/xiaobingo/install, follow THE ORDER:
1. firstly install Google protocol buffers c++ binding, by running
1) cd to <Google protocol buffers c++ binding> home directory
2) ./configure --prefix=/home/xiaobingo/install
3) make
4) make install
2. then install Google protocol buffers c binding, in this step, you may get error like: ERROR: protobuf headers are required.,
follow the instructions,
1) cd to <Google protocol buffers c binding> home directory
2) ./configure --prefix=/home/xiaobingo/install CXXFLAGS=-I/home/xiaobingo/install/include LDFLAGS=-L/home/xiaobingo/install/lib
3) make
4) make install
If you installed Google protocol buffers c and c++ bindings to customized directory, e.g./home/xiaobingo/install, MAKE SURE TO EXPORT them,
1. vim/gedit ~/.bashrc
2. append the following lines to the end,
export USER_LIB=/home/xiaobingo/install/lib/
export USER_INCLUDE=/home/xiaobingo/install/include/
export LD_LIBRARY_PATH=/home/xiaobingo/install/lib/
3. source ~/.bashrc
4. cd to ZHT src directory
5. TO COMPILE ZHT, RUN
make
, otherwise, you probably fail to compile ZHT or locate dependencies required by ZHT runtime.
-------------------RUN THE BENCHMARK-------------------
To run benchmark:
1. ./zhtserver -z zht.conf -n neighbor.conf, OR
./zhtserver -z zht.conf -n neighbor.conf -p 40000, to specify your own port that overrides PORT in zht.conf, OR
./zhtserver -z zht.conf -n neighbor.conf -p 40000 -f novoht.db, to specify your own disk file to which data is persisted.
2. ./zht_ben -z zht.conf -n neighbor.conf -o 1000
NOTE that:
The port you specified to listen on at zhtserver(e.g. the PORT in zht.conf, or ./zhtserver -z zht.conf -n neighbor.conf -p 40000)
MUST BE CONSISTENT WITH the port of that machine in neighbor.conf, otherwise, the zhtclient CANNOT reach the zhtserver by that port.
In ./zhtserver -z zht.conf -n neighbor.conf -p 40000 or
./zhtserver -z zht.conf -n neighbor.conf -p 40000 -f novoht.db
zht.conf: is to configure PROTOCOL and PORT, and other parameters,
neighbor.conf: is to configure neighbored nodes(or ZHT servers) in ZHT overlay network,
40000: means zhtserver port to listen on, which overrides PORT in zht.conf, if missing, that PORT in zht.conf will be used.
novoht.db: is the disk file to which the data are persisted, if missing, the data reside in memory. Please specify different
files if you are going to run multiple zhtservers in the same node.
In ./zht_ben -z zht.conf -n neighbor.conf -o 1000
zht.conf is to configure PROTOCOL and PORT, and other parameters,
neighbor.conf is to configure neighbored nodes(or ZHT servers) in ZHT overlay network,
1000 means running 1000 sequential iterations of INSERT, LOOKUP, REMOVE and APPEND.
NOTE that:
The parameter ID and VALUE(e.g. PROTOCOL TCP) for both zht.conf and neighbor.conf is SPACE delimited.
The conf files support #-style comment-out.
The blank lines will be ignored as comment-outs while parsing the conf files.
-------------------RUN TEST CASES FOR C++ BINDING-------------------
To run test cases for zht c++ binding:
1. cd to ZHT src directory
2. make
3. ./zhtserver -z zht.conf -n neighbor.conf
4. ./zht_cpptest -z zht.conf -n neighbor.conf
-------------------RUN TEST CASES FOR C BINDING-------------------
To run test cases for zht c binding:
1. cd to ZHT src directory
2. make
3. ./zhtserver -z zht.conf -n neighbor.conf
4. ./zht_ctest -z zht.conf -n neighbor.conf
-------------------ZHT STANDALONE DEPLOYMENT-------------------
To deploy ZHT on a single node and run your own ZHT application:
1. cd to ZHT src directory
2. ./zhtserver -z zht.conf -n neighbor.conf
3. ./your_zht_app -z zht.conf -n neighbor.conf
NOTE that your_zht_app is your binary that invokes ZHT client to INSERT/LOOKUP/REMOVE/APPEND key/value pairs.
-------------------ZHT CLUSTER DEPLOYMENT-------------------
To deploye ZHT on a cluster of nodes and run your own ZHT application:
1. cd to ZHT src directory
2. copy zht binary(e.g. zhtserver) to every node, or using shared NFS directory.
3. edit neighbor.conf to contain IP and PORT pair(e.g. 192.168.1.100 50000) of all ZHT servers you are going to start, DO NOT USE localhost.
4. copy zht.conf and neighbor.conf to every node, or using shared NFS directory.
5. In every node, run ./zhtserver -z zht.conf -n neighbor.conf, assuming <PORT 50000> in zht.conf is used.
6. run your ZHT applications.
-------------------DEVELOPE YOUR ZHT APPLICATIONS-------------------
To develope your own ZHT applications, please refer to <c_zhtclient_test.c>, which extensively tests ZHT client c binding, internally invoking c++ binding.
OR see also <cpp_zhtclient_test.cpp> on how to use ZHT API in c++ programs.
To compile your own ZHT applications, link to <libzht.a> and include cpp_zhtclient.h and c_zhtclient.h for C++ and C programs, respectively.
-------------------HOW TO COMPILE ZHT TO USE MPI PROTOCOL-------------------
1. INSTALL MPI, e.g. MPICH or OpenMPI, assuming that the mpi is installed to /users/xiaobingo/install
2. vim/gedit ~/.bashrc
3. append the following lines to the end,
export PATH=/users/xiaobingo/install/bin:$PATH
4. source ~/.bashrc
5. cd to ZHT src directory
6. TO COMPILE ZHT, RUN
make
-------------------ZHT(USING MPI) STANDALONE DEPLOYMENT-------------------
1. TO START zht-mpiserver and zht-mpibroker(e.g. ON SINGLE NODE), RUN:
mpiexec -np 4 ./zht-mpiserver -z zht.conf -n neighbor.mpi.conf : ./zht-mpibroker -z zht.conf -n neighbor.mpi.conf
3. TO RUN THE BENCHMARK:
see also <RUN THE BENCHMARK> section.
4. TO RUN TEST CASES FOR C++ BINDING:
see also <RUN TEST CASES FOR C++ BINDING> section
5. TO RUN TEST CASES FOR C BINDING:
see also <RUN TEST CASES FOR C BINDING> section
NOTE THAT:
mpiexec: mpi command to launch mpi jobs.
-np 4 ./zht-mpiserver: allocate 4 processors to run the zht-mpiserver processes.
-z zht.conf: zht conf file as input argument to zht-mpiserver or zht-mpibroker, the same file mentioned above.
-n neighbor.mpi.conf: negibor conf file as input argument to zht-mpiserver or zht-mpibroker, containing localhost in this case.
zht-mpibroker: the process deployed on each node to talk to zht-mpiserver on behalf of zhtclient.
PAY SPECIAL ATTENTIONS:
There is ONLY ONE INSTANCE of zht-mpibroker allowed to be deployed ON EVERY NODE.
zht-mpibroker MUST appear after zht-mpiserver IN ORDER in order to ensure the processes of zht-mpiserver
are assigned to proper mpi-ranks that serve processing incoming requests from zhtclients.
-------------------ZHT(USING MPI) CLUSTER DEPLOYMENT-------------------
1. TO START zht-mpiserver and zht-mpibroker(e.g. ON MANY NODEs), RUN:
mpiexec -f neighbor.mpi.conf -np <#processors> ./zht-mpiserver -z zht.conf -n neighbor.mpi.conf : ./zht-mpibroker -z zht.conf -n neighbor.mpi.conf
OR
mpiexec -hosts 192.168.1.100,192.168.1.101 -np <#processors> ./zht-mpiserver -z zht.conf -n neighbor.mpi.conf : ./zht-mpibroker -z zht.conf -n neighbor.mpi.conf
2. TO RUN THE BENCHMARK:
see also <RUN THE BENCHMARK> section.
3. TO RUN TEST CASES FOR C++ BINDING:
see also <RUN TEST CASES FOR C++ BINDING> section
4. TO RUN TEST CASES FOR C BINDING:
see also <RUN TEST CASES FOR C BINDING> section
NOTE THAT:
mpiexec: mpi command to launch mpi jobs.
-f neighbor.mpi.conf: negibor conf file containing nodes on which MPI processors run, in which each line is ip address(e.g. 192.168.1.100), DO NOT USE localhost.
-hosts 192.168.1.100,192.168.1.101: two nodes(comma separated ip addresses) on which MPI processors run.
-np <#processors> ./zht-mpiserver: allocate <#processors> processors to run the zht-mpiserver processes.
-z zht.conf: zht conf file as input argument to zht-mpiserver or zht-mpibroker, the same file mentioned above.
-n neighbor.mpi.conf: negibor conf file as input argument to zht-mpiserver or zht-mpibroker, in which each line is ip address(e.g. 192.168.1.100), DO NOT USE localhost.
zht-mpibroker: the process deployed on each node to talk to zht-mpiserver on behalf of zhtclient.
PAY SPECIAL ATTENTIONS:
There is ONLY ONE INSTANCE of zht-mpibroker allowed to be deployed ON EVERY NODE.
zht-mpibroker MUST appear after zht-mpiserver IN ORDER in order to ensure the processes of zht-mpiserver
are assigned to proper mpi-ranks which serve processing incoming requests from zhtclients.
-------------------HOW TO CONFIGURE-------------------
There are two conf files needed, one is zht.conf, the other is neighbor.conf or neighbor.mpi.conf.
zht.conf:
PROTOCOL and PORT are the options to configure what protocols and ports(if applicable)
over which zhtclient and zhtserver talk to each other.
If you specified your own port by -p option(e.g. ./zhtserver -z zht.conf -n neighbor.conf -p 40000), this will override PORT defined in
zht.conf.
If you specified the disk file by -f option(e.g. -f novoht.db), the data will be persisted to that file, otherwise, they reside in memory.
Please specify different files if you are going to run multiple zhtservers in the same node.
neighbor.conf:
This file is used for TCP/UDP protocol.
The IP and PORT pairs of neighbored nodes(or ZHT servers) in ZHT overlay network are configured, e.g. 192.168.1.100 50000, 192.168.1.100 50001
The port you defined to listen on at neighbored node(or ZHT server)(e.g. the PORT in zht.conf, or ./zhtserver -z zht.conf -n neighbor.conf -p 40000)
MUST BE CONSISTENT WITH the port of that machine in neighbor.conf, otherwise, the zhtclient CANNOT reach the zhtserver by that port.
neighbor.mpi.conf:
This file is used for MPI protocol.
The IP(s) of neighbored nodes(or ZHT servers) in ZHT overlay network are configured, e.g. 192.168.1.100, no port is necessary for MPI protocol.
NOTE that:
The parameter ID and VALUE(e.g. PROTOCOL TCP) for both zht.conf and neighbor.conf is SPACE delimited.
The conf files support #-style comment-out.
The blank lines will be ignored as comment-outs while parsing.