Skip to content

Commit

Permalink
ZOOKEEPER-2994. Added documentation and startup scripts
Browse files Browse the repository at this point in the history
  • Loading branch information
@anmolnar
anmolnar committed Apr 19, 2018
1 parent a69d729 commit 221760c
Show file tree
Hide file tree
Showing 5 changed files with 194 additions and 0 deletions.
24 changes: 24 additions & 0 deletions bin/zkTxnLogToolkit.cmd
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
@echo off
REM Licensed to the Apache Software Foundation (ASF) under one or more
REM contributor license agreements. See the NOTICE file distributed with
REM this work for additional information regarding copyright ownership.
REM The ASF licenses this file to You under the Apache License, Version 2.0
REM (the "License"); you may not use this file except in compliance with
REM the License. You may obtain a copy of the License at
REM
REM http://www.apache.org/licenses/LICENSE-2.0
REM
REM Unless required by applicable law or agreed to in writing, software
REM distributed under the License is distributed on an "AS IS" BASIS,
REM WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
REM See the License for the specific language governing permissions and
REM limitations under the License.

setlocal
call "%~dp0zkEnv.cmd"

set ZOOMAIN=org.apache.zookeeper.server.persistence.TxnLogToolkit
call %JAVA% -cp "%CLASSPATH%" %ZOOMAIN% %*

endlocal

38 changes: 38 additions & 0 deletions bin/zkTxnLogToolkit.sh
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
#!/usr/bin/env bash

# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

#
# If this scripted is run out of /usr/bin or some other system bin directory
# it should be linked to and not copied. Things like java jar files are found
# relative to the canonical path of this script.
#

# use POSIX interface, symlink is followed automatically
ZOOBIN="${BASH_SOURCE-$0}"
ZOOBIN="$(dirname "${ZOOBIN}")"
ZOOBINDIR="$(cd "${ZOOBIN}"; pwd)"

if [ -e "$ZOOBIN/../libexec/zkEnv.sh" ]; then
. "$ZOOBINDIR"/../libexec/zkEnv.sh
else
. "$ZOOBINDIR"/zkEnv.sh
fi

"$JAVA" -cp "$CLASSPATH" $JVMFLAGS \
org.apache.zookeeper.server.persistence.TxnLogToolkit "$@"


62 changes: 62 additions & 0 deletions docs/zookeeperAdmin.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -324,6 +324,9 @@ <h3>A Guide to Deployment and Administration</h3>
<li>
<a href="#sc_filemanagement">File Management</a>
</li>
<li>
<a href="#Recovery+-+TxnLogToolkit">Recovery - TxnLogToolkit</a>
</li>
</ul>
</li>
<li>
Expand Down Expand Up @@ -2513,6 +2516,65 @@ <h4>File Management</h4>

</div>
</div>
<a name="Recovery+-+TxnLogToolkit"></a>
<h4>Recovery - TxnLogToolkit</h4>
<p>TxnLogToolkit is a command line tool shipped with ZooKeeper which
is capable of recovering transaction log entries with broken CRC.</p>
<p>Running it without any command line parameters or with the "-h,--help"
argument, it outputs the following help page:</p>
<pre class="code">
$ bin/zkTxnLogToolkit.sh

usage: TxnLogToolkit [-dhrv] txn_log_file_name
-d,--dump Dump mode. Dump all entries of the log file. (this is the default)
-h,--help Print help message
-r,--recover Recovery mode. Re-calculate CRC for broken entries.
-v,--verbose Be verbose in recovery mode: print all entries, not just fixed ones.
-y,--yes Non-interactive mode: repair all CRC errors without asking
</pre>
<p>The default behaviour is safe: it dumps the entries of the given
transaction log file to the screen: (same as using '-d,--dump' parameter)</p>
<pre class="code">
$ bin/zkTxnLogToolkit.sh log.100000001
ZooKeeper Transactional Log File with dbid 0 txnlog format version 2
4/5/18 2:15:58 PM CEST session 0x16295bafcc40000 cxid 0x0 zxid 0x100000001 createSession 30000
<strong>CRC ERROR - 4/5/18 2:16:05 PM CEST session 0x16295bafcc40000 cxid 0x1 zxid 0x100000002 closeSession null</strong>
4/5/18 2:16:05 PM CEST session 0x16295bafcc40000 cxid 0x1 zxid 0x100000002 closeSession null
4/5/18 2:16:12 PM CEST session 0x26295bafcc90000 cxid 0x0 zxid 0x100000003 createSession 30000
4/5/18 2:17:34 PM CEST session 0x26295bafcc90000 cxid 0x0 zxid 0x200000001 closeSession null
4/5/18 2:17:34 PM CEST session 0x16295bd23720000 cxid 0x0 zxid 0x200000002 createSession 30000
4/5/18 2:18:02 PM CEST session 0x16295bd23720000 cxid 0x2 zxid 0x200000003 create '/andor,#626262,v{s{31,s{'world,'anyone}}},F,1
EOF reached after 6 txns.
</pre>
<p>There's a CRC error in the 2nd entry of the above transaction log file. In <strong>dump</strong>
mode, the toolkit only prints this information to the screen without touching the original file. In
<strong>recovery</strong> mode (-r,--recover flag) the original file still remains
untouched and all transactions will be copied over to a new txn log file with ".fixed" suffix. It recalculates
CRC values and copies the calculated value, if it doesn't match the original txn entry.
By default, the tool works interactively: it asks for confirmation whenever CRC error encountered.</p>
<pre class="code">
$ bin/zkTxnLogToolkit.sh -r log.100000001
ZooKeeper Transactional Log File with dbid 0 txnlog format version 2
CRC ERROR - 4/5/18 2:16:05 PM CEST session 0x16295bafcc40000 cxid 0x1 zxid 0x100000002 closeSession null
Would you like to fix it (Yes/No/Abort) ?
</pre>
<p>Answering <strong>Yes</strong> means the newly calculated CRC value will be outputted
to the new file. <strong>No</strong> means that the original CRC value will be copied over.
<strong>Abort</strong> will abort the entire operation and exits.
(In this case the ".fixed" will not be deleted and left in a half-complete state: contains only entries which
have already been processed or only the header if the operation was aborted at the first entry.)</p>
<pre class="code">
$ bin/zkTxnLogToolkit.sh -r log.100000001
ZooKeeper Transactional Log File with dbid 0 txnlog format version 2
CRC ERROR - 4/5/18 2:16:05 PM CEST session 0x16295bafcc40000 cxid 0x1 zxid 0x100000002 closeSession null
Would you like to fix it (Yes/No/Abort) ? y
EOF reached after 6 txns.
Recovery file log.100000001.fixed has been written with 1 fixed CRC error(s)
</pre>
<p>The default behaviour of recovery is to be silent: only entries with CRC error get printed to the screen.
One can turn on verbose mode with the -v,--verbose parameter to see all records.
Interactive mode can be turned off with the -y,--yes parameter. In this case all CRC errors will be fixed
in the new transaction file.</p>
<a name="sc_commonProblems"></a>
<h3 class="h4">Things to Avoid</h3>
<p>Here are some common problems you can avoid by configuring
Expand Down
Binary file modified docs/zookeeperAdmin.pdf
View file
Binary file not shown.
70 changes: 70 additions & 0 deletions src/docs/src/documentation/content/xdocs/zookeeperAdmin.xml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2159,6 +2159,76 @@ server.3=zoo3:2888:3888</programlisting>
individual settings in which it is being deployed. </para>
</note>
</section>

<section>
<title>Recovery - TxnLogToolkit</title>

<para>TxnLogToolkit is a command line tool shipped with ZooKeeper which
is capable of recovering transaction log entries with broken CRC.</para>
<para>Running it without any command line parameters or with the "-h,--help"
argument, it outputs the following help page:</para>

<programlisting>
$ bin/zkTxnLogToolkit.sh

usage: TxnLogToolkit [-dhrv] txn_log_file_name
-d,--dump Dump mode. Dump all entries of the log file. (this is the default)
-h,--help Print help message
-r,--recover Recovery mode. Re-calculate CRC for broken entries.
-v,--verbose Be verbose in recovery mode: print all entries, not just fixed ones.
-y,--yes Non-interactive mode: repair all CRC errors without asking
</programlisting>

<para>The default behaviour is safe: it dumps the entries of the given
transaction log file to the screen: (same as using '-d,--dump' parameter)</para>

<programlisting>
$ bin/zkTxnLogToolkit.sh log.100000001
ZooKeeper Transactional Log File with dbid 0 txnlog format version 2
4/5/18 2:15:58 PM CEST session 0x16295bafcc40000 cxid 0x0 zxid 0x100000001 createSession 30000
<emphasis role="bold">CRC ERROR - 4/5/18 2:16:05 PM CEST session 0x16295bafcc40000 cxid 0x1 zxid 0x100000002 closeSession null</emphasis>
4/5/18 2:16:05 PM CEST session 0x16295bafcc40000 cxid 0x1 zxid 0x100000002 closeSession null
4/5/18 2:16:12 PM CEST session 0x26295bafcc90000 cxid 0x0 zxid 0x100000003 createSession 30000
4/5/18 2:17:34 PM CEST session 0x26295bafcc90000 cxid 0x0 zxid 0x200000001 closeSession null
4/5/18 2:17:34 PM CEST session 0x16295bd23720000 cxid 0x0 zxid 0x200000002 createSession 30000
4/5/18 2:18:02 PM CEST session 0x16295bd23720000 cxid 0x2 zxid 0x200000003 create '/andor,#626262,v{s{31,s{'world,'anyone}}},F,1
EOF reached after 6 txns.
</programlisting>

<para>There's a CRC error in the 2nd entry of the above transaction log file. In <emphasis role="bold">dump</emphasis>
mode, the toolkit only prints this information to the screen without touching the original file. In
<emphasis role="bold">recovery</emphasis> mode (-r,--recover flag) the original file still remains
untouched and all transactions will be copied over to a new txn log file with ".fixed" suffix. It recalculates
CRC values and copies the calculated value, if it doesn't match the original txn entry.
By default, the tool works interactively: it asks for confirmation whenever CRC error encountered.</para>

<programlisting>
$ bin/zkTxnLogToolkit.sh -r log.100000001
ZooKeeper Transactional Log File with dbid 0 txnlog format version 2
CRC ERROR - 4/5/18 2:16:05 PM CEST session 0x16295bafcc40000 cxid 0x1 zxid 0x100000002 closeSession null
Would you like to fix it (Yes/No/Abort) ?
</programlisting>

<para>Answering <emphasis role="bold">Yes</emphasis> means the newly calculated CRC value will be outputted
to the new file. <emphasis role="bold">No</emphasis> means that the original CRC value will be copied over.
<emphasis role="bold">Abort</emphasis> will abort the entire operation and exits.
(In this case the ".fixed" will not be deleted and left in a half-complete state: contains only entries which
have already been processed or only the header if the operation was aborted at the first entry.)</para>

<programlisting>
$ bin/zkTxnLogToolkit.sh -r log.100000001
ZooKeeper Transactional Log File with dbid 0 txnlog format version 2
CRC ERROR - 4/5/18 2:16:05 PM CEST session 0x16295bafcc40000 cxid 0x1 zxid 0x100000002 closeSession null
Would you like to fix it (Yes/No/Abort) ? y
EOF reached after 6 txns.
Recovery file log.100000001.fixed has been written with 1 fixed CRC error(s)
</programlisting>

<para>The default behaviour of recovery is to be silent: only entries with CRC error get printed to the screen.
One can turn on verbose mode with the -v,--verbose parameter to see all records.
Interactive mode can be turned off with the -y,--yes parameter. In this case all CRC errors will be fixed
in the new transaction file.</para>
</section>
</section>

<section id="sc_commonProblems">
Expand Down

0 comments on commit 221760c

Please sign in to comment.