This document is to reproduce the Klaytn consensus message monitoring experiment environment. Let us describe briefly our experimental setup for the simple controlled experiment involving seven CNs. Each CN operates on separate virtual machines (VMs) and all of their peer-to-peer sessions go over the common software Open vSwitch (OvS). One POX controller is connected to the software switch. Then we captured all packets per network interface we defined (tap1, tap2 and so on).
Consensus node(CN) of Klaytn is built atop the Istanbul Byzantine Fault Tolerance(IBFT) protocol. IBFT is a consensus protocol for blockchain that guarantees immediate finality. There are four distinct types of consensus messages in IBFT: pre-prepare, prepare, commit, and roundchange. In terms of implementation, several types of messages are used to maintain the Klaytn system, including transaction msg, messages for the rlpx protocol, and consensus message. Consensus message refers to a message type utilized by CNs to achieve consensus. Each consensus message contains consensus message type, sender’s address, sender’s signature, message including round number and so on. All of these messages are encrypted with keys that are unique to each TCP session. To obtain the round number and message type, I extracted keys and decrypted the data.
Monitoring consensus messages represents the list of messages received per round for each CN. If one of the CNs becomes the proposer at a particular round number, it will not receive a preprepare message. I utilized Grafana's dashboard to represent the number of messages received from each CN.
This file does many things as described below: creating a txt file from a pcap file that contains every packet sent during a tcp session. identifying a secret key for every tcp session. decrypt each RLPx frame. record the outcome in a MySQL database. This python file uses Crypto, sha3, snappy, rlp library to decode each frame. You need to modify this file based on your system. This file is used in your host machine.
This file describes some useful functions for data_processing.py. You need to modify this file based on your system. This file is used in your host machine.
The Klaytn kcn file is created using this file. I slightly altered the code in order to obtain the secret key. Don't forget to run "make" once the original Klaytn rlpx has been converted to this. Your virtual machine uses this file. Also, make a SecData directory and you need to modify some code based on your system.
This file transmits the secret key extracted from rlpx.go to the host machine. You need to make a ‘secrets’ directory in your host machine and modify this code based on your system.
We decrypt and decode the received RLPx messages. Figure 3 shows the RLPx message structure. We use the counter-mode symmetric key decryption with AES. And the Snappy compression is used to decompress messages.
I used the open-source PacketSorter library to gather data. It sorts packets ordered by sequence number and any duplicated and empty messages are eliminated from the output pcap file. Also, this program attaches to each network interface. In order to capture all incoming and outgoing packets, I run PacketSorter at tap1. I used the Scapy package to read pcap files in Python.
The data processing.py file handles data decryption. The frame is decrypted using Crypto and the sha3 library. And the frame is decompressed using Snappy.
This project can run only on Ubuntu.
Things that you should keep in mind before the installment. There are many things that must be installed manually, so it will take some time to handle everything. The intended reader is Klaytn developers, so simple steps may be omitted. If you are familiar with Klaytn node implementation using virtual machines (VMs), it may be of great assistance. Additionally, I just figured I'd run 7 CNs. As a result, numerous variables are set to 7. This is something you should keep in mind and adjust for your project. Above all, Google is our best friend.
Without POX, the source and destination mac addresses of each path had to be hardcoded. This is accomplished automatically by POX, which is very convenient.
It is an open-source library that records each TCP flow and has a number of options for customizing the pcap file, including the ability to filter packets by sequence number and delete empty packets.
- x represents the vm number.
- Verify
- VM{x} connects to Bridged Network, tap{x}. Configure the two VMs as follow
- On VM{x} (use ifconfig to see VM’s network interface):
Two network interfaces may be connected to the VM. One should be connected to the tap, and the other should be connected to the NAT, enabling Internet communication.
- Download the POX folder from the git
- Need to open taps
- Update openflow rule in the OVS automatically.
- Now, you can ping between VMs. Inside the VM, ping to each other so that you make sure that you installed correctly.
- Installation:
- Save pcap file(remember {x} always stands for the VM number). This code captures packets passing through tap1, meaning the source or destination address is CN{x}.
- Make directories
- run data_processing
// you need to modify something in data_processing.py and data_processing_header.py according to your system. I stated what you have to do in “TODO”.
8. What VMs need to do (You must implement this procedure for each VM. Therefore, I recommend that you execute the first VM flawlessly and then simply duplicate it.)
- Install GO
- Install Klaytn
make (for ‘make’, make sure that you have gcc and make installed. If not, sudo apt install gcc && sudo apt install make)
- After this, you can follow the link below and you can skip STEP0 and the extracting part of the STEP1.
- clone the github repogitory
- Modify some code
// follow the link below to make auto-authentication with your host machine https://alvinalexander.com/linux-unix/how-use-scp-without-password-backups-copy/
- Setting is done. Run klaytn CN.
A script that automates the installation of Klaytn CNs would be of great assistance to those who wish to conduct network experiments with Klaytn CNs.