-
Notifications
You must be signed in to change notification settings - Fork 140
CounterBraids
This project provides an implementation of Counter Braids, a novel counter architecture for packet count measurement per TCP/UDP flow.
This implementation features an FPGA-based hardware datapath and a message-passing software decoder. The FPGA-based hardware datapath supports measuring per-flow packet counts in a compressed form for four 1Gbps Ethernet ports at the line rate. The message-passing software decoder decodes the compressed form of counts collected from the hardware datapath and recovers the per-flow packet count for all TCP/UDP flows.
- Status :
- Known issues :
- Version :
- Authors :
- NetFPGA source :
You need to download and install the NetFPGA base package, create the NF2 directory and set up all environment variables required for the NetFPGA base package. Please read the guide on www.netfpga.org with regard to downloading the gateware and software, and installing the software on the host PC.
Download the tarball here. Run the following commands to unpack the downloaded tarball to your NF2 directory.
cd ${NF2_ROOT}/..
tar -zxvf <path for the downloaded netfpga.counter_braids.1_0.tar.gz>
Use a text editor to open your ~/.bashrc file. Since the NetFPGA base package has been installed on the host PC, you should see the environment variable definitions including NF2_ROOT, NF2_DESIGN_DIR, etc at the end of your .bashrc file. These environment variable definitions were appended to the ~/.bashrc file when the NetFPGA base package was installed. If you don't see them, please carefully follow the guide on www.netfpga.org to download the gateware and software, and install the software on the host PC.
To install the Counter Braids package, a few environment variables must be changed or added. Please add the following few lines to the end of your ~/.bashrc file. These lines must appear after the NF2_ROOT, NF2_DESIGN_DIR, etc environment variable definitions in your ~/.bashrc file.
if [ -f ${NF2_ROOT}/projects/counter_braids/bashrc_addon ]; then
source ${NF2_ROOT}/projects/counter_braids/bashrc_addon
fi
To make it take effect immediately in your shell, run this command:
source ~/.bashrc
If the installed the NetFPGA base package on your host PC is version 2.0 , a minor fix to a Perl file is needed to run the Counter Braids application. Run the command below to apply the fix.
cp ${NF2_DESIGN_DIR}/misc/COutput.pm ${NF2_ROOT}/lib/Perl5/NF2/RegSystem
If the installed NetFPGA base package version is 2.1 or higher, you need not apply this fix, as the fix is already included in the NetFPGA base package version 2.1 or higher.
The regression tests verify the functionality of the Counter Braids implementation. In order to run the tests, you need to use Ethernet cables to connect eth1 to nf2c0, eth2 to nf2c1.
Test 1: Hardware produces compressed counts, software decodes them to produce per-flow packet count for all flows
- Name :
- Description :
- Load the design bit file onto the NetFPGA hardware, build the software
- Estimate the epoch period for the hardware-adapter software
- Estimate the test elapse time to run the hardware-adapter software and replay two pcap files to send packets out of eth1 and eth2
- Figure out the number of unique flows, the flow IDs and the per-flow packet count for the packets appearing in the two pcap files
- While running software to read the compressed counts out of NetFPGA, replay the two pcap files and send packets out of eth1 and eth2
- Aggregate the compressed counts saved for all epochs to produce cumulative compressed counts
- Run the decoder software to recover the per-flow packet count for all flows
- Compare the per-flow packet count recovered by the decoder software against the per-flow packet count precomputed from the two pcap files
- Location
${NF2_DESIGN_DIR}/regress/test_1
The steps to complete the test are explained in detail in the Usage section.
Use two Ethernet cables to connect the host port eth1 to the NetFPGA port nf2c0 and the host port eth2 to the NetFPGA port nf2c1.
Run the command below to load the design bit file onto the NetFPGA hardware:
sudo nf2_download ${NF2_ROOT}/bitfiles/counter_braids.bit (alternatively, log on as root and run this command)
Run the following command to build the header files from the register definition files in the Counter Braids package. There are hundreds of thousands of registers defined in the Counter Braids NetFPGA design, so it takes about 23 minutes to build the header files for this large number of registers. Your patience would be appreciated.
nf2_register_gen.pl --project counter_braids
CPCI is a design included in the NetFPGA base package. The Counter Braids software needs to access some registers in the CPCI design. Please run the following command to build the header files from the register definition files for the CPCI design included in the NetFPGA base package:
nf2_register_gen.pl --project CPCI_2.1
Now run the following commands to build the Counter Braids software:
cd ${NF2_DESIGN_DIR}/sw
make
The NetFPGA hardware for Counter Braids maintains two sets of compressed form of counts internally. In every epoch one set is updated to count the incoming packet, while the other set is read out by the hardware-adapter software and saved to text files. When an epoch expires, a new epoch starts and the two sets exchanges their roles.
An epoch must be long enough so that the Counter Braids hardware-adapter software may read one set of compressed counts on NetFPGA and write back to clear the compressed counts. Run the commands below to find how long it takes the Counter Braids hardware-adapter software to do this task when no pcap files are being replayed on the host PC at the same time.
cd ${NF2_DESIGN_DIR}/regress/test_1/work_area
sudo time -f "%e" estimate_cb_epoch (alternatively, log on as root and run this command)
Program "estimate_cb_epoch" writes and reads one set of compressed counts on NetFPGA and saves them to files on the host PC. No pcap files are being replayed at the same time. After estimate_cb_epoch finishes, the shell command "time" prints to the "standard error" output stream the elapsed real time in seconds to run estimate_cb_epoch. So the number appearing in the last line of the output stream is the elapsed real time to run estimate_cb_epoch to the finish.
The recommended value for "epoch_period" is computed as the integer ceiling of (the elapsed real time to run estimate_cb_epoch) * 3, where the inflation factor 3, accounts for the potential interference from the replay of the pcap files. For example, if the elapsed real time to run estimate_cb_epoch on the host PC is 3.5 seconds, the recommended value for "epoch_period" is 11 seconds. The value of "epoch_period" will be needed in the step of "While running software to read the compressed counts out of NetFPGA, replay the two pcap files".
Estimate the test elapse time to run the hardware-adapter software and replay two pcap files to send packets out of eth1 and eth2
Open two xterm windows, replay two pcap files at the two xterm windows simultaneously. Measure how long it takes to finish replaying both pcap files.
At one xterm window, run commands:
cd ${NF2_DESIGN_DIR}/regress/test_1/work_area
sudo time -f "%e" tcpreplay --intf1=eth1 --topspeed ../preserved_area/med_size_0.pcap (alternatively, log on as root and run this command)
At the other xterm window, run commands:
cd ${NF2_DESIGN_DIR}/regress/test_1/work_area
sudo time -f "%e" tcpreplay --intf1=eth2 --topspeed ../preserved_area/med_size_1.pcap (alternatively, log on as root and run this command)
No Counter Braids hardware-adapter software is running on the host PC at the same time. At each xterm window after the tcpreplay command finishes, the shell command "time" prints to the "standard error" output stream the elapsed real time in seconds to run a pcap file replay. So at each xterm window the number appearing in the last line of the output stream is the elapsed real time to run the pcap file replay to the finish. Find the longer of the two elapse times for the two pcap file replays.
In a step to do later the hardware-adapter software will run for "test_elapse_time" seconds to deal with the packets sent out when the two pcap files are replayed. The recommended value for "test_elapse_time" is computed as the integer ceiling of (the longer of the two pcap file replay elapse times) * 3, where the inflation factor 3, accounts for the potential interference from the Counter Braids hardware-adapter software. For example, if the longer of the two pcap file replay elapse times is 10.5 seconds, the recommended value for "test_elapse_time" is 32 seconds. The value of "test_elapse_time" will be needed in the step of "While running software to read the compressed counts out of NetFPGA, replay the two pcap files".
Figure out the number of unique flows, the flow IDs and the per-flow packet count for the packets appearing in the two pcap files
Two pcap files, med_size_0.pcap and med_size_1.pcap, are available in the directory of ${NF2_DESIGN_DIR}/regress/test_1/preserved_area.
There are two options to finish this step:
1. Use a tool called "ipsumdump" to extract the flow IDs out of the pcap files.
2. Copy the extracted flow IDs from directory.
If ipsumdump is not installed on the host PC, you may download it from http://www.cs.ucla.edu/~kohler/ipsumdump and install it on the host PC.
If ipsumdump has been installed on the host PC, run the following commands.
cd ${NF2_DESIGN_DIR}/regress/test_1/work_area
ipsumdump -s -d -S -D -p ../preserved_area/med_size_0.pcap | grep -E 'T||U' | grep -v -E '!|-' > pkt_0.txt
ipsumdump -s -d -S -D -p ../preserved_area/med_size_1.pcap | grep -E 'T||U' | grep -v -E '!|-' > pkt_1.txt
cat pkt_0.txt pkt_1.txt > pkt.txt
sort pkt.txt -o pkt.txt
uniq pkt.txt > no_dup_no_cnt_pkt.txt
count_dup.pl pkt.txt > no_dup_with_cnt_pkt.txt
wc -l no_dup_no_cnt_pkt.txt
no_dup_no_cnt_pkt.txt is the file with all unique TCP/UDP flow IDs appearing in either med_size_0.pcap or med_size_1.pcap or both. no_dup_with_cnt_pkt.txt is the file with the unique flow IDs and the number of packets per flow appearing in the two pcap files.
The line count reported by the shell command "wc" tells the number of unique flows. This number of unique flows is denoted as num_flows hereafter. Take note for num_flows, as it is needed by the Counter Braids decoder software. Specifically for the two pcap files, med_size_0.pcap and med_size_1.pcap, num_flows is 959.
If you don't want to install ipsumdump on the host PC, you may simply copy no_dup_no_cnt_pkt.txt and no_dup_with_cnt_pkt.txt from the preserved_area directory and thus finish this step.
cd ${NF2_DESIGN_DIR}/regress/test_1/work_area
cp ../preserved_area/no_dup_no_cnt_pkt.txt .
cp ../preserved_area/no_dup_with_cnt_pkt.txt .
wc -l no_dup_no_cnt_pkt.txt
As described in the option of "Use ipsumdump", the shell command "wc" reports the number of unique flows. Take note of this number as num_flows for future use.
While running software to read the compressed counts out of NetFPGA, replay the two pcap files and send packets out of eth1 and eth2
You need to have three xterm windows open to go through this step.
At the first xterm window, run commands:
cd ${NF2_DESIGN_DIR}/regress/test_1/work_area
sudo cb <epoch_period> <test_elapse_time> (alternatively, log on as root and run this command)
While cb is running, run commands at the second xterm window:
cd ${NF2_DESIGN_DIR}/regress/test_1/work_area
sudo tcpreplay --intf1=eth1 --topspeed ../preserved_area/med_size_0.pcap (alternatively, log on as root and run this command)
While cb is running, run commands at the third xterm window:
cd ${NF2_DESIGN_DIR}/regress/test_1/work_area
sudo tcpreplay --intf1=eth2 --topspeed ../preserved_area/med_size_1.pcap (alternatively, log on as root and run this command)
Ensure that replaying the two pcap files finishes before cb finishes running. If not, increase the command-line argument test_elapse_time for cb to make it run longer, and repeat the commands for the three xterm windows.
The Counter Braids hardware-adapter software runs for a number of epochs and saves the compressed counts read out in every epoch to files in the dump/ directory. The number of epochs for which it runs is computed as the integer ceiling of (test_elapse_time / epoch_period). This number of epochs is denoted as num_epochs hereafter. For example, if test_elapse_time is 32 seconds and epoch_period is 11 seconds, num_epochs is 3. Take note for num_epochs, as it is needed in the step of "Aggregate the compressed counts".
In this test, we simulate the decoding of one epoch recorded by Counter Braids. However, due to the asynchronous nature of the Counter Braids epoch and the replay of traces, packet counts can end up in different epochs. We hence run the commands below to generate a set of cumulative compressed counts which is the sum of the per-epoch compressed counts saved for all epochs. In reality, Counter Braids decodes the flow sizes from each epoch independently.
cd ${NF2_DESIGN_DIR}/regress/test_1/work_area
multi_epoch <num_epochs>
Two files are created in the dump/ directory. One is cum_l1_cnt_dump.txt and the other is cum_l2_cnt_dump.txt.
Run the Counter Braids decoder software to recover the per-flow packet count for all flows.
cd ${NF2_DESIGN_DIR}/regress/test_1/work_area
mul <num_flows> no_dup_no_cnt_pkt.txt dump/cum_l1_cnt_dump.txt dump/cum_l2_cnt_dump.txt
One file, flow_estimate.txt, is created. This file holds the recovered per-flow packet count for all flows.
Compare the per-flow packet count recovered by the decoder software against the per-flow packet count precomputed from the two pcap files
After all steps described above are completed, the Counter Braids decoder software recovers the per-flow packet count. Now the recovered per-flow packet count is checked against the per-flow packet count precomputed from the two pcap files.
cd ${NF2_DESIGN_DIR}/regress/test_1/work_area
diff flow_estimate.txt no_dup_with_cnt_pkt.txt
The shell command "diff" shows no difference between the recovered per-flow packet count and the precomputed per-flow packet count. This validates the Counter Braids implementation.