Microwalk reads its entire run configuration from a single YAML file. The file specifies the loaded pipeline modules and their settings.
The pipeline is structured as follows: First, the testcase
stage produces/loads a number of test case files (= inputs), which are passed to the trace
stage.
The trace
stage uses these test cases to generate raw execution traces using a language-specific backend (e.g., Pin for x86 binaries or Jalangi2 for JavaScript).
Then, the raw traces are passed to the preprocess
stage, that parses the raw traces and converts them into Microwalk's generic trace format. The resulting generic
traces can finally be analyzed by various analysis
modules.
A configuration file generally looks as follows:
# Configuration preprocessor section (optional)
base-file: base-file.yml
constants:
CONSTANT_NAME: some value
---
# Actual configuration
general:
logger:
# Logger settings
testcase:
module: # Test case generator module name
module-options:
# Test case generator module options
trace:
module: # Trace module name
module-options:
# Trace module options
options:
# General trace stage options
preprocess:
module: # Preprocessor module name
module-options:
# Preprocessor module options
options:
# General preprocessor stage options
analysis:
modules: # List of analysis module settings
- module: # 1st analysis module name
module-options:
# 1st analysis module options
- module: # 2nd analysis module name
module-options:
# 2nd analysis module options
# ...
options:
# General analysis stage options
A valid configuration file must specify at least one module for each stage.
Microwalk has a simple configuration preprocessor that supports environment variables, constants and loading other configuration files/parts. The preprocessor is executed before handling the configuration itself.
Base configuration file which should be loaded and parsed before the current one (nesting is supported). Keys existing in both the base file and the current file are always superseded by those of the current file.
A number of arbitrary string constants, which can be referenced by $CONSTANT_NAME
in other parts of the file.
Predefined constants:
-
CONFIG_PATH
Absolute path of the directory where the main configuration file resides. -
CONFIG_FILENAME
Name of the main configuration file, without extension (e.g.,target-aes
fortarget-aes.yml
).
All environment variables of the current process environment are available as $$VARIABLE_NAME
(mind the double $
) and are treated just like constants.
Controls the logger.
-
log-level
(optional)
Sets the minimal log level messages need in order to be printed to stdout.Allowed values:
debug
warning
error
(default)
It is recommended to test the configuration file with
warning
level enabled, as some modules may report possible misconfiguration as warnings. -
file
(optional)
Output file for the log. This file will receive the same log messages as stdout.
Configures process monitoring.
Currently, this does only track total memory usage.
-
enable
(optional)
If set totrue
, enables process monitoring. If this isfalse
, all other monitoring options are ignored.Default:
false
-
sample-rate
(optional)
Sets the sample rate (milliseconds) for querying process statistics.Default: 500
Loads existing test case files (*.testcase
) from a given directory.
Our templates usually only use this module, as we generate the test cases through the analysis script.
Options:
input-directory
Input directory containing test case files.
Generates random byte arrays of a given length and stores them as test cases.
Options:
-
length
Amount of bytes per test case. -
amount
Number of test cases. -
output-directory
Output directory for generated test cases.
Calls an external application to generate test cases. The given command is executed for each test case.
The working directory is set to output-directory
.
Options:
-
amount
Number of test cases. -
output-directory
Output directory for generated test cases. -
exe
Path/name of the program to be called.Example:
openssl
-
args
Format string for the program argument. The following placeholders are available:{0}
: Integer test case ID{1}
: Name of the test case file{2}
: Full path and name of the test case file
Example:
genrsa -out {1} 2048
The above examples would yield the following command line:
openssl genrsa -out 0.testcase 2048
General options:
-
input-buffer-size
(optional)
Size of trace stage input buffer. This controls the number of active pending test cases which are not currently processed by the next stage. Since test case objects are quite small, buffering a few does not cause much harm; especially, if test case generation takes a bit (e.g., random asymmetric keys).Default: 1
-
max-parallel-threads
(optional)
Amount of concurrent trace threads. This is only applied when the selected trace module supports parallelism.Default: 1
Loads existing raw traces from a given directory. This module tries to compute the trace file names from test case IDs, and makes the following assumptions:
- The testcases are loaded using the
load
module; - The trace files have not been renamed, i.e. their names follow the
t<ID>.trace
format.
Options:
input-directory
Input directory containing trace files.
Passes through the test cases without generating traces. This module is designed to be used in conjunction with the preprocessed trace loader, where raw traces are not needed.
Generates traces using a Pin tool.
Options:
-
pin-tool-path
Path to the compiled Pin tool (PinTracer
binary). -
wrapper-path
Path to the wrapper executable (based onPinTracerWrapper
). -
output-directory
Output directory for raw traces. -
images
List of names of interesting images (= binaries). Only instructions from these images are traced; all others are ignored.Example:
images: - wrapper.exe - mylibrary.dll
-
pin-path
(optional)
Path to thepin
executable.Default:
pin
-
wrapper-args
(optional)
List of arguments passed to the wrapper binary.Example:
wrapper-args: - trace - ecb128
-
stack-tracking
(optional)
Enable tracking of stack allocations and deallocations. Enabling this setting allows the preprocessor to assign memory accesses to specific stack frames.Default:
false
-
rdrand
(optional)
Constant value to use as output of the x86rdrand
instruction.Expects a 64-bit hex number, e.g.
0x0000ffff0000ffff
. -
cpu
(optional)
Simulated CPU model. This can be used to trigger specific code paths in a library, which depend on the availability of certain CPU features.Currently supported processors and microarchitectures:
0
(default): CPU as used in the host system.1
: (Intel) Pentium III2
: (Intel) Merom3
: (Intel) Westmere4
: (Intel) Ivy Brigde
-
stack-tracking
(optional)
Enable stack tracking. This is an experimental feature for tracking individual stack frames, instead of referencing the stack as a whole.Default:
false
-
environment
(optional)
A list of enviroment variables which should be passed to the process.
General options:
-
input-buffer-size
(optional)
Size of preprocessor stage input buffer. This throttles the trace generator: Raw traces can be quite large and take up a lot of disk space. If only existing raw traces are loaded from disk, this setting does not significantly influence execution.Default: 1
-
max-parallel-threads
(optional)
Amount of concurrent trace threads. This is only applied when the selected preprocessor module supports parallelism.Default: 1
Loads existing preprocessed traces from a given directory. This module tries to compute the trace file names from test case IDs, and makes the following assumptions:
- The testcases are loaded using the
load
module; - The preprocessed trace files have not been renamed, i.e. their names follow the
t<ID>.trace.preprocessed
format.
Raw traces are ignored, thus it is recommended to use the passthrough
module for the trace stage.
Options:
input-directory
Input directory containing preprocessed trace files.
Passes through the test cases and raw traces without preprocessing. This module should only be used with an empty (passthrough
) analysis stage, since no preprocessed traces are inserted into the pipeline.
Preprocesses raw traces generated with the Microwalk Pin tracer backend.
Options:
-
store-traces
(optional)
Controls whether preprocessed traces are written to the file system. If set tofalse
, preprocessed traces are only kept in memory and are discarded after the analysis has finished.Default:
false
-
output-directory
(optional)
Output directory for preprocessed traces. Must be set whenstore-traces
istrue
. -
keep-raw-traces
(optional)
Controls whether raw traces are kept after preprocessing has completed. Deleting raw traces may free up disk space.Default:
false
Dumps raw Pin trace files in a human-readable form. Primarily intended for debugging.
Options:
-
output-directory
Output directory for trace text files. -
map-files
(optional)
A list of MAP files which contain a mapping of image offsets to symbol names.Example:
map-files: - wrapper.exe.map - mylibrary.dll.map
Note that the
pin-dump
module is a debugging tool, so for the sake of stability it is pretty conservative when trying to retrieve symbol information. The raw trace will always include the raw address, independent from the success of the symbol resolution. -
map-directory
(optional)
Path to a directory containing MAP files. This loads all files that end with.map
from the given directory, in addition to the ones specified manually through themap-files
key.
Preprocesses raw traces generated with the Microwalk Jalangi2 tracer backend.
Options:
-
store-traces
(optional)
Controls whether preprocessed traces are written to the file system. If set tofalse
, preprocessed traces are only kept in memory and are discarded after the analysis has finished. If set totrue
, preprocessed traces are streamed to binary files and are not kept in memory -- thus, analysis needs to load them again.Default:
false
-
output-directory
(optional)
Output directory for preprocessed traces. Must be set whenstore-traces
istrue
. -
map-directory
(optional)
Output directory for MAP files. -
columns-bits
(optional)
Number of bits used for encoding a column number in a 32-bit integer. The remaining bits are used for encoding a line number.The default value should work for reasonable settings; however, when dealing with extreme cases (e.g., a minified library residing in a single, very long line), make sure to adjust this in order to avoid erroneous traces.
Default: 13
General options:
-
input-buffer-size
(optional)
Size of analysis stage input buffer. Note that the buffer only contains pending preprocessed traces, i.e., in addition to the ones that are already processed by the analysis stage(s).Default: 1
-
max-parallel-threads
(optional)
Amount of concurrent analysis threads. This is only applied when the selected analysis module supports parallelism.Default: 1
Ignores all passed traces. Intended for "offline" trace generation/preprocessing without immediate analysis.
Dumps preprocessed trace files in a human-readable form.
Options:
-
output-directory
Output directory for trace text files. -
include-prefix
(optional)
Controls whether the trace prefix is added to each text file. This will yield complete trace dumps, but will also take up more disk space.Default:
false
-
map-files
(optional)
A list of MAP files which contain a mapping of image offsets to symbol names.Example:
map-files: - wrapper.exe.map - mylibrary.dll.map
-
map-directory
(optional)
Path to a directory containing MAP files. This loads all files that end with.map
from the given directory, in addition to the ones specified manually through themap-files
key. -
skip-memory-accesses
(optional)
Controls whether memory accesses should be skipped when writing the trace dump.Default:
false
-
skip-jumps
(optional)
Controls whether "jump" branches should be skipped when writing the trace dump.Default:
false
-
skip-returns
(optional)
Controls whether "return" branches should be skipped when writing the trace dump.Default:
false
Calculates several trace leakage measures for each memory accessing instruction.
This is a legacy module. Use the call-stack-memory-access-trace-leakage
, which is better optimized and yields more detailed results.
Options:
-
output-directory
Output directory for analysis results. -
output-format
(optional)
The output format of analysis results.Supported formats:
txt
: Formats each analysis result as a line in a text file.csv
(default): Creates a CSV file, where lines are instructions and columns are the results of the different measures.
In addition, a
-
dump-full-data
(optional)
Writes the entire final state of the analysis module into a separate output file. This may be useful for identifying the specific test cases which generated a particular trace and analysis result.Default:
false
-
map-files
(optional)
A list of MAP files which contain a mapping of image offsets to symbol names.Example:
map-files: - wrapper.exe.map - mylibrary.dll.map
Calculates several trace leakage measures for each memory accessing instruction and call stack.
This module yields more accurate results than the instruction-memory-access-trace-leakage
module:
The leakage measures are computed over each unique combination of instruction and call stack, instead of only considering instructions.
Additionally, the dump-full-data
mode outputs detailed information over the encountered call stacks and their respective hit counts.
Note that a leakage shown for a given memory access does not imply that this access is non-constant-time: The leakage may have also been caused by a control flow variation higher up in the call chain.
Thus, while this module is quite fast due to its focus on memory access traces, it fails at accurately localizing and attributing control flow leakages. If possible, we recommend using the control-flow-leakage
module, which needs a bit more resources, but yields very accurate leakage assessments.
Options:
-
output-directory
Output directory for analysis results. -
output-format
(optional)
Output format of the leakage report.Supported formats:
txt
: Formats each analysis result as a line in a text file.csv
(default): Creates a CSV file, where lines are instructions and columns are the results of the different measures.
In addition, text/CSV files containing information about the detected call stacks are generated.
-
dump-full-data
(optional)
Writes the entire final state of the analysis module into a separate output file. This may be useful for identifying the specific test cases which generated a particular trace and analysis result.Default:
false
-
map-files
(optional)
A list of MAP files which contain a mapping of image offsets to symbol names.Example:
map-files: - wrapper.exe.map - mylibrary.dll.map
-
map-directory
(optional)
Path to a directory containing MAP files. This loads all files that end with.map
from the given directory, in addition to the ones specified manually through themap-files
key.
Merges all preprocessed traces into a call tree, while encoding trace divergences as "splits" in the tree structure.
This encoding enables an accurate analysis that manages to attribute leakages to individual branch/memory access instructions. In addition, it yields a rough assessment of the leakage severity through several measures.
The implementation of this module is highly optimized, so it should work on a typical machine for reasonable workloads.
A guide for interpreting the generated reports can be found here.
Options:
-
output-directory
Output directory for analysis results. -
map-files
(optional)
A list of MAP files which contain a mapping of image offsets to symbol names.Example:
map-files: - wrapper.exe.map - mylibrary.dll.map
-
map-directory
(optional)
Path to a directory containing MAP files. This loads all files that end with.map
from the given directory, in addition to the ones specified manually through themap-files
key. -
include-testcases-in-call-stacks
(optional)
Controls whether test case ID trees should be included in the analysis result. While those are not strictly necessary for interpreting a reported leakages, they may help by specifying which input led to which behavior.Default:
true
-
dump-call-tree
(optional)
If enabled, this dumps the entire call tree to a text file in the output directory.This feature is mostly intended for debugging: The call tree dump is quite verbose and thus very large.
Default:
false
-
include-memory-accesses-in-dump
(optional)
Controls whether memory accesses should be included in the call tree dump. If this is set tofalse
, only branches are written to the dump.Default:
true