-
Notifications
You must be signed in to change notification settings - Fork 588
Usage
rr can currently only record and replay 32-bit objects. This restriction will be lifted in a future version.
rr record /path/to/my/program --args
rr replayIf you're running rr on an (un-docked) laptop, the CPU scaling governor can make a big difference in recording overhead; at least up to 2x. This has been observed to happen whether or not your laptop is on AC power. (However, laptops running in desktop docks have been observed not to be affected by this issue.)
The easiest way to squeeze out max perf is to change the CPU governor to 'performance'.
On Fedora:
sudo yum install kernel-tools sudo cpupower frequency-set -g performance
On Ubuntu:
sudo apt-get install cpufrequtils sudo cpufreq-set -g performance
This setting is only temporary however; it only lasts until the next reboot.
Frequency scaling can also be disabled in the BIOS.
Running rr in "record" mode saves the execution history of your application into a "trace directory". To invoke the recorder, run
rr record /path/to/your/application [arguments to application]
The recorded trace is saved to the path ~/.rr/application-$n by default.
The overhead of recording an application with rr is pretty low, but it depends on the application's workload. In general you should expect around a 1.2x to 1.4x slowdown. At the lowest end, a purely CPU-bound tracee will incur around 1.1x-1.2x overhead. At the high end, a tracee that just makes syscalls in a tight loop can have 4x slowdown or more.
rr doesn't record shared-memory multithreading, so it forces your application's threads to execute serially. Your application can see an additional slowdown if it takes advantage of multicore parallelism.
The record token is optional.
To debug your most recent recording, run
$ rr replay GNU gdb (GDB) Fedora 7.6.50.20130731-19.fc20 ... 0x4cee2050 in _start () from /lib/ld-linux.so.2 (gdb)
rr automatically spawns a gdb client and connects it to the replay server. You can set breakpoints within the gdb session, step, continue, interrupt, etc. as you would normally in gdb.
To replay a recording other than your most recently recorded one, specify its trace directory: rr replay some-other-trace-dir.
What's special about rr is restarting the replayed execution. Within the same gdb session, you can replay execution from the beginning again by issuing the run command:
(gdb) run The program being debugged has been started already. Start it from the beginning? (y or n) y ...
gdb preserves all your breakpoints and debugging state across the run command. Because rr replay is entirely deterministic, the restarted replay session will execute exactly the same sequence of instructions as the previous replay session. This allows you to take debugging "shortcuts" like hard-coding memory addresses to examine: all pointers have the same values in every replaying of your recorded trace.
To more precisely target your debugging session, you can set things up so that the debugger is only spawned after a target OS process is forkd or execd, and/or when a particular execution event is reached. If your test suite prints a message that reveals the process IDs of launched subprocesses, like for example
[2789] WARNING: '!compMgr', file /home/cjones/rr/mozilla-central/xpcom/glue/nsComponentManagerUtils.cpp, line 59 ^^^^
then you can program rr to launch gdb when that process is execd by using the -p PID option:
$ rr replay -p 2789 ... For application/x-test found plugin libnptest.so -------------------------------------------------- ---> Reached target process 2789 at event 159937. -------------------------------------------------- ... (gdb)
(Remember, rr sets things up so that processes look like they have the same PID in replay as they did during recording.) If you instead want to launch the debugger when that process is forkd, use the -f PID option.
rr associates an "event number" with each event it records. You can have rr mark writes to stdio with the corresponding event number and PID that made the write by using the -m switch
$ rr -m replay ... [rr 2789 163548]LoadPlugin() /tmp/tmpUrp7e7/plugins/libnptest.so returned 5a7c7140 [rr 2789 164788][2789] WARNING: '!compMgr', file /home/cjones/rr/mozilla-central/xpcom/glue/nsComponentManagerUtils.cpp, line 59
Note that rr tagged the output with [rr PID EVENT-NUMBER], in this case 2789/164788. You can program rr to launch the debugger at a specific event by using the -g EVENT options
$ rr -m replay -g 164788 ... [rr 2789 163548]LoadPlugin() /tmp/tmpUrp7e7/plugins/libnptest.so returned 5a7c7140 -------------------------------------------------- ---> Reached target process 0 at event 164788. -------------------------------------------------- ... (gdb)
The target process and/or event persist through your debugging session. That means issuing the gdb run command to restart replay will go back to the targeted process/event.
(gdb) run The program being debugged has been started already. Start it from the beginning? (y or n) y ... [rr 2789 163548]LoadPlugin() /tmp/tmpUrp7e7/plugins/libnptest.so returned 5a7c7140 -------------------------------------------------- ---> Reached target process 0 at event 164788. --------------------------------------------------
If you want to restart replay at a different event, you can pass the event number as an argument to the gdb run command, like so
(gdb) run 159937 ... [rr 2734 159846]For application/x-test found plugin libnptest.so -------------------------------------------------- ---> Reached target process 0 at event 159937. --------------------------------------------------
If you just want to replay your recording without attaching a debugger client, invoke
rr -m replay -a
When you call a program function from the debugger, rr temporarily clones the current program state, runs your function in the clone (echoing console output), and then throws away the clone so replay can continue from the state before running the function. Therefore attempting to alter program state persistently by calling functions will not work.
rr supports gdb's checkpoint, restart and delete checkpoint commands.
Currently, the /path/to/your/application image you recorded must not change before you replay the recording. If the executable image changes, all kinds of bad things will happen. This will be fixed in the future.
Run rr -h or rr --help to see the most up-to-date list. The options below are most useful for rr users, as opposed to developers of rr.
General options:
-
-v, --verbose: log messages that may not be urgently critical to the user. -
-m, --mark-stdio: write "current event number" before every stdio output line (seerr replay -gbelow).
Recorder parameters:
-
-c, --num-cpu-ticks=<NUM>: maximum number of 'CPU ticks' (currently retired conditional branches) to allow a task to run before interrupting it. -
-e, --num-events=<NUM>: maximum number of events (syscall enter/exit, signal, CPU interrupt, ...) to allow a task before descheduling it.
Recording traces under different scheduling params can help reproduce nondeterministic bugs. rr's scheduler is relatively deterministic.
Replay parameters:
-
-f, --onfork=<PID>: debug when forked -
-p, --onprocess=<PID>: debug when execed -
-g, --goto=<EVENT-NUM>: execute forward until event is reached before debugging