uftrace on Android

uftrace on Android (WIP)

Currently uftrace is not running properly under Android.
Things that don't work -> Argument Tracing, Dynamic Tracing

This document provides a brief introduction to how uftrace can be built and run on Android.

Introduction

Running uftrace on Android can be done in several ways (context).

• chroot'ed(?) env. (eg. inside termux app)

  • install uftrace package on Termux
  • compile uftrace from source on Termux

• system shell env. (ADB)

  • run uftrace binary under chroot'ed env
  • build uftace as NDK binary

Testing Environment

• AOSP 10 - Android Q

• Google Pixel (aarch64)

  • Android: aosp_sailfish-eng 10 (android-10.0.0_r1)
  • Kernel: 3.18.137 (fixed to 3.18.x)

• Virtual Device cuttlefish (x86)

  • Android: aosp_cf_x86_phone (aosp_master@eef9850)
  • Kernel: 5.4.6 (stock kernel. custom kernel available)

General preperation

Get AOSP image

To compile AOSP, follow instruction under AOSP Document.
For sudo permission in adb shell, build AOSP with userdebug or eng varients.

If you're trying to test on cuttlefish Virtual Device, you don't have to compile it. Jump to Setting up Cuttlefish.

Compile kernel with ftrace (function)

The stock kernel included in Android doesn't support with ftrace function tracer.
In order to use kernel tracing with uftrace, you should compile kernel with FUNCTION_TRACER option enabled.

• CONFIG_FUNCTION_TRACER=y
• CONFIG_DYNAMIC_FTRACE=y

Follow instruction for compiling AOSP kernel
For more information: AOSP Dynamic Ftrace

Set up environment

With your phone

For every Android phone in the world, there are different procedures for installing AOSP (unlock phone) and there might be even impossible phones. So in this section will only briefly cover the concepts.

1. Unlock your phone's bootloader : Search Here
2. Get or build AOSP image
3. Boot your device with fastboot : adb reboot fastboot
4. Flash image to your phone with fastboot : fastboot flash

To find the proper way to prepare AOSP on your device, Google it.

With cuttlefish Virtual Device

1. Build cuttlefish host tool (like emulator)

git clone https://github.com/google/android-cuttlefish
cd android-cuttlefish
apt install build-essential devscripts debhelper
apt install cdbs config-package-dev
debuild -i -us -uc -b
sudo dpkg -i ../cuttlefish-common_*_amd64.deb
sudo apt-get install -f

2. Get cuttlefish AOSP image from Android CI
   1. From aosp-master branch, click aosp_cf_x86_phone userdebug varient
   2. At Artifacts tab, download below files
      • aosp_cf_x86_phone-img-xxxxxx.zip
      • cvd-host_package.tar.gz
3. Extract downloaded files

mkdir cf && cd cf
tar xvf /path/to/cvd-host_package.tar.gz
unzip /path/to/aosp_cf_x86_phone-img-xxxxxx.zip

4. Run cuttlefish virtual machine

HOME=$PWD ./bin/launch_cvd
# To boot with custom compiled kernel, use -kernel-path option
# HOME=$PWD ./bin/launch_cvd -kernel_path /path/to/kernel/../x86/boot/bzImage
# If the graphic is slow, try below option
# HOME=$PWD ./bin/launch_cvd -gpu_mode=drm_virgl

5. Connect cuttlefish with VNC
   To view the screen, use VNC client (Tiger VNC) to access localhost 6444 port.

java -jar tightvnc-jviewer.jar -ScalingFactor=50 -Tunneling=no -host=localhost -port=6444

For more information: cuttlefish git

Install Uftrace on Android

Install Termux App

1. Download Termux APK: F-droid Store
2. Install APK with ADB

adb install com.termux*.apk

3. Open termux app

Install uftrace termux package

There is already Uftrace Package in the Termux package repo.

pkg update
pkg install uftrace
uftrace --version
# uftrace v0.9.3 ( python tui perf sched dynamic )

Compile Termux App (Not Working, WIP)

For using latest feature of uftrace, compile inside termux.
Since termux doesn't have gcc, use clang instead. (gcc available with unofficial repo).

To meet the dependency requirements, install below pacakges.

• pkg-config
• libelf
• python
• capstone
• libluajit

pkg update
pkg install git make clang
pkg install pkg-config libelf python capstone libluajit
./configure --prefix=$PREFIX
make -j 8
# WILL FAIL! Source code and LDFLAGS modification needed. 
# libandroid-spawn, argp, shmem etc..

Since there is no libdw package support from termux, separate compilation is required.
Unfortunately, elfutils doesn't support compiling with clang. (It has to be done with some hacky way)

Refer to the following link to find out how to do it: libelf termux building

Thanks to contributor @gonapps, there's a demo for compiling and running uftrace on Android.
(The demo below is different from the environment in this document)

For custom compile, refer to Termux package build script:
https://github.com/termux/termux-packages/tree/master/packages/uftrace

Running Uftrace on Android

Inside chroot'ed env (termux)

From termux app, run uftrace.

uftrace --version
# uftrace v0.9.3 ( python tui perf sched dynamic )

Inside system shell (ADB)

From the Host side, elevate to root permission and execute uftrace under termux directory.
(su is allowed only on AOSP phonedebug, eng varient.)

# From Host side, type:
adb shell
# Connect into android system shell
# vsoc_x86:/ $ 
whoami                      # shell
su
whoami                      # root
export PREFIX=/data/data/com.termux/files/usr
$PREFIX/bin/uftrace -L $PREFIX/lib --version
# uftrace v0.9.3 ( python tui perf sched dynamic )

Tracing sample program with uftrace

Since termux environment doesn't have symbols for mcount, compile with -pg option will fail.
When compiling sources inside termux, compile with -lmcount option.

# clang is binded to gcc
gcc -pg $PREFIX/uftrace/tests/s-abc.c
# ============= Result Truncated =============
# $PREFIX/bin/aarch64-linux-android-ld: $PREFIX/tmp/s-abc-833b6b.o: in function `c':
# s-abc.c:(.text+0xcc): undefined reference to `_mcount'
# clang-9: error: linker command failed with exit code 1 (use -v to see invocation)

gcc -pg -lmcount $PREFIX/uftrace/tests/s-abc.c
uftrace record a.out
uftrace replay
## DURATION    TID     FUNCTION
#            [ 2163] | main() {
#            [ 2163] |   a() {
#            [ 2163] |     b() {
#   0.880 us [ 2163] |       c();
#   3.475 us [ 2163] |     } /* b */
#   4.448 us [ 2163] |   } /* a */
#   5.631 us [ 2163] | } /* main */

Kernel Tracing with uftrace

Custom kernel with ftrace - function graph support is required!
(If no symbol name is shown, set kptr_restrict value to 0.)

Termux environment doesn't support for root permission natively.
So in here, we'll running uftrace from system shell (ADB).

If you can get elevated permission inside (e.g. rooted phone) just try with su instead of ADB.

# 0. Prepare environment with ftrace - function graph supported kernel
# In here, I'll show how to boot custom kernel with Phone (Pixel 1, aarch64, sailfish, 3.18.xx)
# For booting custom kernel with cuttlefish, look [4. Run cuttlefish virtual machine] section.
adb reboot fastboot
fastboot boot Image.gz-dtb             # Your own custom kernel image.

# 1. Before start tracing, compile s-mmap.c inside termux
gcc -pg -lmcount $PREFIX/uftrace/tests/s-mmap.c -o t-mmap

# 2. From Host side, type and connect into android system shell
adb shell                              # sailfish:/ $ 
su
whoami                                 # root

# 3. Check current kernel supports ftrace - function graph.
cat /sys/kernel/debug/tracing/available_tracers
# blk function_graph function nop

# (optional) check kernel pointer restrict is set to 0.
cat /proc/sys/kernel/kptr_restrict
echo 0 > /proc/sys/kernel/kptr_restrict

# 4. Start Kernel trace inside Android
export PREFIX=/data/data/com.termux/files/usr
cd $PREFIX/../home/uftrace/tests
$PREFIX/bin/uftrace -L $PREFIX/lib record -K 10 t-mmap
# DURATION     TID     FUNCTION
#            [  3635] | main() {
#            [  3635] |   foo() {
#            [  3635] |     sys_openat() {
#            [  3635] |       handle_IPI() {
#            [  3635] |         irq_enter() {
#   1.719 us [  3635] |           rcu_irq_enter();
#   1.355 us [  3635] |           irqtime_account_irq();
...

Example demo recorded file: mmap_f-AOSP-kernel-aarch64.data.tar.xz