Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

debug: add tools for task aware debug #4810

Merged
merged 3 commits into from
Nov 10, 2021
Merged

debug: add tools for task aware debug #4810

merged 3 commits into from
Nov 10, 2021

Conversation

zhuyanlinzyl
Copy link
Contributor

@zhuyanlinzyl zhuyanlinzyl commented Nov 9, 2021 •
edited
Loading

Summary

Patch 1: add tcb strcut for task-aware debug information
Patch 2: add tcbinfo struct in procfs
Patch 3: add plugin in JLinkGDBServer for nuttx thread aware

Impact

No. Enhance nuttx debug feature.

Testing

When nuttx config with CONFIG_DEBUG_TCBINFO.
We can debug nuttx with nuttx thread aware.

make -f Makefile.host jlink-nuttx.so
JLinkGDBServer -if SWD -device Cortex-M55 -rtos /home/zyl/miwear/nuttx/tools/jlink-nuttx

1 show nuttx threads

  Id   Target Id                                           Frame 
* 2    Thread 1 ([PID:000]Idle Task:0003[PRI:000])         nx_start () at init/nx_start.c:797
  3    Thread 2 ([PID:001]hpwork:0005[PRI:224])            arm_switchcontext (saveregs=0x3c00927c, restoreregs=0x3c00c5ac) at /home/zyl/code/m1ap/nuttx/include/arch/armv8-m/syscall.h:121
  4    Thread 19 ([PID:018]rpmsg-uorb-sens:0005[PRI:100])  arm_switchcontext (saveregs=0x3c015fbc, restoreregs=0x3c00efdc) at /home/zyl/code/m1ap/nuttx/include/arch/armv8-m/syscall.h:121
  5    Thread 4 ([PID:003]bes_main:0005[PRI:101])          arm_switchcontext (saveregs=0x3c00ae1c, restoreregs=0x3c00a2ec) at /home/zyl/code/m1ap/nuttx/include/arch/armv8-m/syscall.h:121
  6    Thread 5 ([PID:004]rptun:0005[PRI:224])             arm_switchcontext (saveregs=0x3c00c5ac, restoreregs=0x3c010fac) at /home/zyl/code/m1ap/nuttx/include/arch/armv8-m/syscall.h:121
  7    Thread 6 ([PID:005]rptun:0005[PRI:224])             arm_switchcontext (saveregs=0x3c00d43c, restoreregs=0x3c012acc) at /home/zyl/code/m1ap/nuttx/include/arch/armv8-m/syscall.h:121
  8    Thread 7 ([PID:006]rptun:0005[PRI:224])             arm_switchcontext (saveregs=0x3c00e29c, restoreregs=0x2000972c <g_idletcb+140>) at /home/zyl/code/m1ap/nuttx/include/arch/armv8-m/syscall.h:121
  9    Thread 8 ([PID:007]init:0005[PRI:100])              arm_switchcontext (saveregs=0x3c00efdc, restoreregs=0x2000972c <g_idletcb+140>) at /home/zyl/code/m1ap/nuttx/include/arch/armv8-m/syscall.h:121
  10   Thread 11 ([PID:010]thread-10:0005[PRI:101])        arm_switchcontext (saveregs=0x3c00a2ec, restoreregs=0x3c00efdc) at /home/zyl/code/m1ap/nuttx/include/arch/armv8-m/syscall.h:121
  11   Thread 13 ([PID:012]kvdbd:0005[PRI:100])            arm_switchcontext (saveregs=0x3c011cfc, restoreregs=0x3c013cdc) at /home/zyl/code/m1ap/nuttx/include/arch/armv8-m/syscall.h:121
  12   Thread 14 ([PID:013]rpmsg-gpio:0005[PRI:224])       arm_switchcontext (saveregs=0x3c012acc, restoreregs=0x3c00efdc) at /home/zyl/code/m1ap/nuttx/include/arch/armv8-m/syscall.h:121
  13   Thread 15 ([PID:014]rpmsg-uorb-audio:0005[PRI:100]) arm_switchcontext (saveregs=0x3c013cdc, restoreregs=0x3c014e2c) at /home/zyl/code/m1ap/nuttx/include/arch/armv8-m/syscall.h:121
  14   Thread 16 ([PID:015]rpmsg-uorb-cp:0005[PRI:100])    arm_switchcontext (saveregs=0x3c014e2c, restoreregs=0x3c015fbc) at /home/zyl/code/m1ap/nuttx/include/arch/armv8-m/syscall.h:121

2 change nuttx thread

(gdb) thread 4
[Switching to thread 4 (Thread 19)]
#0  arm_switchcontext (saveregs=0x3c015fbc, restoreregs=0x3c00efdc) at /home/zyl/code/m1ap/nuttx/include/arch/armv8-m/syscall.h:121
121          return reg0;

3 show task backtrace

(gdb) bt
#0  arm_switchcontext (saveregs=0x3c015fbc, restoreregs=0x3c00efdc) at /home/zyl/code/m1ap/nuttx/include/arch/armv8-m/syscall.h:121
#1  0x2c016ab6 in up_block_task (tcb=tcb@entry=0x3c015f30, task_state=task_state@entry=TSTATE_WAIT_SEM) at armv8-m/arm_blocktask.c:139
#2  0x2c008338 in nxsem_wait (sem=sem@entry=0x3c016b44) at semaphore/sem_wait.c:153
#3  0x2c03d054 in poll_semtake (sem=0x3c016b44) at vfs/fs_poll.c:59
#4  nx_poll (fds=fds@entry=0x3c015ca8, nfds=32, timeout=1006721824) at vfs/fs_poll.c:439
#5  0x2c03d0c0 in poll (fds=fds@entry=0x3c015ca8, nfds=<optimized out>, timeout=<optimized out>) at vfs/fs_poll.c:500
#6  0x2c00927c in ppoll (fds=fds@entry=0x3c015ca8, nfds=738386121, nfds@entry=32, timeout_ts=timeout_ts@entry=0x0, sigmask=sigmask@entry=0x3c016c00) at signal/sig_ppoll.c:122
#7  0x2c02e0c8 in uorb_rpmsg_task (argc=<optimized out>, argv=<optimized out>) at uORB/uORBRpmsg.cpp:404
#8  0x2c00fa42 in nxtask_startup (entrypt=entrypt@entry=0x2c02dfd5 <uorb_rpmsg_task(int, char**)>, argc=<optimized out>, argv=<optimized out>) at sched/task_startup.c:151
#9  0x2c00971a in nxtask_start () at task/task_start.c:130
#10 0x00000000 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

@zhuyanlinzyl
arch:debug: add struct for task aware debug.
dcbf5b3 
When enable DEBUG_TCBINFO config, a global struct will
provide, then debuggers can aware nuttx task infomation.

Signed-off-by: zhuyanlin <[email protected]>
@zhuyanlinzyl
fs:procfs: add tcbinfo procfs interface
a49b3cc 
Add tcbinfo in procfs system.

Signed-off-by: zhuyanlin <[email protected]>
@zhuyanlinzyl
Copy link
Contributor Author

Please ignore the "Mixed case" checkpatch error. As there are SEGGER JLINK plugin API defination.

gustavonihei
gustavonihei reviewed Nov 9, 2021
View reviewed changes
tools/jlink-nuttx.c Outdated Show resolved Hide resolved
@zhuyanlinzyl
tools: add gdbplugin for JLinkGDBServer nuttx thread aware
2bf4e5b 
Use `JLinkGDBServer -rtos libnuttxplugin` to add this plugin.
Then in gdb can use command:
  - `info threads` to show all threads infos
  - `thread (id) ` to switch thread.
  - `bt` to show thread backtrace.

Signed-off-by: zhuyanlin <[email protected]>
@acassis acassis merged commit 407bc04 into apache:master Nov 10, 2021
@zhuyanlinzyl zhuyanlinzyl deleted the jlink branch November 11, 2021 09:31
@fjpanag
Copy link
Contributor

fjpanag commented Nov 13, 2021

This is not building for me.

After enabling CONFIG_DEBUG_TCBINFO I get:

nuttx/fs/procfs/fs_procfstcbinfo.c:227: undefined reference to `g_tcbinfo'

I think that arm_doirq.c is not included in the build.

@xiaoxiang781216
Copy link
Contributor

@fjpanag you need add arm_tcbinfo.c to your SoC Make.defs.

@fjpanag
Copy link
Contributor

fjpanag commented Nov 14, 2021

@xiaoxiang781216 What do you mean SoC Make.defs?

I am trying this on an ARM Cortex M3, and I am using the NuttX arch files.
I have only provided the board part, which does not seem related.

Shouldn't this file be included in the build just like the rest of the arm_*.c files?
The rest seem properly being build by NuttX, I didn't provide any external rules for them.

@fjpanag
Copy link
Contributor

fjpanag commented Nov 14, 2021

I just freshly cloned nuttx and apps.

I tried:

./tools/configure.sh -l stm32f4discovery:nsh
make all

and the build succeeds.

I then enabled DEBUG_TCBINFO and the build fails in the same way.

I guess this does not have to do with my configuration.
Build fails even for the standard configurations, as provided out-of-the-box.

@xiaoxiang781216
Copy link
Contributor

arm_tcbinfo.c is a new addition, so you need add it to this file:
https://github.com/apache/incubator-nuttx/blob/master/arch/arm/src/stm32/Make.defs#L21-L31
just like other common source file

@fjpanag
Copy link
Contributor

fjpanag commented Nov 19, 2021

@xiaoxiang781216 sorry for the late reply.

I added arm_tcbinfo.c to the propposed file, and indeed NuttX builds and runs fine now.
Shouldn't this PR also add arm_tcbinfo.c to the makefiles?

Furthermore, I tried building the JLink lib, but it also fails:

$ make -C tools -f Makefile.host jlink-nuttx
make: Entering directory '/home/a/b/c/nuttx/tools'
/usr/bin/ld: /tmp/ccDMkc8Y.o: relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making a shared object; recompile with -fPIC
collect2: error: ld returned 1 exit status
make: *** [Makefile.host:231: jlink-nuttx.so] Error 1
make: Leaving directory '/home/a/b/c/nuttx/tools'

@xiaoxiang781216
Copy link
Contributor

@xiaoxiang781216 sorry for the late reply.

I added arm_tcbinfo.c to the propposed file, and indeed NuttX builds and runs fine now. Shouldn't this PR also add arm_tcbinfo.c to the makefiles?

could be.

Furthermore, I tried building the JLink lib, but it also fails:

$ make -C tools -f Makefile.host jlink-nuttx
make: Entering directory '/home/a/b/c/nuttx/tools'
/usr/bin/ld: /tmp/ccDMkc8Y.o: relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making a shared object; recompile with -fPIC

What's result if you add -fPIC?

collect2: error: ld returned 1 exit status
make: *** [Makefile.host:231: jlink-nuttx.so] Error 1
make: Leaving directory '/home/a/b/c/nuttx/tools'

Which OS do you compile? @zhuyanlinzyl could you take a look?

@fjpanag
Copy link
Contributor

fjpanag commented Nov 24, 2021

What's result if you add -fPIC?

It builds successfully, I have yet to test it though...

Which OS do you compile? @zhuyanlinzyl could you take a look?

Fedora 34
Linux workstation 5.14.16-201.fc34.x86_64 #1 SMP Wed Nov 3 13:57:29 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

with GCC:

$ gcc --version
gcc (GCC) 11.2.1 20210728 (Red Hat 11.2.1-1)
Copyright (C) 2021 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

@fjpanag
Copy link
Contributor

fjpanag commented Nov 24, 2021

Well, in fact I just tried it.

It crashes with a SEGFAULT:

$ JLinkGDBServer -if swd -device STM32F427VI -endian little -speed 1000 -port 2331 -swoport 2332 -telnetport 2333 -vd -ir -localhostonly 1 -singlerun -strict -timeout 0 -nogui -rtos /path/nuttx/tools/jlink-nuttx.so
SEGGER J-Link GDB Server V7.54c Command Line Version

JLinkARM.dll V7.54c (DLL compiled Sep 20 2021 13:59:21)

Command line: -if swd -device STM32F427VI -endian little -speed 1000 -port 2331 -swoport 2332 -telnetport 2333 -vd -ir -localhostonly 1 -singlerun -strict -timeout 0 -nogui -rtos /path/nuttx/tools/jlink-nuttx.so
-----GDB Server start settings-----
GDBInit file:                  none
GDB Server Listening port:     2331
SWO raw output listening port: 2332
Terminal I/O port:             2333
Accept remote connection:      localhost only
Generate logfile:              off
Verify download:               on
Init regs on start:            on
Silent mode:                   off
Single run mode:               on
Target connection timeout:     0 ms
------J-Link related settings------
J-Link Host interface:         USB
J-Link script:                 none
J-Link settings file:          none
------Target related settings------
Target device:                 STM32F427VI
Target interface:              SWD
Target interface speed:        1000kHz
Target endian:                 little

Connecting to J-Link...
J-Link is connected.
Firmware: J-Link V11 compiled Sep  7 2021 11:22:27
Hardware: V11.00
S/N: ********
Feature(s): GDB
Checking target voltage...
Target voltage: 3.28 V
Listening on TCP/IP port 2331
Connecting to target...
Connected to target
Waiting for GDB connection...Connected to 127.0.0.1
Reading all registers
Read 4 bytes @ address 0x00000000 (Data = 0x10008000)
Read 4 bytes @ address 0xFFFFFFFC (Data = 0x00000000)
Read 4 bytes @ address 0x00000000 (Data = 0x10008000)
Read 4 bytes @ address 0xFFFFFFFC (Data = 0x00000000)
Read 2 bytes @ address 0x00000000 (Data = 0x8000)
Read 2 bytes @ address 0xFFFFFFFE (Data = 0x0000)
Read 2 bytes @ address 0xFFFFFFFC (Data = 0x0000)
Read 2 bytes @ address 0x00000000 (Data = 0x8000)
Read 2 bytes @ address 0xFFFFFFFE (Data = 0x0000)
Read 2 bytes @ address 0xFFFFFFFC (Data = 0x0000)
Read 4 bytes @ address 0x00000000 (Data = 0x10008000)
Read 4 bytes @ address 0xFFFFFFFC (Data = 0x00000000)
Read 4 bytes @ address 0x00000000 (Data = 0x10008000)
Read 4 bytes @ address 0xFFFFFFFC (Data = 0x00000000)
Read 4 bytes @ address 0x00000000 (Data = 0x10008000)
Read 4 bytes @ address 0x00000000 (Data = 0x10008000)
Read 4 bytes @ address 0x00000000 (Data = 0x10008000)
Read 2 bytes @ address 0x00000000 (Data = 0x8000)
Received monitor command: speed 1000
Target interface speed set to 1000 kHz
Received monitor command: clrbp
Received monitor command: reset
Resetting target
Received monitor command: halt
Halting target CPU...
...Target halted (PC = 0x000001B4)
Received monitor command: regs
R0 = 00000000, R1 = 00000000, R2 = 00000000, R3 = 00000000
R4 = 00000000, R5 = 00000000, R6 = 00000000, R7 = 00000000
R8 = 00000000, R9 = 00000000, R10= 00000000, R11= 00000000
R12= 00000000, R13= 10008000, MSP= 10008000, PSP= 00000000
R14(LR) = FFFFFFFF, R15(PC) = 000001B4
XPSR 01000000, APSR 00000000, EPSR 01000000, IPSR 00000000
CFBP 00000000, CONTROL 00, FAULTMASK 00, BASEPRI 00, PRIMASK 00

Security extension regs:
MSP_S  = 00000000, MSPLIM_S  = 00000000
PSP_S  = 00000000, PSPLIM_S  = 00000000
MSP_NS = 10008000, MSPLIM_NS = 00000000
PSP_NS = 00000000, PSPLIM_NS = 00000000
CONTROL_S  00, FAULTMASK_S  00, BASEPRI_S  00, PRIMASK_S  00
CONTROL_NS 00, FAULTMASK_NS 00, BASEPRI_NS 00, PRIMASK_NS 00

Reading all registers
Received monitor command: speed auto
Select auto target interface speed (2000 kHz)
Received monitor command: flash breakpoints 1
Flash breakpoints enabled
Received monitor command: semihosting enable
Semi-hosting enabled (Handle on BKPT)
Received monitor command: semihosting IOClient 1
Semihosting I/O set to TELNET Client
Received monitor command: SWO DisableTarget 0xFFFFFFFF
SWO disabled successfully.
Received monitor command: SWO EnableTarget 0 0 0x1 0
SWO enabled successfully.
Loading RTOS plugin: //path/nuttx/tools/jlink-nuttx.so...
RTOS plugin (API v1.0) loaded successfully
RTOS plugin: Loaded
Received symbol: g_pidhash (0x200009E4)
Received symbol: g_npidhash (0x200009E8)
Received symbol: g_tcbinfo (0x0003A938)
Received symbol: g_readytorun (0x200009B4)
All mandatory symbols successfully loaded.
All mandatory symbols successfully loaded.
Downloading 16240 bytes @ address 0x00000000 - Verified OK
Downloading 16368 bytes @ address 0x00003F70 - Verified OK
Downloading 160 bytes @ address 0x00007F60 - Verified OK
Downloading 4096 bytes @ address 0x00008000 - Verified OK
Downloading 412 bytes @ address 0x00010000 - Verified OK
Downloading 256 bytes @ address 0x00010200 - Verified OK
Downloading 15936 bytes @ address 0x00010300 - Verified OK
Downloading 16016 bytes @ address 0x00014140 - Verified OK
Downloading 16064 bytes @ address 0x00017FD0 - Verified OK
Downloading 16016 bytes @ address 0x0001BE90 - Verified OK
Downloading 16032 bytes @ address 0x0001FD20 - Verified OK
Downloading 16032 bytes @ address 0x00023BC0 - Verified OK
Downloading 16032 bytes @ address 0x00027A60 - Verified OK
Downloading 16016 bytes @ address 0x0002B900 - Verified OK
Downloading 16016 bytes @ address 0x0002F790 - Verified OK
Downloading 16032 bytes @ address 0x00033620 - Verified OK
Downloading 13621 bytes @ address 0x000374C0 - Verified OK
Downloading 8 bytes @ address 0x0003A9F8 - Verified OK
Downloading 716 bytes @ address 0x0003AA00 - Verified OK
Segmentation fault (core dumped)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@gustavonihei gustavonihei gustavonihei left review comments

@Ouss4 Ouss4 Awaiting requested review from Ouss4

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@zhuyanlinzyl @fjpanag @xiaoxiang781216 @gustavonihei @acassis