Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

qemu SMP support #85

Closed
thomascp opened this issue Jan 20, 2019 · 22 comments
Closed

qemu SMP support #85

thomascp opened this issue Jan 20, 2019 · 22 comments

Comments

@thomascp
Copy link

Hi Mr/Ms,

There are five machine supported by riscv qemu.
sifive_e
sifive_u
spike_v1.10
spike_v1.9.1
virt

And I found that only virt support cpu number larger than 1 which is SMP. But I met some strange issues when using SMP cpu=4 of virt mahine mode.
For example:
One hart send software interrupt to another hart, the other hart can not receive the interrupt.
But when I ctrl+c to stop the gdb which is attached to the qemu, then just continue by typing "c" command and made no any modification. Then it can receive the software interrupt.
It is weird! Seem ctrl+c trigger the other hart to receive the interrupt.

Yet I don't know whether this is caused by my codes.
And I see the Makefile, seems linux doesn't boot with SMP, only with one hart.
I only want to confirm first, whether the SMP with virt machine mode is supported?
Thanks!

BR,
Peng

@jim-wilson
Copy link

If you do a git log in the freedom-u-sdk/riscv-qemu you will see that it is from April 2018, i.e. it is old. Also, if you do a git log in the linux tree, you will see that it is also old, it is a circa 2017 pre-upstream copy with various patches backported. freedom-u-sdk is a useful tutorial for how to get started with RISC-V linux, but it isn't for real work.

You should get up-to-date copies of both the linux kernel and qemu. You can use either the upstream trees where the master sources live, or the riscv/riscv-* trees which are still maintained for now.

Yes, smp works. I do "make -j4" builds all of the time on fedora running on qemu, and I see the CPU utilitization for qemu go over 300% in top, which means all 4 threads are doing work.

@thomascp
Copy link
Author

Hi Wilson,

Thanks! I will try this.

BR,
Peng

@jim-wilson
Copy link

If you want a real linux distro, Debian, Fedora, and OpenSuse have working ports that can boot on qemu. Also there is a OpenEmbedded/Yocto port in riscv/meta-riscv if you want something that you can configure and build yourself. There are also other linux ports, and *BSD ports, and RTOS ports, etc.

@thomascp
Copy link
Author

Sorry, should I call you Jim instead of Wilson?
Thanks!
I had tried the newest riscv-qemu branch, the issue still appears.
It seems the bug is in my codes, not linux, I just wrote some codes for fun.
It is weird, only ctrl+c in gdb can trigger the other hart to receive the software interrupt.
Any suggestion?
Thanks!

@jim-wilson
Copy link

An American would call me Jim, but Wilson works too. I'm not an OS level programmer. I don't know how that kind of stuff works.

@thomascp
Copy link
Author

Thanks Jim!
Another question, my gdb seems old too, when I want to show the special register, for example"sie",
it told me unavailable.
So, can I use riscv/riscv-binutils-gdb? Is this good enough to solve this problem?

@jim-wilson
Copy link

You need a gdb with XML register set support, and a qemu with XML register set support, I've posted patches for qemu both upstream and to riscv-qemu. I need to ping the upstream patches. I'm not sure about the riscv-qemu patches, they may or may not have gone in.

XML register set support was added to FSF gdb in November I think, and then riscv-gnu-toolchain was modified to use the FSF gdb port. If you check out riscv-gnu-toolchain you should get the right gdb.

binutils and gdb share a common development tree, but have separate release branches. if you check out riscv-binutils-gdb you get the binutils release branch, which probably has a broken gdb. You need the fsf-gdb-with-sim branch for gdb. The fact that we need two different release branches checked out is why riscv-gnu-toolchain has riscv-binutils and riscv-gdb trees that are both from riscv-binutils-gdb.

@thomascp
Copy link
Author

I had checkout fsf-gdb-with-sim branch from riscv/riscv-binutils-gdb.
Compile:
./configure --prefix=/newdisk/rvqemu/riscv-binutils-gdb/build/ --target=riscv64-unknown-linux-gnu
make && make install
But, when connecting the qemu, it shows:
(gdb) target remote 127.0.0.1:12345
Remote debugging using 127.0.0.1:12345
warning: No executable has been specified and target does not support
determining executable automatically. Try using the "file" command.

Any mistake steps? Thanks!

@jim-wilson
Copy link

Did you try using the file command like the error message suggests?

@thomascp
Copy link
Author

Yes, it works fine. But it doesn't support "target remote" commands.

@jim-wilson
Copy link

You can use file with target remote. You must provide a file to gdb, otherwise it has no idea what program you are debugging on the remote target. Normally you would provide the file to gdb on the gdb command line.

@thomascp
Copy link
Author

I don't really understand you, let me explain more.
My environment is:

  1. qemu runs with gdb server "-S -gdb tcp::12345".
  2. gdb to connect to qemu.

With the freedom-u-sdk built gdb, I can use the below commands to connect:
"target remote 127.0.0.1:12345"
Actually, I can even work with no symbol file, because my codes are mostly assembly codes, so I can use
"display /i $pc" to show the codes if the gdb connected to the qemu successfully.

Add symbol-file only makes me to see the source file.
But now, new built gdb can not connect to qemu, I don't understand "file" commands can help this.

@jim-wilson
Copy link

I am having trouble understanding you also. If I don't have enough info to reproduce your problem, then I may not be able to help you. Usually when you get the error message saying that there is no program, it is user error not specifying a program and the fix is to use the file command as the error message says.

You need a gdb with XML register set support, and a qemu with XML register set support. You apparently have the right gdb. Do you have the right qemu? I've submitted patches for riscv-qemu and upstream qemu, but don't know the status of them. See for instance
riscvarchive/riscv-qemu#182
for riscv-qemu, and for upstream qemu see
http://lists.nongnu.org/archive/html/qemu-devel/2018-12/msg06187.html

@thomascp
Copy link
Author

  1. I had get the newest version for qemu and gdb
    qemu from https://git.qemu.org/git/qemu.git , branch is master, commit is "Date: Fri Jan 18 16:56:15 2019 +0000"
    gdb from https://github.com/riscv/riscv-binutils-gdb.git, branch is fsf-gdb-with-sim, commit is "Date: Tue Nov 27 12:22:33 2018 -0800".

  2. I used the remote debug method.
    Use this command to run riscv and setup gdb server.
    qemu-system-riscv64 -nographic -machine virt -smp 4 -kernel /disk/riscv/freedom-u-sdk/work/riscv-pk/bbl -S -gdb tcp::12345
    Use this command to connet to gdb server and debug.
    /newdisk/rvqemu/riscv-binutils-gdb/build/bin/riscv-gdb
    target remote 127.0.0.1

You doesn't need to add file or symbols in this method, because gdb are used to connect to the remote gdb server which is provided by qemu.

  1. I used these commands to build gdb, is this correct?
    ./configure --prefix=/newdisk/rvqemu/riscv-binutils-gdb/build/ --target=riscv64-unknown-linux-gnu
    make && make install

@thomascp
Copy link
Author

thomascp commented Jan 22, 2019

Hi Jim,

  1. This is a good gdb log:
    riscv64-unknown-linux-gnu-gdb
    GNU gdb (GDB) 8.0.50.20170724-git
    Copyright (C) 2017 Free Software Foundation, Inc.
    License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html
    This is free software: you are free to change and redistribute it.
    There is NO WARRANTY, to the extent permitted by law. Type "show copying"
    and "show warranty" for details.
    This GDB was configured as "--host=x86_64-pc-linux-gnu --target=riscv64-unknown-linux-gnu".
    Type "show configuration" for configuration details.
    For bug reporting instructions, please see:
    http://www.gnu.org/software/gdb/bugs/.
    Find the GDB manual and other documentation resources online at:
    http://www.gnu.org/software/gdb/documentation/.
    For help, type "help".
    Type "apropos word" to search for commands related to "word".
    (gdb)
    (gdb)
    (gdb) target remote 127.0.0.1:12345
    Remote debugging using 127.0.0.1:12345
    warning: No executable has been specified and target does not support
    determining executable automatically. Try using the "file" command.
    0x0000000000001000 in ?? ()

  2. This is the newest gdb log:
    riscv64-unknown-linux-gnu-gdb
    GNU gdb (GDB) 8.2.50.20181127-git
    Copyright (C) 2018 Free Software Foundation, Inc.
    License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html
    This is free software: you are free to change and redistribute it.
    There is NO WARRANTY, to the extent permitted by law.
    Type "show copying" and "show warranty" for details.
    This GDB was configured as "--host=x86_64-pc-linux-gnu --target=riscv64-unknown-linux-gnu".
    Type "show configuration" for configuration details.
    For bug reporting instructions, please see:
    http://www.gnu.org/software/gdb/bugs/.
    Find the GDB manual and other documentation resources online at:
    http://www.gnu.org/software/gdb/documentation/.

For help, type "help".
Type "apropos word" to search for commands related to "word".
(gdb) target remote 127.0.0.1:12345
Remote debugging using 127.0.0.1:12345
warning: No executable has been specified and target does not support
determining executable automatically. Try using the "file" command.
Remote 'g' packet reply is too long (expected 264 bytes, got 520 bytes): 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
(gdb)

The new gdb doesn't connect to qemu successfully. I think this is the issue.
"Remote 'g' packet reply is too long (expected 264 bytes, got 520 bytes)"
@jim-wilson

@jim-wilson
Copy link

You have the wrong qemu. You need a qemu with gdb-xml/riscv* files. Please keep in mind that the RISC-V architecture is new, and not all toolchain development work is done yet. The gdb xml support was added in Oct/Nov, and then in Nov/Dec I did the qemu xml support, but my qemu patches have not been accepted yet.

So you either need to check out my riscv-qemu github branch that has the qemu xml support. Or you need to check out upstream qemu and add my 5 patch set to it. I gave pointers to both in a previous message.

@thomascp
Copy link
Author

Hi Jim,
It works fine now! Thanks a lot for your help!
BR,
Peng

@thomascp
Copy link
Author

By the way, with new tools,my SMP are OK now! Thanks, Jim!

@thomascp
Copy link
Author

@jim-wilson
I had done some jobs, Thank you again for your great help on building and debugging environment.
This is my stuff, but unfortunately it can not be open sourced due to vxWorks license issue.
https://github.com/thomascp/VxWorks7-RISC-V

@franzflasch
Copy link

franzflasch commented Oct 16, 2019

Just tested this with upstream riscv32-none-elf-gdb and upstream qemu-system-riscv32, I get the same error as mentioned above:

GNU gdb (GDB) 8.3
Copyright (C) 2019 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "--host=x86_64-pc-linux-gnu --target=riscv32-none-elf".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from qemu_compiled_files/lui_qemu.elf...
(No debugging symbols found in qemu_compiled_files/lui_qemu.elf)
Remote debugging using localhost:1234
Remote 'g' packet reply is too long (expected 132 bytes, got 388 bytes): 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
end pos 541065324
No registers.
$> qemu-system-riscv32 --version
QEMU emulator version 4.1.50 (v4.1.0-1571-g69b81893bc-dirty)
Copyright (c) 2003-2019 Fabrice Bellard and the QEMU Project developers
$> riscv32-none-elf-gdb --version
GNU gdb (GDB) 8.3
Copyright (C) 2019 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

I built qemu and the riscv32 toolchain from scratch.

Anything I missed here?

@jim-wilson
Copy link

This looks like an arch disagreement between gdb and qemu. The 388 is 32 * 4 + 32 * 8 + 4, which means you have 32-bit integer registers and 64-bit FP registers. The 132 is 32 * 4 + 4 which means you have 32-bit integer registers and no FP registers. So qemu and gdb have different architecture defaults. Gdb defaults from the binary you are trying to run. If you loaded an elf file, it should get arch defaults from there. I don't know what qemu does offhand, but if you are feeding it a raw file with no ELF headers, then it is probably falling back on some default that may be incorrect for your use case. There is probably some way to pass arch info to qemu, but I don't know it off hand. I mostly use the user-mode qemu which has ELF headers to read. Maybe if you specify a target board that has the right CPU arch on it?

The disagreement seems curious though as both gdb and qemu should be using xml register file descriptions so they should be negotiating the number and size of registers when gdb connects to the target. You can see this if you use "set debug remote 1" before the "target remote:1234" command.

You didn't provide enough info for me to reproduce. You didn't give the gdb command, the qemu command, or the example binary. This appears to work for me when I try it.

@franzflasch
Copy link

I've rebuilt it with debian buster and now it works. With the new build I also got this message in gdb:

warning: Can not parse XML target description; XML support was disabled at compile time

However gdb worked regardless of the warning.

I've now also built it with libexpat and now the warning is gone.

Anyway, thanks for your help.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants