arch/sim: add sim audio support #2124

GUIDINGLI · 2020-10-26T12:56:53Z

Summary

arch/sim: add sim audio support

Impact

sim audio will call alsa to playback/capture data

Testing

nxplayer
nxrecoder

acassis · 2020-10-26T17:20:43Z

Very nice!

masayuki2009 · 2020-10-26T23:09:20Z

@GUIDINGLI
Could you please add a defconfig file so that we can try audio on sim?

btashton · 2020-10-27T04:56:11Z

I probably won't be able to really look at this until the weekend, but if someone wants to review and merge ahead of that I can look at it in master then.

arch/sim/src/Makefile

arch/sim/src/sim/up_audio.c

arch/sim/src/sim/up_initialize.c

acassis · 2020-10-29T19:08:55Z

@GUIDINGLI
Could you please add a defconfig file so that we can try audio on sim?

In fact, this is a good idea to have a sim:audio example in place.

btashton · 2020-10-29T19:56:52Z

I'll update the test repo to be able to support this when I do my testing. We will need to have the alsa libraries available.

xiaoxiang781216

@GUIDINGLI please take @acassis and @btashton comment and update the patch.

btashton · 2020-10-31T06:01:31Z

@xiaoxiang781216 should I go ahead and get the ALSA libraries/headers in the CI images? I can also reserve a name for the sim with an exception for Darwin. That way this can run against CI, I don't see any reason this wont eventually make it in.

xiaoxiang781216 · 2020-10-31T06:18:54Z

Sure we need install
sudo apt-get -y install libpulse-dev libpulse-dev:i386
sudo apt-get -y install libasound2-dev libasound2-dev:i386
sudo apt-get -y install libasound2-plugins libasound2-plugins:i386

btashton · 2020-10-31T06:21:05Z

Sure we need install
sudo apt-get -y install libpulse-dev libpulse-dev:i386
sudo apt-get -y install libasound2-dev libasound2-dev:i386
sudo apt-get -y install libasound2-plugins libasound2-plugins:i386

I'll do this tomorrow when do a quick verification on my hardware.

btashton · 2020-10-31T06:49:49Z

@xiaoxiang781216 I went ahead and just added it. Note I only included libasound2-dev libasound2-dev:i386 as it is all that should be required for building. Adding pulseaudio pulls in a lot of deps so I hope that we can avoid that so we only add less than 10MB instead of 200MB+

@GUIDINGLI The config name I reserved is alsa so if you use that the CI should work for you.

N/A Cause lots of apps who use external library needs big stack size. e.g. alsa, ffmpeg Change-Id: I3b46333da9b18d103ea2ea71ed6e81d79a2d1d6c Signed-off-by: ligd <[email protected]>

acassis · 2020-11-03T12:06:17Z

@btashton could you please verify if everything were fixed?

arch/sim/src/sim/up_internal.h

masayuki2009 · 2020-11-04T02:25:41Z

@GUIDINGLI @xiaoxiang781216
I tried this PR with ubuntu18.04 x86_64, but nuttx crashed when playback an WAV file via hostfs.
Could you tell me what environment are you using?

Thread 1 "nuttx" received signal SIGSEGV, Segmentation fault.                                                                                                                   
0x00005555555714a2 in file_vioctl (filep=0x10000000d8, req=4100, ap=0x7ffff2c8b520) at vfs/fs_ioctl.c:87                                                                        
87        inode = filep->f_inode;                                                                                                                                                                                                                                                                                                                     
p filep                                                                                                                                                                         
$1 = (struct file *) 0x10000000d8                                                                                                                                               
(gdb) p *filep                                                                                                                                                                  
Cannot access memory at address 0x10000000d8                                                                                                                                    
(gdb) up                                                                                                                                                                        
#1  0x0000555555571682 in nx_vioctl (fd=4, req=4100, ap=0x7ffff2c8b520) at vfs/fs_ioctl.c:173                                                                                   
173           ret = file_vioctl(filep, req, ap);                                                                                                                                
(gdb) where                                                                                                                                                                     
#0  0x00005555555714a2 in file_vioctl (filep=0x10000000d8, req=4100, ap=0x7ffff2c8b520) at vfs/fs_ioctl.c:87                                                                    
#1  0x0000555555571682 in nx_vioctl (fd=4, req=4100, ap=0x7ffff2c8b520) at vfs/fs_ioctl.c:173                                                                                   
#2  0x000055555557197a in ioctl (fd=4, req=4100) at vfs/fs_ioctl.c:259                                                                                                          
#3  0x000055555558f487 in nxplayer_setvolume (pplayer=0x7ffff2bfa0f0, volume=400) at nxplayer.c:1226                                                                            
#4  0x000055555558f164 in nxplayer_playthread (pvarg=0x7ffff2bfa0f0) at nxplayer.c:957                                                                                          
#5  0x0000555555583970 in pthread_start () at pthread/pthread_create.c:192                                                                                                      
#6  0x0000000000000000 in ?? ()

GUIDINGLI · 2020-11-04T03:04:05Z

I tried this PR with ubuntu18.04 x86_64, but nuttx crashed when playback an WAV file via hostfs.
Could you tell me what environment are you using?

Thread 1 "nuttx" received signal SIGSEGV, Segmentation fault.                                                                                                                   
0x00005555555714a2 in file_vioctl (filep=0x10000000d8, req=4100, ap=0x7ffff2c8b520) at vfs/fs_ioctl.c:87                                                                        
87        inode = filep->f_inode;                                                                                                                                                                                                                                                                                                                     
p filep                                                                                                                                                                         
$1 = (struct file *) 0x10000000d8                                                                                                                                               
(gdb) p *filep                                                                                                                                                                  
Cannot access memory at address 0x10000000d8                                                                                                                                    
(gdb) up                                                                                                                                                                        
#1  0x0000555555571682 in nx_vioctl (fd=4, req=4100, ap=0x7ffff2c8b520) at vfs/fs_ioctl.c:173                                                                                   
173           ret = file_vioctl(filep, req, ap);                                                                                                                                
(gdb) where                                                                                                                                                                     
#0  0x00005555555714a2 in file_vioctl (filep=0x10000000d8, req=4100, ap=0x7ffff2c8b520) at vfs/fs_ioctl.c:87                                                                    
#1  0x0000555555571682 in nx_vioctl (fd=4, req=4100, ap=0x7ffff2c8b520) at vfs/fs_ioctl.c:173                                                                                   
#2  0x000055555557197a in ioctl (fd=4, req=4100) at vfs/fs_ioctl.c:259                                                                                                          
#3  0x000055555558f487 in nxplayer_setvolume (pplayer=0x7ffff2bfa0f0, volume=400) at nxplayer.c:1226                                                                            
#4  0x000055555558f164 in nxplayer_playthread (pvarg=0x7ffff2bfa0f0) at nxplayer.c:957                                                                                          
#5  0x0000555555583970 in pthread_start () at pthread/pthread_create.c:192                                                                                                      
#6  0x0000000000000000 in ?? ()

@masayuki2009

Could you show me your nxplayer cmd line ?

mine :
nsh> mount -t hostfs -o fs=/home/ligd/platform/FFmpeg/st /data
nsh> nxplayer
nxplayer> device /dev/audio/pcm0p
nxplayer> playraw /data/test2.wav 1 16 48000

masayuki2009 · 2020-11-04T03:35:52Z

I tried this PR with ubuntu18.04 x86_64, but nuttx crashed when playback an WAV file via hostfs.
Could you tell me what environment are you using?

Thread 1 "nuttx" received signal SIGSEGV, Segmentation fault.                                                                                                                   
0x00005555555714a2 in file_vioctl (filep=0x10000000d8, req=4100, ap=0x7ffff2c8b520) at vfs/fs_ioctl.c:87                                                                        
87        inode = filep->f_inode;                                                                                                                                                                                                                                                                                                                     
p filep                                                                                                                                                                         
$1 = (struct file *) 0x10000000d8                                                                                                                                               
(gdb) p *filep                                                                                                                                                                  
Cannot access memory at address 0x10000000d8                                                                                                                                    
(gdb) up                                                                                                                                                                        
#1  0x0000555555571682 in nx_vioctl (fd=4, req=4100, ap=0x7ffff2c8b520) at vfs/fs_ioctl.c:173                                                                                   
173           ret = file_vioctl(filep, req, ap);                                                                                                                                
(gdb) where                                                                                                                                                                     
#0  0x00005555555714a2 in file_vioctl (filep=0x10000000d8, req=4100, ap=0x7ffff2c8b520) at vfs/fs_ioctl.c:87                                                                    
#1  0x0000555555571682 in nx_vioctl (fd=4, req=4100, ap=0x7ffff2c8b520) at vfs/fs_ioctl.c:173                                                                                   
#2  0x000055555557197a in ioctl (fd=4, req=4100) at vfs/fs_ioctl.c:259                                                                                                          
#3  0x000055555558f487 in nxplayer_setvolume (pplayer=0x7ffff2bfa0f0, volume=400) at nxplayer.c:1226                                                                            
#4  0x000055555558f164 in nxplayer_playthread (pvarg=0x7ffff2bfa0f0) at nxplayer.c:957                                                                                          
#5  0x0000555555583970 in pthread_start () at pthread/pthread_create.c:192                                                                                                      
#6  0x0000000000000000 in ?? ()

@masayuki2009

Could you show me your nxplayer cmd line ?

mine :
nsh> mount -t hostfs -o fs=/home/ligd/platform/FFmpeg/st /data
nsh> nxplayer
nxplayer> device /dev/audio/pcm0p
nxplayer> playraw /data/test2.wav 1 16 48000

@GUIDINGLI
The followings are actual sequences.

NuttShell (NSH) NuttX-8.2.0                                                                                                                               
nsh> mount -t hostfs -o fs=/home/ishikawa/nuttx_root /host                                                                                                
mount -t hostfs -o fs=/home/ishikawa/nuttx_root /host                                                                                                     
nsh> ls -l /host                                                                                                                                          
ls -l /host                                                                                                                                               
/host:                                                                                                                                                    
 drwxrwxr-x    4096 ..                                                                                                                                    
 -rw-rw-r--34108636 sample1.wav                                                                                                                           
 drwxrwxr-x    4096 .                                                                                                                                     
nsh> ls -l /dev/audio                                                                                                                                     
ls -l /dev/audio                                                                                                                                          
/dev/audio:                                                                                                                                               
 crw-rw-rw-       0 pcm0c                                                                                                                                 
 crw-rw-rw-       0 pcm0p                                                                                                                                 
nsh> nxplayer                                                                                                                                             
nxplayer                                                                                                                                                  
NxPlayer version 1.05                                                                                                                                     
h for commands, q to exit                                                                                                                                 
                                                                                                                                                          
nxplayer> device /dev/audio/pcm0p                                                                                                                         
device /dev/audio/pcm0p                                                                                                                                   
nxplayer> playraw /host/sample1.wav 2 16 44100

GUIDINGLI · 2020-11-04T04:13:54Z

@masayuki2009

Have you pick another patch for increase stack size ?

d86d7bc

From your log:
#0 0x00005555555714a2 in file_vioctl (filep=0x10000000d8, req=4100, ap=0x7ffff2c8b520) at vfs/fs_ioctl.c:87

filep address is invaild

in my gdb:

filep is the same area with ap, both 0x7ffff2c...

#0 file_vioctl (filep=0x7ffff2c7a8c8, req=4100, ap=0x7ffff2c9af40) at vfs/fs_ioctl.c:87
#1 0x000055555556fb4c in nx_vioctl (fd=4, req=4100, ap=0x7ffff2c9af40) at vfs/fs_ioctl.c:173
#2 0x000055555556fe44 in ioctl (fd=4, req=4100) at vfs/fs_ioctl.c:259
#3 0x000055555558cc96 in nxplayer_setvolume (pplayer=0x7ffff2bfa0f0, volume=400) at nxplayer.c:1226
#4 0x000055555558c973 in nxplayer_playthread (pvarg=0x7ffff2bfa0f0) at nxplayer.c:957
#5 0x0000555555581e3a in pthread_start () at pthread/pthread_create.c:192
#6 0x0000000000000000 in ?? ()

masayuki2009 · 2020-11-04T04:23:08Z

Have you pick another patch for increase stack size ?

@GUIDINGLI
Yes, because I checked out your repository and branch.

GUIDINGLI · 2020-11-04T04:33:34Z

@masayuki2009

You should also update apps,:

apache/nuttx-apps#461

Squashed commit of the following: sim audio: call alsa to playback/capture data sim/audio: correct the format capability sim/audio: add pause/resume support sim/audio: add auto stop when meet AUDIO_APB_FINAL sim/audio: fix abort when set small buffer_size sim/audio: move sim_audio.c to sim_alsa.c Change-Id: I8e00ece79159e844ca17fd4c363480b985ee0490 Signed-off-by: ligd <[email protected]>

masayuki2009 · 2020-11-04T04:55:51Z

You should also update apps,:

apache/nuttx-apps#461

@GUIDINGLI
Thanks, it works now!

masayuki2009 · 2020-11-04T08:49:30Z

@xiaoxiang781216 I went ahead and just added it. Note I only included libasound2-dev libasound2-dev:i386 as it is all that should be required for building. Adding pulseaudio pulls in a lot of deps so I hope that we can avoid that so we only add less than 10MB instead of 200MB+

@GUIDINGLI The config name I reserved is alsa so if you use that the CI should work for you.

@btashton @xiaoxiang781216 @GUIDINGLI
I only installed libasound2-dev onto my ubuntu18.04 x86_64 machine.

acassis · 2020-11-04T12:29:29Z

Hi @masayuki2009 could you please give an Approved from your side? If nobody is against it we can merge it now!

btashton · 2020-11-04T12:33:39Z

No. My comment has not been addressed.

xiaoxiang781216 · 2020-11-04T12:48:44Z

@btashton could you review again? I think your comment is already addressed in the latest revision.

btashton · 2020-11-04T12:57:16Z

arch/sim/src/sim/up_alsa.c

+  direction = priv->playback ? SND_PCM_STREAM_PLAYBACK
+                             : SND_PCM_STREAM_CAPTURE;
+
+  ret = snd_pcm_open(&pcm, "default", direction, 0);


Ah I guess this is actually where my comment should be. Don't we want to specify this?

Do you want to change the device name? Ok, it depends on how many pcm instances we want to create. only one playback/capture is created in the current design, "default" should be the best choice in this case. On the other hand, sim_audio_initialize need accept name argument and open the different sound device if we want to simulate multiple sound devices. But I think it's better to create a new patch to change both the alsa driver and board initialization for the mulitple devices. Do you think so?

I guess I don't care if we limit to one device, but I want to be able to at least specify it via a config. "default" is certainly the correct default choice.

My default microphone is a poor choice in this case (I'm doing a loop back)

In ALSA design, user can modify alsa.conf to map "default" to the different sound hardware. That's why we don't make the device name configurable initially. If you really need this feature, we can add it.

Just seems trivial to add the two kconfig values here and then not require people to modify their system for testing. This makes it a lot easier to use an alsa loopback device.

If you want to still merge fine, it just does not align with other sim hardware where the user can specify how it connects to to the host hardware.

How about we merge this patch first? and @GUIDINGLI provide a new patch to support configuration in tomorrow.

acassis requested review from btashton, xiaoxiang781216 and masayuki2009 October 26, 2020 21:20

btashton reviewed Oct 27, 2020

View reviewed changes

arch/sim/src/Makefile Outdated Show resolved Hide resolved

arch/sim/src/sim/up_audio.c Outdated Show resolved Hide resolved

arch/sim/src/sim/up_initialize.c Show resolved Hide resolved

xiaoxiang781216 reviewed Oct 31, 2020

View reviewed changes

btashton mentioned this pull request Oct 31, 2020

Add ALSA support in CI apache/nuttx-testing#77

Merged

GUIDINGLI force-pushed the mine3 branch 3 times, most recently from 1dac91b to ba24caf Compare November 3, 2020 06:51

sim: increase default stack size to 64K on SIM

d86d7bc

N/A Cause lots of apps who use external library needs big stack size. e.g. alsa, ffmpeg Change-Id: I3b46333da9b18d103ea2ea71ed6e81d79a2d1d6c Signed-off-by: ligd <[email protected]>

GUIDINGLI force-pushed the mine3 branch from ba24caf to 3011540 Compare November 3, 2020 06:54

xiaoxiang781216 reviewed Nov 3, 2020

View reviewed changes

arch/sim/src/sim/up_internal.h Outdated Show resolved Hide resolved

GUIDINGLI mentioned this pull request Nov 4, 2020

nxplayer & nxrecorder: make main thread configurable, set to default apache/nuttx-apps#461

Merged

GUIDINGLI force-pushed the mine3 branch from 3011540 to 07055b7 Compare November 4, 2020 04:36

btashton reviewed Nov 4, 2020

View reviewed changes

btashton merged commit 73282fe into apache:master Nov 4, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

arch/sim: add sim audio support #2124

arch/sim: add sim audio support #2124

GUIDINGLI commented Oct 26, 2020

acassis commented Oct 26, 2020

masayuki2009 commented Oct 26, 2020

btashton commented Oct 27, 2020

acassis commented Oct 29, 2020

btashton commented Oct 29, 2020

xiaoxiang781216 left a comment

btashton commented Oct 31, 2020

xiaoxiang781216 commented Oct 31, 2020

btashton commented Oct 31, 2020

btashton commented Oct 31, 2020

acassis commented Nov 3, 2020

masayuki2009 commented Nov 4, 2020

GUIDINGLI commented Nov 4, 2020 •

edited

Loading

masayuki2009 commented Nov 4, 2020

GUIDINGLI commented Nov 4, 2020

masayuki2009 commented Nov 4, 2020

GUIDINGLI commented Nov 4, 2020

masayuki2009 commented Nov 4, 2020

masayuki2009 commented Nov 4, 2020

acassis commented Nov 4, 2020

btashton commented Nov 4, 2020

xiaoxiang781216 commented Nov 4, 2020

btashton Nov 4, 2020

xiaoxiang781216 Nov 4, 2020 •

edited

Loading

btashton Nov 4, 2020 •

edited

Loading

xiaoxiang781216 Nov 4, 2020

btashton Nov 4, 2020

btashton Nov 4, 2020

xiaoxiang781216 Nov 4, 2020

arch/sim: add sim audio support #2124

arch/sim: add sim audio support #2124

Conversation

GUIDINGLI commented Oct 26, 2020

Summary

Impact

Testing

acassis commented Oct 26, 2020

masayuki2009 commented Oct 26, 2020

btashton commented Oct 27, 2020

acassis commented Oct 29, 2020

btashton commented Oct 29, 2020

xiaoxiang781216 left a comment

Choose a reason for hiding this comment

btashton commented Oct 31, 2020

xiaoxiang781216 commented Oct 31, 2020

btashton commented Oct 31, 2020

btashton commented Oct 31, 2020

acassis commented Nov 3, 2020

masayuki2009 commented Nov 4, 2020

GUIDINGLI commented Nov 4, 2020 • edited Loading

masayuki2009 commented Nov 4, 2020

GUIDINGLI commented Nov 4, 2020

masayuki2009 commented Nov 4, 2020

GUIDINGLI commented Nov 4, 2020

masayuki2009 commented Nov 4, 2020

masayuki2009 commented Nov 4, 2020

acassis commented Nov 4, 2020

btashton commented Nov 4, 2020

xiaoxiang781216 commented Nov 4, 2020

btashton Nov 4, 2020

Choose a reason for hiding this comment

xiaoxiang781216 Nov 4, 2020 • edited Loading

Choose a reason for hiding this comment

btashton Nov 4, 2020 • edited Loading

Choose a reason for hiding this comment

xiaoxiang781216 Nov 4, 2020

Choose a reason for hiding this comment

btashton Nov 4, 2020

Choose a reason for hiding this comment

btashton Nov 4, 2020

Choose a reason for hiding this comment

xiaoxiang781216 Nov 4, 2020

Choose a reason for hiding this comment

GUIDINGLI commented Nov 4, 2020 •

edited

Loading

xiaoxiang781216 Nov 4, 2020 •

edited

Loading

btashton Nov 4, 2020 •

edited

Loading