Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

arch/sim: add sim audio support #2124

Merged
merged 2 commits into from
Nov 4, 2020
Merged

arch/sim: add sim audio support #2124

merged 2 commits into from
Nov 4, 2020

Conversation

GUIDINGLI
Copy link
Contributor

Summary

arch/sim: add sim audio support

Impact

sim audio will call alsa to playback/capture data

Testing

nxplayer
nxrecoder

@acassis
Copy link
Contributor

acassis commented Oct 26, 2020

Very nice!

@masayuki2009
Copy link
Contributor

@GUIDINGLI
Could you please add a defconfig file so that we can try audio on sim?

@btashton
Copy link
Contributor

I probably won't be able to really look at this until the weekend, but if someone wants to review and merge ahead of that I can look at it in master then.

arch/sim/src/Makefile Outdated Show resolved Hide resolved
arch/sim/src/sim/up_audio.c Outdated Show resolved Hide resolved
arch/sim/src/sim/up_initialize.c Show resolved Hide resolved
@acassis
Copy link
Contributor

acassis commented Oct 29, 2020

@GUIDINGLI
Could you please add a defconfig file so that we can try audio on sim?

In fact, this is a good idea to have a sim:audio example in place.

@btashton
Copy link
Contributor

I'll update the test repo to be able to support this when I do my testing. We will need to have the alsa libraries available.

Copy link
Contributor

@xiaoxiang781216 xiaoxiang781216 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@GUIDINGLI please take @acassis and @btashton comment and update the patch.

@btashton
Copy link
Contributor

@xiaoxiang781216 should I go ahead and get the ALSA libraries/headers in the CI images? I can also reserve a name for the sim with an exception for Darwin. That way this can run against CI, I don't see any reason this wont eventually make it in.

@xiaoxiang781216
Copy link
Contributor

Sure we need install
sudo apt-get -y install libpulse-dev libpulse-dev:i386
sudo apt-get -y install libasound2-dev libasound2-dev:i386
sudo apt-get -y install libasound2-plugins libasound2-plugins:i386

@btashton
Copy link
Contributor

Sure we need install
sudo apt-get -y install libpulse-dev libpulse-dev:i386
sudo apt-get -y install libasound2-dev libasound2-dev:i386
sudo apt-get -y install libasound2-plugins libasound2-plugins:i386

I'll do this tomorrow when do a quick verification on my hardware.

@btashton
Copy link
Contributor

@xiaoxiang781216 I went ahead and just added it. Note I only included libasound2-dev libasound2-dev:i386 as it is all that should be required for building. Adding pulseaudio pulls in a lot of deps so I hope that we can avoid that so we only add less than 10MB instead of 200MB+

@GUIDINGLI The config name I reserved is alsa so if you use that the CI should work for you.

N/A

Cause lots of apps who use external library needs big stack size.
e.g. alsa, ffmpeg

Change-Id: I3b46333da9b18d103ea2ea71ed6e81d79a2d1d6c
Signed-off-by: ligd <[email protected]>
@acassis
Copy link
Contributor

acassis commented Nov 3, 2020

@btashton could you please verify if everything were fixed?

@masayuki2009
Copy link
Contributor

@GUIDINGLI @xiaoxiang781216
I tried this PR with ubuntu18.04 x86_64, but nuttx crashed when playback an WAV file via hostfs.
Could you tell me what environment are you using?

Thread 1 "nuttx" received signal SIGSEGV, Segmentation fault.                                                                                                                   
0x00005555555714a2 in file_vioctl (filep=0x10000000d8, req=4100, ap=0x7ffff2c8b520) at vfs/fs_ioctl.c:87                                                                        
87        inode = filep->f_inode;                                                                                                                                                                                                                                                                                                                     
p filep                                                                                                                                                                         
$1 = (struct file *) 0x10000000d8                                                                                                                                               
(gdb) p *filep                                                                                                                                                                  
Cannot access memory at address 0x10000000d8                                                                                                                                    
(gdb) up                                                                                                                                                                        
#1  0x0000555555571682 in nx_vioctl (fd=4, req=4100, ap=0x7ffff2c8b520) at vfs/fs_ioctl.c:173                                                                                   
173           ret = file_vioctl(filep, req, ap);                                                                                                                                
(gdb) where                                                                                                                                                                     
#0  0x00005555555714a2 in file_vioctl (filep=0x10000000d8, req=4100, ap=0x7ffff2c8b520) at vfs/fs_ioctl.c:87                                                                    
#1  0x0000555555571682 in nx_vioctl (fd=4, req=4100, ap=0x7ffff2c8b520) at vfs/fs_ioctl.c:173                                                                                   
#2  0x000055555557197a in ioctl (fd=4, req=4100) at vfs/fs_ioctl.c:259                                                                                                          
#3  0x000055555558f487 in nxplayer_setvolume (pplayer=0x7ffff2bfa0f0, volume=400) at nxplayer.c:1226                                                                            
#4  0x000055555558f164 in nxplayer_playthread (pvarg=0x7ffff2bfa0f0) at nxplayer.c:957                                                                                          
#5  0x0000555555583970 in pthread_start () at pthread/pthread_create.c:192                                                                                                      
#6  0x0000000000000000 in ?? ()     

@GUIDINGLI
Copy link
Contributor Author

GUIDINGLI commented Nov 4, 2020

I tried this PR with ubuntu18.04 x86_64, but nuttx crashed when playback an WAV file via hostfs.
Could you tell me what environment are you using?

Thread 1 "nuttx" received signal SIGSEGV, Segmentation fault.                                                                                                                   
0x00005555555714a2 in file_vioctl (filep=0x10000000d8, req=4100, ap=0x7ffff2c8b520) at vfs/fs_ioctl.c:87                                                                        
87        inode = filep->f_inode;                                                                                                                                                                                                                                                                                                                     
p filep                                                                                                                                                                         
$1 = (struct file *) 0x10000000d8                                                                                                                                               
(gdb) p *filep                                                                                                                                                                  
Cannot access memory at address 0x10000000d8                                                                                                                                    
(gdb) up                                                                                                                                                                        
#1  0x0000555555571682 in nx_vioctl (fd=4, req=4100, ap=0x7ffff2c8b520) at vfs/fs_ioctl.c:173                                                                                   
173           ret = file_vioctl(filep, req, ap);                                                                                                                                
(gdb) where                                                                                                                                                                     
#0  0x00005555555714a2 in file_vioctl (filep=0x10000000d8, req=4100, ap=0x7ffff2c8b520) at vfs/fs_ioctl.c:87                                                                    
#1  0x0000555555571682 in nx_vioctl (fd=4, req=4100, ap=0x7ffff2c8b520) at vfs/fs_ioctl.c:173                                                                                   
#2  0x000055555557197a in ioctl (fd=4, req=4100) at vfs/fs_ioctl.c:259                                                                                                          
#3  0x000055555558f487 in nxplayer_setvolume (pplayer=0x7ffff2bfa0f0, volume=400) at nxplayer.c:1226                                                                            
#4  0x000055555558f164 in nxplayer_playthread (pvarg=0x7ffff2bfa0f0) at nxplayer.c:957                                                                                          
#5  0x0000555555583970 in pthread_start () at pthread/pthread_create.c:192                                                                                                      
#6  0x0000000000000000 in ?? ()     

@masayuki2009

Could you show me your nxplayer cmd line ?

mine :
nsh> mount -t hostfs -o fs=/home/ligd/platform/FFmpeg/st /data
nsh> nxplayer
nxplayer> device /dev/audio/pcm0p
nxplayer> playraw /data/test2.wav 1 16 48000

@masayuki2009
Copy link
Contributor

I tried this PR with ubuntu18.04 x86_64, but nuttx crashed when playback an WAV file via hostfs.
Could you tell me what environment are you using?

Thread 1 "nuttx" received signal SIGSEGV, Segmentation fault.                                                                                                                   
0x00005555555714a2 in file_vioctl (filep=0x10000000d8, req=4100, ap=0x7ffff2c8b520) at vfs/fs_ioctl.c:87                                                                        
87        inode = filep->f_inode;                                                                                                                                                                                                                                                                                                                     
p filep                                                                                                                                                                         
$1 = (struct file *) 0x10000000d8                                                                                                                                               
(gdb) p *filep                                                                                                                                                                  
Cannot access memory at address 0x10000000d8                                                                                                                                    
(gdb) up                                                                                                                                                                        
#1  0x0000555555571682 in nx_vioctl (fd=4, req=4100, ap=0x7ffff2c8b520) at vfs/fs_ioctl.c:173                                                                                   
173           ret = file_vioctl(filep, req, ap);                                                                                                                                
(gdb) where                                                                                                                                                                     
#0  0x00005555555714a2 in file_vioctl (filep=0x10000000d8, req=4100, ap=0x7ffff2c8b520) at vfs/fs_ioctl.c:87                                                                    
#1  0x0000555555571682 in nx_vioctl (fd=4, req=4100, ap=0x7ffff2c8b520) at vfs/fs_ioctl.c:173                                                                                   
#2  0x000055555557197a in ioctl (fd=4, req=4100) at vfs/fs_ioctl.c:259                                                                                                          
#3  0x000055555558f487 in nxplayer_setvolume (pplayer=0x7ffff2bfa0f0, volume=400) at nxplayer.c:1226                                                                            
#4  0x000055555558f164 in nxplayer_playthread (pvarg=0x7ffff2bfa0f0) at nxplayer.c:957                                                                                          
#5  0x0000555555583970 in pthread_start () at pthread/pthread_create.c:192                                                                                                      
#6  0x0000000000000000 in ?? ()     

@masayuki2009

Could you show me your nxplayer cmd line ?

mine :
nsh> mount -t hostfs -o fs=/home/ligd/platform/FFmpeg/st /data
nsh> nxplayer
nxplayer> device /dev/audio/pcm0p
nxplayer> playraw /data/test2.wav 1 16 48000

@GUIDINGLI
The followings are actual sequences.

NuttShell (NSH) NuttX-8.2.0                                                                                                                               
nsh> mount -t hostfs -o fs=/home/ishikawa/nuttx_root /host                                                                                                
mount -t hostfs -o fs=/home/ishikawa/nuttx_root /host                                                                                                     
nsh> ls -l /host                                                                                                                                          
ls -l /host                                                                                                                                               
/host:                                                                                                                                                    
 drwxrwxr-x    4096 ..                                                                                                                                    
 -rw-rw-r--34108636 sample1.wav                                                                                                                           
 drwxrwxr-x    4096 .                                                                                                                                     
nsh> ls -l /dev/audio                                                                                                                                     
ls -l /dev/audio                                                                                                                                          
/dev/audio:                                                                                                                                               
 crw-rw-rw-       0 pcm0c                                                                                                                                 
 crw-rw-rw-       0 pcm0p                                                                                                                                 
nsh> nxplayer                                                                                                                                             
nxplayer                                                                                                                                                  
NxPlayer version 1.05                                                                                                                                     
h for commands, q to exit                                                                                                                                 
                                                                                                                                                          
nxplayer> device /dev/audio/pcm0p                                                                                                                         
device /dev/audio/pcm0p                                                                                                                                   
nxplayer> playraw /host/sample1.wav 2 16 44100  

@GUIDINGLI
Copy link
Contributor Author

@masayuki2009

Have you pick another patch for increase stack size ?

d86d7bc

From your log:
#0 0x00005555555714a2 in file_vioctl (filep=0x10000000d8, req=4100, ap=0x7ffff2c8b520) at vfs/fs_ioctl.c:87

filep address is invaild

in my gdb:

filep is the same area with ap, both 0x7ffff2c...

#0 file_vioctl (filep=0x7ffff2c7a8c8, req=4100, ap=0x7ffff2c9af40) at vfs/fs_ioctl.c:87
#1 0x000055555556fb4c in nx_vioctl (fd=4, req=4100, ap=0x7ffff2c9af40) at vfs/fs_ioctl.c:173
#2 0x000055555556fe44 in ioctl (fd=4, req=4100) at vfs/fs_ioctl.c:259
#3 0x000055555558cc96 in nxplayer_setvolume (pplayer=0x7ffff2bfa0f0, volume=400) at nxplayer.c:1226
#4 0x000055555558c973 in nxplayer_playthread (pvarg=0x7ffff2bfa0f0) at nxplayer.c:957
#5 0x0000555555581e3a in pthread_start () at pthread/pthread_create.c:192
#6 0x0000000000000000 in ?? ()

@masayuki2009
Copy link
Contributor

Have you pick another patch for increase stack size ?

@GUIDINGLI
Yes, because I checked out your repository and branch.

@GUIDINGLI
Copy link
Contributor Author

@masayuki2009

You should also update apps,:

apache/nuttx-apps#461

Squashed commit of the following:

sim audio: call alsa to playback/capture data
sim/audio: correct the format capability
sim/audio: add pause/resume support
sim/audio: add auto stop when meet AUDIO_APB_FINAL
sim/audio: fix abort when set small buffer_size
sim/audio: move sim_audio.c to sim_alsa.c

Change-Id: I8e00ece79159e844ca17fd4c363480b985ee0490
Signed-off-by: ligd <[email protected]>
@masayuki2009
Copy link
Contributor

You should also update apps,:

apache/nuttx-apps#461

@GUIDINGLI
Thanks, it works now!

@masayuki2009
Copy link
Contributor

@xiaoxiang781216 I went ahead and just added it. Note I only included libasound2-dev libasound2-dev:i386 as it is all that should be required for building. Adding pulseaudio pulls in a lot of deps so I hope that we can avoid that so we only add less than 10MB instead of 200MB+

@GUIDINGLI The config name I reserved is alsa so if you use that the CI should work for you.

@btashton @xiaoxiang781216 @GUIDINGLI
I only installed libasound2-dev onto my ubuntu18.04 x86_64 machine.

@acassis
Copy link
Contributor

acassis commented Nov 4, 2020

Hi @masayuki2009 could you please give an Approved from your side? If nobody is against it we can merge it now!

@btashton
Copy link
Contributor

btashton commented Nov 4, 2020

No. My comment has not been addressed.

@xiaoxiang781216
Copy link
Contributor

@btashton could you review again? I think your comment is already addressed in the latest revision.

direction = priv->playback ? SND_PCM_STREAM_PLAYBACK
: SND_PCM_STREAM_CAPTURE;

ret = snd_pcm_open(&pcm, "default", direction, 0);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah I guess this is actually where my comment should be. Don't we want to specify this?

Copy link
Contributor

@xiaoxiang781216 xiaoxiang781216 Nov 4, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you want to change the device name? Ok, it depends on how many pcm instances we want to create. only one playback/capture is created in the current design, "default" should be the best choice in this case. On the other hand, sim_audio_initialize need accept name argument and open the different sound device if we want to simulate multiple sound devices. But I think it's better to create a new patch to change both the alsa driver and board initialization for the mulitple devices. Do you think so?

Copy link
Contributor

@btashton btashton Nov 4, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess I don't care if we limit to one device, but I want to be able to at least specify it via a config. "default" is certainly the correct default choice.

My default microphone is a poor choice in this case (I'm doing a loop back)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In ALSA design, user can modify alsa.conf to map "default" to the different sound hardware. That's why we don't make the device name configurable initially. If you really need this feature, we can add it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just seems trivial to add the two kconfig values here and then not require people to modify their system for testing. This makes it a lot easier to use an alsa loopback device.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you want to still merge fine, it just does not align with other sim hardware where the user can specify how it connects to to the host hardware.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about we merge this patch first? and @GUIDINGLI provide a new patch to support configuration in tomorrow.

@btashton btashton merged commit 73282fe into apache:master Nov 4, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants