opensmile-python in real time

[ThomasJanssoone]

Hello,

I would like to use opensmile in real-time to get prosodic features from a microphone.
I looked at the documentation and previous issue, but I don't understand if that's doable (and if so, how to do it)

Could you please indicate the feasibility of this

Thanks in advance

Best

[chausner-audeering]

There are multiple different ways to use openSMILE for live audio processing.

• Using the inbuilt cPortaudioSource component in openSMILE to do the recording within openSMILE. You will need to build openSMILE from source for this and enable Portaudio support in the build flags. The openSMILE distribution that ships with opensmile-python has not been built with Portaudio support.
• Recording the audio outside of openSMILE (e.g. using PyAudio) and streaming the samples to openSMILE via cExternalAudioSource and opensmile-python.

The latter approach is more complex but also more flexible (e.g. you can programmatically control when to start/stop recording). If you want to go with the first approach, I recommend reading about Portaudio and live recording in the openSMILE documentation to get started. For the second approach, you may consult the documentation on the SMILEapi and cExternalAudioSource.

[fabiodrbarros]

Hello,

this is a really interesting topic but also a bit confusing.... Do you think you could provide more information on how to do it? Maybe a tutorial or something similar.

[chausner-audeering]

Unfortunately, documentation on the SMILEapi is currently very limited. If you want to get started with it, I suggest to take a look at the source code of this Python package (Python source and the openSMILE config files) to get an idea how the feature works.

I can also recommend reading the SMILEapi header file: https://github.com/audeering/opensmile/blob/master/progsrc/include/smileapi/SMILEapi.h and the Python wrapper definition: https://github.com/audeering/opensmile/blob/master/progsrc/smileapi/python/opensmile/SMILEapi.py

[fabiodrbarros]

I think this is fantastic, but also confusing. I think all OpenSmile users will profit from some examples, such as your use of the PyAudio stream, direct from microfone or others. My first question, it is possible? The second is in line with the first, how to do it?!

[aniketzz]

changing the configurations to accept ffmpeg source can solve this. and pass an rtmp link as input.

[hagenw]

I guess it also depends how much lack would be ok for you. You could use the Python package sounddevice for recording and then pass frames to the process_signal() method of opensmile.Smile in Python.