Alert going off while dialog active does not pause speak media player...

[wjennings]

IMPORTANT: Before you create an issue, please take a look at our Issue Reporting Guide.

Briefly summarize your issue:

If a timer goes off while a dialog is active (ie the speak mediaplayer is active), the alert will be told to play a single short alert notification, but the speak media player is not told to pause first. This is not expected, and, since I'm not using the gstreamer interface, my media player acts in an undefined manner.

What is the expected behavior?

I believe, the speak mediaplayer should be paused first, the alert tone played, and, when finished, the speak mediaplayer told to resume. once the speak media player is done, the alert can continue like normal.

What behavior are you observing?

speak media player is not being paused, and, when the alert plays, either:
#1, just the alert plays, but the speak mediaplayer doesn't continue
#2, the speak media player continues to play, but the alert doesn't play
#3, the 2 are played in very short clips, one after the other.
None of those are correct IMO.

Provide the steps to reproduce the issue, if applicable:

"alexa, set a timer for 5 seconds"
<as soon as she says, "starting now", say "alexa, whats the weather like today?"
During the playback, the timer will expire, and the alert media player will be told to play while speak mediaplayer is playing.

Tell us about your environment:

I see this on multiple platforms, but both running Linux, using ffmpeg/libav as the backbone of the mediaplayers.

What version of the AVS Device SDK are you using?

1.8.1

Tell us what hardware you're using:

#1, Intel x64 desktop
#2, aarch64 ARM processor smaller smart assistant device

Tell us about your OS (Type & version):

Linux on both (ubunutu 16.04 on desktop)

Relevant logging:

SpeakMediaPlayer: (DEBUG)11:18:50:884 [src/MyMediaPlayer.cpp:sendPlaybackStarted(),2566]: SourceId 0x2
2018-08-20 15:18:50.884 [ 17] 9 SpeechSynthesizer:onPlaybackStarted:callbackSourceId=2
2018-08-20 15:18:50.885 [ 10] 0 SpeechSynthesizer:executePlaybackStarted
2018-08-20 15:18:50.885 [ 10] 9 SpeechSynthesizer:setCurrentStateLocked:state=PLAYING
...
###########################
Speaking... #
###########################
...
2018-08-20 15:18:51.538 [ 1b] 9 AlertScheduler:onAlertReady:alertToken=amzn1.as-ct.v1.Domain:Application:NotificationsV4#DNID#d2c493c1-8bd5-3c5e-9ac6-7bdc03cc2b5e
2018-08-20 15:18:51.538 [ 1b] 9 AlertScheduler:notifyObserver:alertToken=amzn1.as-ct.v1.Domain:Application:NotificationsV4#DNID#d2c493c1-8bd5-3c5e-9ac6-7bdc03cc2b5e,state=READY,reason=
2018-08-20 15:18:51.538 [ 1c] 9 AlertsCapabilityAgent:onAlertStateChange:alertToken=amzn1.as-ct.v1.Domain:Application:NotificationsV4#DNID#d2c493c1-8bd5-3c5e-9ac6-7bdc03cc2b5e,state=READY,reason=
2018-08-20 15:18:51.538 [ 5] 1 AlertsCapabilityAgent:executeOnAlertStateChange:alertToken=amzn1.as-ct.v1.Domain:Application:NotificationsV4#DNID#d2c493c1-8bd5-3c5e-9ac6-7bdc03cc2b5e,state=READY,reason=
2018-08-20 15:18:51.538 [ 5] 9 AlertsCapabilityAgent:acquireChannel
2018-08-20 15:18:51.538 [ 5] 1 FocusManager:acquireChannel:channelName=Alerts,interface=Alerts
2018-08-20 15:18:51.538 [ 5] 1 AlertsCapabilityAgent:executeNotifyObservers:alertToken=amzn1.as-ct.v1.Domain:Application:NotificationsV4#DNID#d2c493c1-8bd5-3c5e-9ac6-7bdc03cc2b5e,state=READY,reason=
2018-08-20 15:18:51.538 [ 13] 9 AlertsCapabilityAgent:onFocusChanged:focusState=BACKGROUND
2018-08-20 15:18:51.538 [ 13] 5 AudioActivityTracker:notifyOfActivityUpdates
2018-08-20 15:18:51.538 [ 5] 1 AlertsCapabilityAgent:executeOnFocusChanged:focusState=BACKGROUND
2018-08-20 15:18:51.539 [ 5] 9 AlertScheduler:updateFocus:focusState=BACKGROUND
2018-08-20 15:18:51.539 [ f] 5 AudioActivityTracker:executeNotifyOfActivityUpdates
2018-08-20 15:18:51.539 [ 5] 9 AlertScheduler:activateNextAlertLocked
2018-08-20 15:18:51.539 [ 5] 9 Alert:activate
2018-08-20 15:18:51.539 [ 5] 9 Alert:startRenderer
2018-08-20 15:18:51.539 [ 1d] 1 Renderer:executeSetObserver
2018-08-20 15:18:51.540 [ 1d] 1 Renderer:executeStart:urls.size=0,loopCount=0,loopPause (ms)=10000
2018-08-20 15:18:51.540 [ 1d] 9 Renderer:executeStart:m_urls.size=0,m_remainingLoopCount=0,m_loopPause (ms)=10000
2018-08-20 15:18:51.540 [ 1d] 9 Renderer:play
2018-08-20 15:18:51.540 [ 1d] 9 Renderer:shouldPlayDefault
2018-08-20 15:18:51.540 [ 1d] 9 Renderer:shouldMediaPlayerRepeat
...
AlertsMediaPlayer: (DEBUG)11:18:51:540 [src/MyMediaPlayer.cpp:play(),250]: Send Play Command

[visayamv]

Hi @wjennings , the Alert should still be able to play as a short alert sound on the background while Dialog is playing on the foreground. The behavior is described here:

https://developer.amazon.com/docs/alexa-voice-service/interaction-model.html#channels

[rbuckley]

Hi @visayamv, we've chosen to pause background channels instead of attenuating them. This is at the very end of that page: When the Content Channel is backgrounded, this refers to the pausing or attenuation of audio playback.

The SDK should notify us when a channel is moved to the background shouldn't it? Or is it up to us to track foreground vs background channels?

In another scenario, when we are playing back music (the content channel) and then the user begins an Alexa interaction (the dialog channel), the SDK sends a pause event when the content channel moves to the background because the dialog channel is active.

[visayamv]

Hello @rbuckley ,
Only the Content Channel will be pause when backgrounded.
The Alert Channel will still continue playing short alerts on the background

You can see the example in the doc:
Ask Alexa to set a timer for 5 seconds. Once Alexa notifies you that the timer has been set, ask Alexa for the weather forecast. As Alexa provides you with the forecast, the timer should go off as a short alert until Alexa has finished speaking. This is because the Dialog channel is active, which means that the Alerts channel must be in the background. Once Alexa has finished speaking, the Alerts channel moves to the foreground, and a long alert should continue to play until a user stops the timer.

To summarize the interaction of the channels:

1. Content and Dialog: Content (background) is paused, DIalog is foreground
2. Content and Alert: Content (background) is paused, Alert is foreground
3. Dialog and Alert: Alert (background) plays short alerts , DIalog is foreground

The capability agent that is responsible for each channel will be notified of the change in focus (i.e. background/foreground/none) using the "onFocusChanged" interface (for example, Alerts has the AlertsCapabilityAgent::onFocusChanged).

Once it's notified of the change in focus, the capability agent should do the appropriate action based on its focus. For example, when Alerts received a background focus, it will play the short alerts instead of the long alert.

Content is handled by the AudioPlayer capability agent.

[rbuckley]

Hi @visayamv, thanks for the detailed response. So the onFocusChanged interface should be used by our application to track different states and pause, or change audio accordingly? It is not an internal method used by the SDK.

My main point of confusion is who is responsible for tracking the foreground/background distinction and making sure that the interaction summary you laid out is adhered to, is it the SDK or our client application?

[visayamv]

Hi @rbuckley , yes, the component that handles the media player should be doing the action based on its notified focus.

The Activity Focus Manager Library (AFML) (see the README section in https://github.com/alexa/avs-device-sdk) is responsible for determining which component/capability agent gets foreground/background focus.

Each component will request focus from AFML and will notify what focus it will be on.

[kclchan]

I am closing this issue due to inactivity. Please feel free to re-open it if it has been closed in error.