MediaLibraryService.MediaLibrarySession::notifyChildrenChanged doesn't update Android Auto Browse

[fourofspades]

Version

Media3 1.1.0

More version details

I have an issue when a MediaItem is added locally on the device, I call MediaLibraryService.MediaLibrarySession::notifyChildrenChanged with the updated library, however the browser in Android Auto isn't updated (the Queue is however updated).

Wondering if there are issues in this area, or if this is not the correct API to call?

Devices that reproduce the issue

Pixel 7 with Sony Android Auto head unit
Emulated Pixel 6 and Android Auto head unit emulator
I believe all other devices.

Devices that do not reproduce the issue

None

Reproducible in the demo app?

Not tested

Reproduction steps

Connect phone to Android Auto head unit
Add an item to the phone's media database
Call the MediaLibraryService.MediaLibrarySession::notifyChildrenChanged API
open the Browser on Android Auto
Notice new stream is not displayed.

Expected result

New stream should be displayed.

Actual result

New stream is not displayed.

Media

N/A

Bug Report

• You will email the zip file produced by adb bugreport to [email protected] after filing this issue.

[ChernyshovYuriy]

Yes, I was about to ask the same. Within MediaLibraryService extension class, what is a way to update media browser, similar way notifyChildrenChanged worked for MediaBrowserService.

[rohitjoins]

Hi @fourofspades,

From the documentation of MediaLibraryService : If the browser isn't subscribing to the parent, nothing will happen. Did you make sure the browser is subscribed?

Also it would easier for us to debug if you can try and reproduce the problem on our demo-session app. Thanks!

[fourofspades]

Done some more work on this, and I'm not exactly sure what fixed this, however I am now getting the browser updating when the phone's library updates.

I had already implemented onSubscribe, my fix was to manually call it again when the local collection changes, taking care not to create a recursive calling loop (I bundled a flag in the params to not refresh the library, as my library call into the library would also do that).

[fourofspades]

See above. It's now working as expected.

[ChernyshovYuriy]

I am sorry for the late response. I think my issue a bit different. What I meant is if there is a class extends MediaLibraryService and internal collection of media items changed what is a way to ask framework to call MediaLibrarySession.Callback: onGetChildren, etc ...
In case of MediaBrowserService it was notifyChildrenChanged.
I am doing Automotive implementation of the service.

[tonihei]

@ChernyshovYuriy Have you tried using the method with the same name?

[ChernyshovYuriy]

Yes, and it doesn't work to me. The use case is simple, I need to call onGetChildren or onGetLibraryRoot to update current browse tree on Automotive. Invoking mSession.notifyChildrenChanged doesn't trigger any callbacks on MediaLibrarySession.Callback.
Unfortunately, your sample app doesnt have automotive variant to test on.

[ChernyshovYuriy]

Just a note from debugger. The callback ended up here, in MediaLibrarySessionImpl.java (from media3 framework):

public void notifyChildrenChanged(
      String parentId, int itemCount, @Nullable LibraryParams params) {
    dispatchRemoteControllerTaskWithoutReturn(
        (callback, seq) -> {
          if (isSubscribed(callback, parentId)) {
            callback.onChildrenChanged(seq, parentId, itemCount, params);
          }
        });
  }

isSubscribed returns false, so execution ending there

[tonihei]

This sounds like the same issue that is described in #561 (comment): The caller is not subscribed, but manually faking a subscription makes the code work again.

@marcbaechinger Have you seen this issue before or are you aware of any pitfalls with Android Auto subscriptions?

[ChernyshovYuriy]

Thank you for re-considering the issue. On another hand, mSession.notifySearchResultChanged leads to onGetSearchResult, works perfectly.
Friendly reminder, this is not Android Auto, this is Automotive implementation of the MediaLibraryService

[ChernyshovYuriy]

To give a more clarity on this case, here is a use case:

• user is on Now Playing View and selected Favorite button (mark this song as Favorite)
• when exit from Now Playing View, not necessary user is on Favorite browse view, but may be on that view as well

So how can I tell to framework to call children to update?

• to update current children if user on Favorites browse view
• to call update of Root children to add / remove Favorites browse view
• to update Favorites browse view once user is back from any other browse view
• etc ...

As I mentioned, when doing extension to MediaBrowseService, call notifyChildenChanged(browse_id) did exactly what it needed to deal with all use cases I provided.

Unfortunately, can't do that with a new design...

[marcbaechinger]

@ChernyshovYuriy

Hey Yuriy! Nice to meet you again. It's been a while! :)

mSession.notifyChildrenChanged

That's the correct API to use.

I tested your case with the session demo app and this seems to work I'm afraid. So not sure how helpful the following is for you:

I changed PlaybackService.onGetChildren of the session demo app as follows:

var categoryChanged = false

override fun onGetChildren(
  session: MediaLibrarySession,
  browser: ControllerInfo,
  parentId: String,
  page: Int,
  pageSize: Int,
  params: LibraryParams?
): ListenableFuture<LibraryResult<ImmutableList<MediaItem>>> {
  Log.d("subscribe", "onGetChildren called for $parentId")
  val children =
    MediaItemTree.getChildren(parentId)
      ?: return Futures.immediateFuture(
        LibraryResult.ofError(LibraryResult.RESULT_ERROR_BAD_VALUE)
      )

  if (!categoryChanged && parentId == "[artist]The Kyoto Connection") {
    categoryChanged = true
    Log.d("subscribe", "post delayed")
    Handler(Looper.myLooper()!!).postDelayed({
      Log.d("subscribe", "trigger notifyChildrenChanged for $parentId")
      session.notifyChildrenChanged(browser, parentId, children.size, params)
    }, 10_000)
  }

  return Futures.immediateFuture(LibraryResult.ofItemList(children, params))
}

Unfortunately, your sample app doesnt have automotive variant to test on.

This worked for me by just deploying the session demo app from Android Studio to the Automotive emulator. Then select another app (like News), and then select the demo app again to make it load the library root.

This change adds an if statement that is executed once when the children of the parentId with value [artist]The Kyoto Connection are requested. I then post a delayed Runnable that after 10 seconds calls session.notifyChildrenChanged(browser, parentId, children.size, params).

I added a bit more logs in other classes, but essentially I see at the end that 10 seconds after the category is requested (19:31:40.609), the call to notifyChildrenChanged makes Automotive request the category again:

// starting the session demo app in Automtoive emulator
19:31:15.516 16112-16112 subscribe           D  onGetChildren called for [rootID]
19:31:15.534 16112-16112 subscribe           D  MediaLibrarySessionImpl.notifyChildrenChanged: [rootID] is subscribed. Calling onChildrenChanged
19:31:15.534 16112-16112 subscribe           D  MediaLibraryServiceLegacyStub.onChildrenChanged: calling MediaBrowserServiceCompat.notifyChildrenChanged for [rootID]
19:31:15.537 16112-16112 subscribe           D  onGetChildren called for [artistID]
19:31:15.555 16112-16112 subscribe           D  MediaLibrarySessionImpl.notifyChildrenChanged: [artistID] is subscribed. Calling onChildrenChanged
19:31:15.555 16112-16112 subscribe           D  MediaLibraryServiceLegacyStub.onChildrenChanged: calling MediaBrowserServiceCompat.notifyChildrenChanged for [artistID]
19:31:15.556 16112-16112 subscribe           D  onGetChildren called for [rootID]
19:31:15.566 16112-16112 subscribe           D  onGetChildren called for [artistID]

// manually select the category `[artist]The Kyoto Connection` that then post the `Runnable`
19:31:30.599 16112-16112 subscribe           D  onGetChildren called for [artist]The Kyoto Connection
19:31:30.599 16112-16112 subscribe           D  post delayed
19:31:30.624 16112-16112 subscribe           D  MediaLibrarySessionImpl.notifyChildrenChanged: [artist]The Kyoto Connection is subscribed. Calling onChildrenChanged
19:31:30.624 16112-16112 subscribe           D  MediaLibraryServiceLegacyStub.onChildrenChanged: calling MediaBrowserServiceCompat.notifyChildrenChanged for [artist]The Kyoto Connection
19:31:30.627 16112-16112 subscribe           D  onGetChildren called for [artist]The Kyoto Connection

// ten seconds later `notifyChildrenChanged` is called and Automotive comes again to get the
// children of the category again
19:31:40.609 16112-16112 subscribe           D  trigger notifyChildrenChanged for [artist]The Kyoto Connection
19:31:40.610 16112-16112 subscribe           D  MediaLibrarySessionImpl.notifyChildrenChanged: [artist]The Kyoto Connection is subscribed. Calling onChildrenChanged
19:31:40.610 16112-16112 subscribe           D  MediaLibraryServiceLegacyStub.onChildrenChanged: calling MediaBrowserServiceCompat.notifyChildrenChanged for [artist]The Kyoto Connection
19:31:40.610 16112-16112 subscribe           D  onGetChildren called for [artist]The Kyoto Connection

I'm testing with Automotive emulator API 29 that I randomly created with the device manager.

isSubscribed returns false, so execution ending there

I noticed that Automotive is only subscribed to a single parentId at the time, which is the categroy that is currently displayed. When the user navigates away from that parentId, the subscribtion is removed. What I'm trying to say is that the expectation has to be that Automotive is only subscribed to the categroy that is currently duisplayed by the UI. I'm not sure from your report whether this is the case when you test, so wanted to clarify.

When testing with https://github.com/googlesamples/android-media-controller I see similar behaviour.

Specifically, with both, Automotive and the controller test app I see that MediaLirbarySessionImpl.subscriptions has the subscription for the given parentId stored as expected.

So this looks fine to me from the library side. The only thing that comes to mind is whether your service overrides subscribe/unsubscribe and returns a result with RESULT_SUCCESS? Because if not, then the subscription is not registered.

[ChernyshovYuriy]

Hello @marcbaechinger ! It's really me :-) I'm thrilled that you remember me, and I'm delighted to be chatting with you again.
I've been working on refactoring the logic for the MediaLibraryService, but I didn't quite get it right on my first attempt.
Let me take another look at the logic I've refactored so far.

[ChernyshovYuriy]

I created a very simple Automotive music app:
https://github.com/ChernyshovYuriy/MediaLibraryServiceApp/tree/master

Tested it on Automotive emulator 1024p landscape API32 12L ""Sv2.

I tried to apply the refresh logic at:
https://github.com/ChernyshovYuriy/MediaLibraryServiceApp/blob/master/shared/src/main/java/com/yuriy/medialibraryserviceapp/shared/MusicService.kt#L115

But no onGetChildren invoked.
Please help me to understand new media3 concept, do I miss a point in a new design?

I use Android Studio Hedgehog | 2023.1.1 Canary 15
Build #AI-231.9225.16.2311.10572941, built on July 27, 2023

[marcbaechinger]

Thanks for the test fixture.

When I override the default implementation of MediaLibraryService.Callback.onSubscribe() in MusicService of the project above I see Automotive requesting the category MEDIA_ID_RADIOS every 10 seconds.

I added the following code which makes it work, you may want to review and change it a bit to fit your requirements:

override fun onSubscribe(
    session: MediaLibrarySession,
    browser: MediaSession.ControllerInfo,
    parentId: String,
    params: LibraryParams?
): ListenableFuture<LibraryResult<Void>> {
    return Futures.immediateFuture(LibraryResult.ofVoid())
}

The reason why this is required is that Callback.onSubscribe needs to return a LibraryResult with RESULT_SUCCESS. Else the caller in not added to the subscription list as you pointed out above. So the caller will never be notified.

Aside: Overriding onUnsubscribe() isn't mandatory to make it work, but an app may want to override this to do the book keeping of currently subscribed clients.

So I think for you app you can just add the snippet above to override Callback.onSubscribe().

For the Media3 library I'd actually think we should make the default implementation of Callback.onSubscribe return a RESULT_SUCCESS by default. Given it just works, that should not harm. At least we should document, that by without overriding the callback it doesn't work.

Can you check whether overriding onSubscribe makes you app work?
Thanks for reporting Yuriy and sorry for this not being at least documented properly.

[ChernyshovYuriy]

Hey @marcbaechinger ,

No need to apologize whatsoever. Being a software engineer myself, I totally get where you're coming from. This is exactly why we have this issue tracking system in place - to improve things.

And regarding your suggestion - it's spot on! I've tested it with the example code and it's definitely working. Big thanks for your assistance, really appreciate it!

[tonihei]

@marcbaechinger Assuming this is a common requirement for Android Auto implementations, would it make sense to maintain this Map<String, List<ControllerInfo>> (keyed by parentId) inside the library instead? And then add a convenience getter along the lines of getSubscribersForParentId(String)?

[yuroyami]

The only thing that comes to mind is whether your service overrides subscribe/unsubscribe and returns a result with RESULT_SUCCESS? Because if not, then the subscription is not registered.

I had a problem where notifyChildrenChanged did nothing even after subscription, but @marcbaechinger's comment pointed me in the right direction. I should have overridden onSubscribe and made it return Futures.immediateFuture(LibraryResult.ofVoid()). This way, the subscription seemingly got registered as it should be and everything worked great. Thank you so much for your assistance.

By the way, I think the Javadocs should put a lot more emphasis on this, because it's pretty easy to miss, or maybe make the onSubscribe return a RESULT_SUCCESS by default ?

[marcbaechinger]

or maybe make the onSubscribe return a RESULT_SUCCESS by default ?

I agree that would be a solution that removes some friction and confusion.

I have a commit that does exactly this. However, when I do return RESULT_SUCCESS from onSubscribe unconditionally then a client app can subscribe a browser for a parentId that does not exist. Actually, a client app that is accessing your app can subscribe for as many parentIds (like millions) until your app has an OutOfMemory exception and crashes.

I can of course limit this, so that we for instance only allow a browser to subscribe to lets say 20 parent ids. This would prevent such an OOM attack but looks a bit arbitrary and can be confusing as well. I'm inclined to implement such an upper limit of lets say 20 parentId a browser can subscribe to. Once this limit is reached, no more subscription would be allowed. The issue/downside is that this upper limit probably also would be in place if an app actually overrides onSubscribe or carefully implements access controls for apps in onConnect.

What are the sentiments of developers having such a productive app? How would you implement onSubscribe yourself? Would you check whether a given parentId exist in your database/catalogue, or would you just unconditionally return RESULT_SUCCESS (I obviously do not recommend this :)) ? Would you do so even after having learned about a malicious app could do what I explained above? Do you check the signature of Android Automotive in onConnect to make sure no other app than Automotive can connect?

Thoughts?

[yuroyami]

I now grasp the reasoning behind the decision to not have onSubscribe automatically return RESULT_SUCCESS. Lemme offer my perspective on this matter, based on my observations of media apps (note that this is subjective) :

I believe that most media apps don't require verification of callers. This is primarily because media content, like music, is generally not sensitive (with an emphasis on "most" here). Additionally, it's rare, if not non-existent, to encounter malicious apps that would attempt to exploit a media app's browser service solely to access its data. Many music apps either play local files or enable remote music playback, such as the likes of Spotify.

In my experience, there's often no need to validate the identity of callers. The need for caller identity verification is quite niche and, therefore, negligible. I don't see any compelling reasons, other than those you've mentioned, for onSubscribe not to return RESULT_SUCCESS by default. Implementing a default limit of 20 subscribers seems practical, and in reality, this limit would likely never be reached. To mitigate any potential confusion, it would be beneficial to include this information in the Javadocs; this way any confusion can be addressed, and users will have clear guidance on the limit.

Let alone the fact that this may potentially fix the (probably) broken Android Auto support, where Android Auto is not refreshing the media library (#644) since it's requiring all browsers to explicitly subscribe in order to receive callbacks. This however will require some way to add the android auto browser passively to the subscriber list.

[yuroyami]

I consider the idea of adopting the logic contained in UAMP's PackageValidator.kt officially under media3 in the sole case where the caller is one of those listed in the allowed_media_callers.xml to be completely valid in order to register the android auto browser as a passive subscriber.

It may even be enough to check the package name (one of the two: "com.google.android.projection.gearhead" or "com.android.auto"). However, it's very easy to mimic a package name, so as I mentioned, the package validator class does the job of detecting the authenticity of the caller.

I am not sure how things work under the hood as i've taken very quick and non-thorough looks on the source code, but I believe it's one of the two:

• Either to check the identity of the caller (whether it is android auto or not) and register it to the subscriber list when the latter requests the library root. I believe this would work best.

• Or when the user calls notifyChildrenChanged, where media3 would need to iterate through the controllers and see if any of them meets the android auto criteria. I know that it already has a list of subscribers that it iterates through, but this can be done at most once, just so that the Android Auto controller does not get left out.

However, since I am not all aware of how media3 works internally, I cannot think of a valid solution. I may be looking at the wrong direction, or maybe the Android Auto list of items not being refreshed is not related at all to this.

[marcbaechinger]

The commit above provides a default implementation of Callback.onSubscribe(parentId). It calls Callback.onGetItem(parentId) and expects the app to return a browsable item with RESULT_SUCCESS. If this is the case the browser is automatically subscribed and notifyChildrenChanged(browser, ...) is called once immediately.

An app can later call MediaLibrarySession.getSubscribedControllers(parentId) to get all controllers subscribed to the parentId and call notifyChildrenChanged(controller, parentId, ...) to inform about further changes.

This behavior can be changed by overriding MediaLibrarySession.Callback.onSubscribe().

This will be included in 1.2.0 for which we will ship the next alpha soon. I'm closing this issue. Please open a new issue if required.

[fdfmobileapps]

Hi @marcbaechinger

Thanks for all the work with Media3, it is a great improvement on the disparate methods required in the previous versions of ExoPlayer, I am certainly noticing the benefits in tidying up and flexibility already.

Right now I am attempting to resolve an issue, which is identical in nature to the one raised earlier regarding onGetChildren not being called for Android Automotive after notifying of a change to children via:

mediaLibrarySession.notifyChildrenChanged(controller,"favourites",allFavouriteItems.count(),LibraryParams.Builder().build())

The onGetChildren for parent Id "favourites" is not being called again. Is called once when the service starts but is not called upon subsequent requests via the notifyChildrenChanged. Hence, if the list of items updates the Automotive UI is not updating children. Which leaves an invalid list presented to the user.

I have attempted to update to the alpha version mentioned hoping to use the onGetItem method to confirm subscription, but before proceeding further I have immediately noticed that my list of customcommands is no longer displaying correctly. These worked perfectly in 1.1.1, but with they look like this:

In 1.2.0 Alpha:

In 1.1.1:

The icons work in both cases - they trigger their callbacks but can't proceed testing out the alpha until I understand why these are now incorrect. So essentially:

1. The icons are not appearing correctly in the alpha version,

and also noticed

2. if(controller.packageName.contains("gearhead", true) || controller.packageName.contains("auto", true)) is no longer a reliable means of testing for the calling package to determine which custom commands to avail. I have tested by removing the if statement and simply pass through the customCommands, but the packageName details appear to have changed in some way. I have not invested much time looking into this point at this stage.

I am eager to resolve the issue around the subscriptions for my parentIds, but not wanting to delve further into the alpha if I am already immediately noticing an issue with customCommands.

Many thanks for your advice in advance!

[marcbaechinger]

Thanks for your report. Lets leave the onChildrenChanged topic to #644.

Regarding custom commands with AA/AAOS. In the main branch, we've updated the demo app, enabled Android Auto support for the session demo and added an Automotive OS module. Both apps use the same DemoMediaLibrarySessionCallback. With this custom commands work for me on the emulator for Android Auto and Automotive OS (and the mobile notification).

• in onConnect, the demo callback sets the custom layout of media notification controller, Auto and Automotive when these connect.

• in onCustomCommand when the command is received, the custom layout of the media notification controller is updated. Please see the JavaDoc of getMediaNotificationControllerInfo() for why in more detail.

In short, set the available commands and custom layout of the media notification controller to set the actions and custom actions of the platform/media1 session where Android Auto and Automotive OS read from.

Note: MediaSession.isAutomotiveController/isAutoCompanionController is not in alpha-02 but will land in beta-01. If you use the same package name comparison as in the source code this works as well. :)

EDIT: Sorry, I forgot to ask can you give me more details on how you configure the custom commands/layout in onConnect/onCustomCommand?

[okmand]

Just another comment regarding best practice for other readers who want to support controllers subscribing to a category:

1. Override MediaLibraryService.Callback.onSubscribe() to maintain a Map<String, List<ControllerInfo>> keyed by the parentId. This gives you all the subscribed controllers/browsers for a given parentId. The callback needs to return aRESULT_SUCCESS for instance with Futures.immediateFuture(LibraryResult.ofVoid()) else Media3 will not put the caller into the list of subscribed controllers.
2. When the children of a given parentId changes, consult the map from above and call session.notifyChildrenChanged(browser, parentId, childrenCount, params) for each of the browsers in the map to inform selected subscribed clients of your choice about the change in the library.
3. Override MediaLibraryService.Callback.onSubscribe() to remove the ControllerInfo from the map you are maintaining.

Hello @marcbaechinger,

Could you explain, why should we use Map<String, List<ControllerInfo>> instead of Map<String, ControllerInfo>? I mean, I have only one implementation of MediaLibraryService and expect 1 call of onSubscribe and notifyChildrenChanged functions. In which cases I may expect 2 different ControllerInfo for one parentId?

The second question: How can I even have several versions of ControllerInfo for different parentId? It is possible to save only one ControllerInfo and use it always when calling notifyChildrenChanged?

[marcbaechinger]

The library can't know how many controller/browser connect or subscribe and other apps may expect more than one call to subscribe.

It can be that 2 apps are connected to your session each with their browser and then both subscribe for the same parentId. So for each parentId there can be multiple controller infos subscribed to the same parentId. So I figure it must be a List<ControllerInfo>. Please let me know if I have misunderstood your question.

The second question: How can I even have several versions of ControllerInfo for different parentId? It is possible to save only one ControllerInfo and use it always when calling notifyChildrenChanged?

I'm not sure I understand the question. If you have multiple controllers connected, then you need to notify multiple controllers. You may want to notify only certain controllers for whatever reasons, so you can select from the list of ControllerInfo. If you want to notify all controllers that are subscribed to a certain parentId you can use notifyChildrenChanged without the controller info argument.

[okmand]

Hi @marcbaechinger,

Thanks a lot for a quick answer! Now I use a list of ControllerInfo and notify all controllers. It solved my issue, thanks. I tried to use notifyChildrenChanged without the controller info argument, but it didn't work for a some reason. But now I use list of controllers and generally it's okay for me

I have also the following question:
Why we need to set itemCount parameter in notifyChildren function and how this field is used? What happens if I will always set 0 as a parameter or just the wrong number for a some reason, for example by mistake.

Thanks in advance!