Move to 2.0 API spec instead of extensions API

[jeffvli]

Preface

I've been following the discussion over on the Symfonium forums and saw that most of the votes went towards the optional extensions. I'd like to see if there is any will for client and server owners to reconsider to move to 2.0.

Rationale

1. Moving to a 2.0 API would be non-breaking for current Subsonic clients: the Subsonic API endpoints can be left as a legacy offering. Client owners do not need to make any changes if they don't want added features from the new standard
2. Extensions API would likely be more difficult to maintain and document: As the extensions would be optional to implement by server owners, clients need to perform checks on each of the endpoints to see what is implemented or not. This may lead to user-experience disparities between servers and also relies on the client owner to have to individually test each server to see if features are implemented uniformly.
3. New API spec won't be limited by existing Subsonic standards:
   • Proper authentication on endpoints
   • Drop support for XML and focus on modern REST standards (with opinionated examples)
     • GET /api/library/albums instead of GET /rest/getAlbumList
     • DELETE /api/playlist/1 instead of GET /rest/deletePlaylist?id=1
   • Able to update all the fields including multi-value fields [API Discussion] Existing single-valued fields that should be multi-valued #8
4. Would be able to create and maintain an OpenAPI specification that all servers and clients can rigidly follow: I saw that @deluan made an attempt at creating one but it was not quite compatible with the existing API.
5. A new modern API may encourage the creation of new clients: It seems some of the concern of not wanting to breaking Subsonic API is due to existing client support. If the API is more tenable to developers, then the chances of new clients appearing increases along with new features.

End

I understand this may be a larger burden on both server and client owners, but I believe it would be worth it especially since these upgrades will likely be the new standard for many years to come.
Let me know your thoughts on the above points and anything else I may have not considered.

Aside: @spl0k also made good arguments towards moving to 2.0.

[Tolriq]

Point 2 is not really true with the way they are defined. By embracing optional yet properly documented API it's easier to adapt to different server needs and possibilities. Not all servers will have the internals to support everything.

Current Subsonic API is already not properly implemented by all servers so requires tweaks and tests. With the extensions it would not be the case.

The main issue if a V2, beside the amount of work and less server following, is that the optional concept still needs to be fully implemented. Not all servers support podcast or music videos or many other details.
In the same vein if we define too rigid things some servers might not be able to follow as not having the data and so requiring major refactoring.

If there was to be a V2 I would fight for it to support differential sync, that require many internal things on server side.

Anyway if for a V2 devs needs to check error codes everywhere to see if API is implemented this is not really better than the extensions.

So yes a V2 would be nice for many reasons, but If we want to move we will need to be pragmatic. Most servers and clients won't have the resources. And if not enough emulsion at the start the spark will die sooner than later.

[jeffvli]

Point 2 is not really true with the way they are defined. By embracing optional yet properly documented API it's easier to adapt to different server needs and possibilities. Not all servers will have the internals to support everything.

Current Subsonic API is already not properly implemented by all servers so requires tweaks and tests. With the extensions it would not be the case.

The main issue if a V2, beside the amount of work and less server following, is that the optional concept still needs to be fully implemented. Not all servers support podcast or music videos or many other details.
In the same vein if we define too rigid things some servers might not be able to follow as not having the data and so requiring major refactoring.
Anyway if for a V2 devs needs to check error codes everywhere to see if API is implemented this is not really better than the extensions.

You're right, I forgot about podcasts and music videos that some servers don't support. Checks would still be needed by a per-feature basis.

If there was to be a V2 I would fight for it to support differential sync, that require many internal things on server side.

Could you explain what differential sync is? I haven't heard about it before.

The main issue if a V2, beside the amount of work and less server following, is that the optional concept still needs to be fully implemented. Not all servers support podcast or music videos or many other details.
In the same vein if we define too rigid things some servers might not be able to follow as not having the data and so requiring major refactoring.

Honestly speaking, I'm not a fan of the current Subsonic API which is my main motivation for wanting servers to move away from it to a new standard (I don't have the capability to easily support the original Subsonic API in my new client which is why it's currently on-hold). I think it could be doable to reorganize the existing Subsonic API functionality into a v2 standard without requiring servers to do large refactoring. For example if servers aren't storing data for multi-valued fields (genres, artists, etc) they could just return an array with a single entry.

I might be a minority in my opinion but I'm also curious what the thoughts are of the server maintainers if they are willing to or not. My opinion may also change once I see the full list of extensions proposals.

[dweymouth]

Could you explain what differential sync is? I haven't heard about it before.

Tolriq could explain better but I believe Symfonium's use case is to request metadata updates for anything that has been added or changed since a given datetime (i.e. the last time it synced)

[Tolriq]

Yes the application is offline first so I have a full copy of the data inside the app for that purpose.
I then need ways to do differential sync, meaning when possible only get changes past a date to get the playcounts and everything related. Including change in metadata.
I also need to know when the database have additions, and have deletion or breaking change that requires a full sync to be run again.

[spl0k]

Anyway if for a V2 devs needs to check error codes everywhere to see if API is implemented this is not really better than the extensions.

Not necessarily. The new API could be designed in a way that allows it to be extended. Kinda the same way as #5, an endpoint or some HTTP header could be used to tell the clients which capabilities the server supports (be it a set of endpoints, added fields on some base objects, or anything else). With that kind of thing the clients won't need to do extra checks as they'll know ahead of time what's available and what's not.

[Tolriq]

@spl0k yes I know was just answering that it's not a differentiator as both will have the same constraints.

We will need optional handling and will have to deal with server differences in both cases.

[dweymouth]

I don't want to opine too strongly one way or the other, but I think a big part of the decision to go V2 or not would come down to the changes we want to make to the existing API behavior, and how many of them require new endpoints vs can be done with tweaks. It's usually easy enough to add things in a backward-compatible way. If we have a lot of things that require new endpoints, then V2 could be the best option.

To that end, here are the existing things I think are broken with the current API:

• Auth
• Artist and Genre being single-valued
• All endpoints that can return a lot of data should support pagination
• Most data-returning endpoints should support sorting the result, not just GetAlbumList endpoints
• Pagination through a single random sort of all albums is impossible
• The create/edit playlist endpoints, and savePlayQueue endpoints should be POST requests, to avoid URL length limitations
• sorting and filtering in the GetAlbumList endpoints are currently mutually exclusive
• the getLyrics endpoint takes a song name instead of an ID as its argument
• the stream endpoint doesn't allow the client to specify multiple formats it can support (this one might actually be easy enough to do in a backward-compatible way)

Any others anyone wants to add?

[dweymouth]

Actually, now that I think about it more, there are some additive things that might be hard to shoehorn into the existing endpoints. E.g. Navidrome is starting to have the concept that a single album can have multiple issues (or editions, or whatever you want to call it). Figuring out how to do this with the current endpoints could be messy and might be best done with a whole new object model. E.g. in Discogs, an artist's discography is a list of master releases, each of which has individual issues of that album as its children.

[Tolriq]

Most providers reports different albums with a release type field to differentiate them.

[dweymouth]

You'd need an edition field, rather than a release type field, to differentiate. And also some field that can be used to relate the different editions of the same album together if they're represented as individual albums in the API object model

[epoupon]

Note: MusicBrainz has ReleaseGroup for that purpose.

[lachlan-00]

That to me is not an album but some kind of parent to the album release.

I don't think many places outside of musicbrainz/discogs group like that and it wouldn't really transfer easily to most subsonic servers I'm aware of.

[dweymouth]

Maybe exposing a ReleaseGroupID field in the album info would work then, similar to MusicBrainz, so it's optional for the server to support it, and optional for the client to do any kind of grouping based on ReleaseGroup. I do think we should support it somehow though. Navidrome is working on it as a feature, and having different editions of the same album is one reason why people self-host instead of just using Spotify et al (which always seem to have the latest remaster as their one version of an album).

I think if the Subsonic extensions/2.0 don't support enough features it will encourage further API fragmentation. (e.g. @jeffvli is currently building the Sonixd successor off of the Navidrome native API, since Subsonic wasn't feature-rich enough)

[lachlan-00]

I would prefer extension now and breaking later.

Clients should have the minimum of disruption to incentivise support for new features.

[dweymouth]

I'm overall in favor of V2 but I suppose I could be onboard with quick wins for v1, but I think we should be really selective of what we'd include in quick fixes, and then move right into V2 planning.

For more intrusive stuff that introduce new notions

This is where I think a lot of the value will be which is why I'm a proponent of V2. I think there is too much broken with the V1 API to take it too much further without somewhat of a complete redesign.

[Tolriq]

@dweymouth in all cases this is a vote that is only for the servers now. We have all made remarks and comments about what we'd like as clients / servers since the start of this.

Now is the time for servers and only servers to decide what they accept and can do now, to start moving on.

This does not matter is 20 clients wants a 2.0 API if no server or only 1 server is willing to do it it will be DOA.

[dweymouth]

Fair enough

[Tolriq]

@gravelld @lachlan-00 @deluan Please see #9 (reply in thread) and vote when you have time so we can start to build this project.

[gravelld]

Voted for extensions for now.