Music app keeps hanging and is basically unusable.

[memorex1]

Steps to reproduce

1.Open the music app, wait a few seconds.
2.
3.My collection on owncloud 7k songs big.

Expected behaviour

Tell us what should happen
Listening to astonishing music.

Actual behaviour

Tell us what happens instead
The loading symbol appears, after a few secs the page is unresponsive. Chrome asks to close the webpage. If you wait long enough you can play a song, but every input leads to an unresponsive page.

Server configuration

Operating system:
ubuntu 16.04
Web server:
apache 2
Database:
MariaDB
PHP version:
7.0
ownCloud version: (see ownCloud admin page)
9.04
Updated from an older ownCloud or fresh install:
updated
Where did you install ownCloud from:
from the tarball on the owncloud website
Signing status (ownCloud 9.0 and above):

Login as admin user into your ownCloud and access
http://example.com/index.php/settings/integrity/failed
paste the results here.
`No errors have been found.

List of activated apps:

If you have access to your command line run e.g.:
sudo -u www-data php occ app:list
from within your ownCloud installation folder

The content of config/config.php:

If you have access to your command line run e.g.:
sudo -u www-data php occ config:list system
from within your ownCloud installation folder

jonathan@37-97-187-174:~$ sudo -u www-data php occ config:list system
PHP Warning: PHP Startup: Unable to load dynamic library '/usr/lib/php/20151012/enchant.so' - /usr/lib/php/20151012/enchant.so: cannot open shared object file: No such file or directory in Unknown on line 0
PHP Warning: PHP Startup: Unable to load dynamic library '/usr/lib/php/20151012/gmp.so' - /usr/lib/php/20151012/gmp.so: cannot open shared object file: No such file or directory in Unknown on line 0
PHP Warning: PHP Startup: Unable to load dynamic library '/usr/lib/php/20151012/imap.so' - /usr/lib/php/20151012/imap.so: cannot open shared object file: No such file or directory in Unknown on line 0
PHP Warning: PHP Startup: Unable to load dynamic library '/usr/lib/php/20151012/interbase.so' - /usr/lib/php/20151012/interbase.so: cannot open shared object file: No such file or directory in Unknown on line 0
PHP Warning: PHP Startup: Unable to load dynamic library '/usr/lib/php/20151012/ldap.so' - /usr/lib/php/20151012/ldap.so: cannot open shared object file: No such file or directory in Unknown on line 0
PHP Warning: PHP Startup: Unable to load dynamic library '/usr/lib/php/20151012/odbc.so' - /usr/lib/php/20151012/odbc.so: cannot open shared object file: No such file or directory in Unknown on line 0
PHP Warning: PHP Startup: Unable to load dynamic library '/usr/lib/php/20151012/pdo_firebird.so' - /usr/lib/php/20151012/pdo_firebird.so: cannot open shared object file: No such file or directory in Unknown on line 0
PHP Warning: PHP Startup: Unable to load dynamic library '/usr/lib/php/20151012/pdo_odbc.so' - /usr/lib/php/20151012/pdo_odbc.so: cannot open shared object file: No such file or directory in Unknown on line 0
PHP Warning: PHP Startup: Unable to load dynamic library '/usr/lib/php/20151012/pdo_pgsql.so' - /usr/lib/php/20151012/pdo_pgsql.so: cannot open shared object file: No such file or directory in Unknown on line 0
PHP Warning: PHP Startup: Unable to load dynamic library '/usr/lib/php/20151012/pdo_sqlite.so' - /usr/lib/php/20151012/pdo_sqlite.so: cannot open shared object file: No such file or directory in Unknown on line 0
PHP Warning: PHP Startup: Unable to load dynamic library '/usr/lib/php/20151012/pgsql.so' - /usr/lib/php/20151012/pgsql.so: cannot open shared object file: No such file or directory in Unknown on line 0
PHP Warning: PHP Startup: Unable to load dynamic library '/usr/lib/php/20151012/pspell.so' - /usr/lib/php/20151012/pspell.so: cannot open shared object file: No such file or directory in Unknown on line 0
PHP Warning: PHP Startup: Unable to load dynamic library '/usr/lib/php/20151012/recode.so' - /usr/lib/php/20151012/recode.so: cannot open shared object file: No such file or directory in Unknown on line 0
PHP Warning: PHP Startup: Unable to load dynamic library '/usr/lib/php/20151012/snmp.so' - /usr/lib/php/20151012/snmp.so: cannot open shared object file: No such file or directory in Unknown on line 0
PHP Warning: PHP Startup: Unable to load dynamic library '/usr/lib/php/20151012/soap.so' - /usr/lib/php/20151012/soap.so: cannot open shared object file: No such file or directory in Unknown on line 0
PHP Warning: PHP Startup: Unable to load dynamic library '/usr/lib/php/20151012/sqlite3.so' - /usr/lib/php/20151012/sqlite3.so: cannot open shared object file: No such file or directory in Unknown on line 0
PHP Warning: PHP Startup: Unable to load dynamic library '/usr/lib/php/20151012/tidy.so' - /usr/lib/php/20151012/tidy.so: cannot open shared object file: No such file or directory in Unknown on line 0
PHP Warning: PHP Startup: Unable to load dynamic library '/usr/lib/php/20151012/xmlrpc.so' - /usr/lib/php/20151012/xmlrpc.so: cannot open shared object file: No such file or directory in Unknown on line 0

or

Insert your config.php content here
(Without the database password, passwordsalt and secret)

'octymipzp2bt', 'passwordsalt' => '', 'secret' => '', 'trusted_domains' => array ( 0 => 'cloud2.ofnie.eu', 1 => 'cloud.ofnie.eu', ), 'datadirectory' => '/owncloudData/data', 'overwrite.cli.url' => 'http://cloud.ofnie.eu', 'dbtype' => 'mysql', 'version' => '9.0.4.1', 'dbname' => 'ownclouddb', 'dbhost' => 'localhost', 'dbtableprefix' => 'oc_', 'dbuser' => 'memorex', 'dbpassword' => '', 'logtimezone' => 'UTC', 'installed' => true, 'mail_smtpmode' => 'smtp', 'mail_smtpsecure' => 'tls', 'mail_smtpauthtype' => 'LOGIN', 'mail_from_address' => '', 'mail_domain' => 'gmail.com', 'mail_smtpauth' => 1, 'mail_smtphost' => 'smtp.gmail.com', 'mail_smtpport' => '587', 'mail_smtpname' => '@gmail.com', 'mail_smtppassword' => '', 'loglevel' => 3, 'theme' => '', 'maintenance' => false, 'htaccess.RewriteBase' => '/', 'memcache.local' => '\OC\Memcache\APCu', 'appstore.experimental.enabled' => true, ); **Are you using external storage, if yes which one:** local/smb/sftp/... No **Are you using encryption:** yes/no Yes **Are you using an external user-backend, if yes which one:** LDAP/ActiveDirectory/Webdav/... No ### Client configuration **Browser:** Chrome **Operating system:** Win7 ### Logs #### Web server error log Insert your webserver log here ```[Mon Aug 08 06:25:07.900535 2016] [mpm_prefork:notice] [pid 4926] AH00163: Apache/2.4.18 (Ubuntu) OpenSSL/1.0.2g-f$ [Mon Aug 08 06:25:07.900570 2016] [core:notice] [pid 4926] AH00094: Command line: '/usr/sbin/apache2' [Sun Aug 07 06:25:07.914045 2016] [mpm_prefork:notice] [pid 4926] AH00163: Apache/2.4.18 (Ubuntu) OpenSSL/1.0.2g-f$ [Sun Aug 07 06:25:07.914138 2016] [core:notice] [pid 4926] AH00094: Command line: '/usr/sbin/apache2' [Mon Aug 08 06:25:07.159556 2016] [mpm_prefork:notice] [pid 4926] AH00171: Graceful restart requested, doing resta$ ownCloud log (data/owncloud.log) Insert your ownCloud log here ``You can find it at: "https://cloud.ofnie.eu/index.php/s/GTSQZ0XiQ06rcnV" password = 'logfile' #### Browser log Insert your browser log here, this could for example include: a) The javascript console log b) The network log c) ...

[chris001]

Have experienced the exact same behavior from the Music app, with 6100 songs on server. For both ownCloud versions 6, 7, 8 and 9, and nextCloud 11 current version. I think the Music app needs to be optimized for any size music collection, even large ones, 10,000 , 100,000, 1 million songs. As a developer I can take a good guess and say that the Music app is probably trying to keep all information in memory, e.g. song names and full paths and song lengths and related data, in PHP memory, and not caching it to disk after calculating it, and running out of memory and just using memory and CPU inefficiently by recalculating always the same data over and over. This is what it's behaving like, anyway. Haven't looked at the code closely enough to be sure. Comments from developers?

[paulijar]

Caching the JSON describing the music collection could help on the page load times but it is not the main problem here. The fact that the web browser is unresponsive tells us that the bottle neck is not the PHP running in the back-end but the javascript running in the front-end. The front-end uses the angularjs framework and I believe that the sheer number of html elements on the page becomes more than what angularjs can handle. The number of elements becomes so high because of the design of the Music app where tracks for all available albums are visible at the same time on the same page.

It might help on the unresponsiveness issue if the elements were being added to the page bit-by-bit instead of adding the all at once. Even then, the user would probably experience some slowness in the operation of the Music app.

Btw, there were substantial performance improvements in the Music app versions 0.3.12 and 0.3.13. With these, the app now works quite nicely on collection of 2k songs. I haven't tried with any larger collections but 6k is probably still more than the app can handle.

[chris001]

Good point, if angularjs is creating browser ui elements in javascript for every single track at the same time, that is not scalable. What are you going to do with 100,000 audio files, and 10,000 albums? It should lazy load, only build ui elements for the albums which are visible on the screen right now. Also, have an alternate to the "detail view" which it has now. An "album list view" which shows one album per line, to make it faster to find the album you're looking for. And a "song list view" with one song per line, grouped by album. Both lazy loading.

[paulijar]

#359 is another report about the same performance issue.

I've been lately working to improve the performance on large music collections. My current working copy is already quite usable on Chrome with 6k songs and I assume that even 10k would be fine. Other browsers are slower. Making the user experience tolerable with 100k songs would require larger architectural changes and I'm not planning to do those.

[chris001]

What larger architectural changes would be required to make the user experience tolerable with 100k songs?

[paulijar]

@chris001 One thing is the overall concept how the fetching of data from back-end to front-end is implemented. Currently, the back-end creates one huge json response containing all the data about the whole music library. The larger the collection is, the longer it takes to create this json. On my Raspberry Pi server, creating the json for some 7000 tracks takes around 30 seconds. I assume that the time taken grows more than linearly with the number of tracks. With 100k tracks, it could take e.g. 10 minutes and no user would wait that long. I have already implemented back-end-side caching for this json, meaning that usually the user doesn't have to wait for the recreation but sometimes she does. To survive the 100k track collection, the logic should be such that front-end would first fetch only data for artists and albums. The data about individual tracks should be fetched only when needed.

Another thing is the number of element bindings watched by angular. As described earlier, all the UI elements composing the albums view are currently alive simultaneously. Some of these elements have to be polled by angular to react to dynamic changes. Ideally, only the visible elements would exist and new elements would be created dynamically when the user scrolls around. I just can't figure out any easy way to implement this. There are several ready-made "virtual scrolling" libraries for angularjs but it seems to me that all of those require that all the listed elements have the same pre-defined height, and this isn't the case with the Music app. Another option are the "infinite scrolling" libraries. Using them, the result would be something similar to how the ownCloud files app works: the scroll-bar reflects only the content which is currently available and new content is loaded only when the user scrolls to bottom. Personally, I think that the user experience in the files app sucks big time and I wouldn't like to see the Music app to behave like that.

In the end of the day, the question is how many users with 100k tracks there are. My best guess is that almost all users have less than 10k tracks. We don't want to hinder the user experience for 99 % of the users just to be able to serve the 1 %.

You can follow the state of the performance improvement work from PR #564.

[chris001]

Maybe the app could offer the 2 user-selectable modes for viewing.
When the user collection is greater than 10k tracks, it defaults to the infinite-scrolling mode.
Less than or equal to 10k, it defaults to the single page view of all artists albums and tracks.

[paulijar]

@chris001 Well, that would be possible. The downside is that this would basically be another view to maintain and test, making it rather expensive choice in the long run. Another, equally expensive option could be to implement the grid view UI style as requested in #245, as it would be much better suited for large collections. On the other hand, there already is the Audio Player app (https://github.com/Rello/audioplayer) offering that kind of UI and I'm not sure if it would serve any purpose to duplicate their UX to Music app.

[paulijar]

Couple of more related or duplicate issues: #90, #240

[paulijar]

The PR #564 improving the performance especially with large collections was merged yesterday. If you are willing to test with development version, I would be interested to hear how it works now with large real-life music collections and what is the practical collection size limit for usability. I would recommend using Chrome browser with large music collections.

[paulijar]

The Music app version 0.4.1 containing #564 has been released to ownCloud marketplace and Nextcloud app store. There's always room for improvement on performance, but IMHO the app is no longer "basically unusable" with music collection of 7k songs as the opening post states. Hence, I'm closing this issue.