Very slow loading of gallery list due to slow getDirectoryListing() on external storage

[LukasReschke]

Ref owncloud/core#10077 (comment)

Opened here as well to allow people to search for it as well.

[oparoz]

Updated the title, as we simply request a list of files using getDirectoryListing()
https://github.com/owncloud/gallery/blob/master/service/filesservice.php#L48

[oparoz]

File Locking results in slowing down stuff massively owncloud/core#19888

[setnes]

I'm not using external storage or the file locking app, but thumbnails are presented very slowly. Is it the same issue? The thumbnails are already generated. Hitting reload is painful.

[oparoz]

@setnes, if you're on 8.2, you're using the files locking app with the database backend. It's slow and there are bottlenecks. Make sure you install and use Redis.
Of course, there could still be other issues :) If it's really the thumbnails, it could be that they're not served from the cache.

[Niluge-KiWi]

I just upgraded from 8.0 to 8.1 to 8.2, and had the same issue as @setnes: very slow gallery display: chrome debug console showed 2-3 seconds to list a directory with 30 photos, then multiple calls to thumbnails each taking from 2 to 10 seconds, total load time for the first row of thumbnails was 18 seconds.

I enabled a local redis for memchache.locking, and APCu for memcache.local, and now list takes 80ms, and thumbnails 150-400ms, total load time for the first row of thumbnails: 2.4 seconds. Much better!

I'm not sure if this config is optimal for my small personal usecase. Maybe APCu for locking & local memcache is OK, maybe redis for both is better. Do you have any tips on that?

Maybe it should be more visible: the default configuration leads to really poor performance now, and it took me some time to find a solution after the upgrade. (doc.owncloud.org being previously down didn't help for sure...)

[setnes]

Installing Redis certainly helped on my end as well. On Gentoo I did the following.

emerge -av dev-db/redis
emerge -av dev-php/pecl-redis
/etc/init.d/redis start
/etc/init.d/apache2 restart
rc-update add redis default

Then I added some magic text to the owncloud config.php file.

  'memcache.local' => '\\OC\\Memcache\\Redis',
  'filelocking.enabled' => 'true',
  'memcache.distributed' => '\\OC\\Memcache\\Redis',
  'memcache.locking' => '\\OC\\Memcache\\Redis',
  'redis' =>
  array (
    'host' => 'localhost',
    'port' => 6379,
    'timeout' => 0,
    'dbindex' => 0,
  ),

I'm not entirely sure how much of that is really needed in a basic install. I see that config as an example in many places though. Someone who understands the caching might be able to explain each line. :) I'm also not sure if Redis should be used for everything, or just distributed and locking... etc. This is working, but I'm not sure if it's the best recommendation.

[LukasReschke]

I'm not entirely sure how much of that is really needed in a basic install. I see that config as an example in many places though. Someone who understands the caching might be able to explain each line. :) I'm also not sure if Redis should be used for everything, or just distributed and locking... etc. This is working, but I'm not sure if it's the best recommendation.

Happy to help on that.

So, when we talk about caching ownCloud uses different caches for different use-cases.

1. memcache.local: The cache where entries should go that are only relevant for a local server. This is usually best APCu as you may use Redis in combination with different servers. However, if you have only one, Redis is fine here as well. These are often entries such as local paths that may differ on clustered setups.
2. memcache.distributed: Cache where entries should go that are the same even in clustered setups. You can keep this empty to use the same one as memcache.local.
3. memcache.locking: Which cache should be used for file locking. We recommend Redis here as others do NOT guarantee the consistency of the entries. Redis does. (e.g. in Memcached an entry may disappear any time, which would make logging useless)

The other stuff is just for the Redis connection. You can remove the dbindex here. timeout, host and port should however stay.

[oparoz]

then multiple calls to thumbnails each taking from 2 to 10 seconds

@Niluge-KiWi - That's normal the first time you launch the app, if there are missing thumbnails. It should always be much faster after that, but I'd be interested to know if Redis also speeds up this operation.

@setnes - Because of the defaults, you only need to set memcache.local and can safely remove the rest.

[Niluge-KiWi]

More metrics: (all after multiple page refresh)

• filelocking.enabled + memcache.local APCu + memcache.locking Redis:
  • avatar/name/id: 70ms
  • config: 110ms
  • list: 80ms
  • thumbnails: 115-165ms (5 requests)
  • page ready: 0,95s
  • total: 1,95s
• filelocking.enabled + memcache.local Redis + memcache.locking Redis:
  • avatar/name/id: 90ms
  • config: 200ms
  • list: 90ms
  • thumbnails: 130-180ms (5 requests)
  • page ready: 1s
  • total: 2,45s
• filelocking.enabled + memcache.local none + memcache.locking Redis:
  • avatar/name/id: 75ms
  • config: 130ms
  • list: 85ms
  • thumbnails: 130-180ms (5 requests)
  • page ready: 1,05s
  • total: 2,45s
• filelocking.enabled + memcache.local none + memcache.locking none:
  • avatar/name/id: 75ms
  • config: 460ms
  • list: 1,95s
  • thumbnails: 5-14s (5 requests)
  • page ready: 1,05s
  • total: 35s
• filelocking.disabled + memcache.local none + memcache.locking none:
  • avatar/name/id: 80ms
  • config: 130ms
  • list: 75ms
  • thumbnails: 115-150ms (5 requests)
  • page ready: 1,05s
  • total: 1,95s

My setup: 27 photos in the directory, 2 cores Intel(R) Core(TM)2 Duo CPU E6550 @ 2.33GHz, 2GB RAM, HDD.

So, at least for me:

• it's indeed the filelocking which creates the massive performance regression.
• memcache.locking Redis helps a lot: it's exactly the same as with no filelocking in terms of speed.
• memcache.local Redis has a small impact on speed vs APCu: some requests have time doubled, and globally it goes from 2 seconds to 2.5 seconds.

So I'll stay with local APCu & locking Redis: full speed, with protections guaranteed by file locking!

[oparoz]

Excellent, thank you very much for this data. I did experience the slow down on 8.0 when enabling file locking, but thought it had been sorted out on 8.1+, forgetting that I had switched to using Redis.
I'm glad more and more people will adopt it as the technology makes it possible to track changes in real time if we want to.
I still prefer Redis as a general memory cache since it's permanent vs a cache which is wiped at each PHP restart.

[oparoz]

Closing this as this is a core issue which is mostly fixed, at least for people using Redis. There is little the app itself can do to improve the situation.