massive/strange db updates after upgrade to 8.1.0

[thomas955]

Steps to reproduce

1. Cant reproduce it by hand. Seems to be that some shares are "dirty".
   If an user hits some of this shares, owncloud starts to make tons of db updates.

Expected behaviour

Owncloud should not allocate all of the memory, spawn hundreds of mysqld's and should not use all cpu to make some INSERTS / UPDATES.

Actual behaviour

Sometimes (nearly every day of the week), owncloud starts to make tons of updates in their db. At first there are only a few updates but later on,
all cpu's / ram are in use. If i do not restart my apache later there is no memory left, the vm starts to hang.

What you can see in the logfile, i attached a part of one line, is that this leads to a locked db if i dont do anything. This line also contains one of the db queuries that owncloud wanted to do at this time.

What i figured out:

I have some very strange db entries in oc_appconfig -> appid: files_sharing:
There i have around 30 entries where configkey = some int between 0 - 300 with values like timestamps. (e.g. 1438090689.0903, 1437992409.7579).

The query looks like this: UPDATE oc_appconfig SET configvalue = '1438605612.4871' WHERE appid = 'files_sharing' AND configkey = '21'

All times the problem occured, the dbms started to make tons of updates into this rows and this never stoped until i restarted my apache.

Server configuration

Operating system:
Ubuntu 12.04

Web server:
Server version: Apache/2.4.12 (Ubuntu)

Database:
mysql Ver 14.14 Distrib 5.5.43, for debian-linux-gnu (x86_64) using readline 6.2

DB-Engine - InnoDB

PHP version:
PHP 5.6.11-1+deb.sury.org~precise+1 (cli)
Copyright (c) 1997-2015 The PHP Group
Zend Engine v2.6.0, Copyright (c) 1998-2015 Zend Technologies
with Zend OPcache v7.0.6-dev, Copyright (c) 1999-2015, by Zend Technologies

ownCloud version: (see ownCloud admin page)
ownCloud 8.1.0 (stable)

Updated from an older ownCloud or fresh install:
update from 6.04 - 7 - 8; did all minor / major upgrades/updates
(all by hand (downloaded code from stable chan))
List of activated apps:
Enabled:

• files: 1.1.9
• files_pdfviewer: 0.7
• files_sharing: 0.6.2 <- is this version correct?
• files_trashbin: 0.6.3
• files_videoviewer: 0.1.3
• gallery: 0.6.0
• updater: 0.6
• user_ldap: 0.6.1

**The content of config/config.p$CONFIG = array (

'instanceid' => 'someid',
'passwordsalt' => 'somesalt',
'datadirectory' => '/directory',
'dbtype' => 'mysql',
'version' => '8.1.0.8',
'dbname' => 'owncloud_db_name',
'dbhost' => 'localhost',
'dbtableprefix' => 'oc_',
'dbuser' => 'someuser',
'dbpassword' => 'secret',
'installed' => true,
'ldapIgnoreNamingRules' => false,
'maintenance' => false,
'logtimezone' => 'Europe/Berlin',
'log_type' => 'owncloud',
'log_authfailip' => true,
'loglevel' => 1,
'allowZipDownload' => true,
'maxZipInputSize' => 838860800,
'theme' => '',
'trusted_domains' =>
array (
0 => 'somedomain.com',
),
'secret' => 'somesecret',
'mail_smtpmode' => 'php',
'mail_smtphost' => '127.0.0.1',
'mail_from_address' => 'owncloud',
'mail_domain' => 'somedomain.com',
'mail_smtpsecure' => 'tls',
'mail_smtpport' => '25',
);

Are you using external storage, if yes which one: local/smb/sftp/...
no
Are you using encryption: yes/no
no
Are you using an external user-backend, if yes which one: LDAP/ActiveDirectory/Webdav/...
LDAP

LDAP configuration (delete this part if not used)

+-------------------------------+------------------------------------------------------------+
| Configuration | |
+-------------------------------+------------------------------------------------------------+
| hasMemberOfFilterSupport | |
| hasPagedResultSupport | |
| homeFolderNamingRule | attr:uid |
| lastJpegPhotoLookup | 0 |
| ldapAgentName | |
| ldapAgentPassword | *** |
| ldapAttributesForGroupSearch | |
| ldapAttributesForUserSearch | |
| ldapBackupHost | |
| ldapBackupPort | 389 |
| ldapBase | dc=some,dc=domain,dc=com |
| ldapBaseGroups | ou=groups,dc=some,dc=domain,dc=com |
| ldapBaseUsers | ou=people,dc=some,dc=domain,dc=com |
| ldapCacheTTL | 600 |
| ldapConfigurationActive | 1 |
| ldapEmailAttribute | |
| ldapExperiencedAdmin | 0 |
| ldapExpertUUIDGroupAttr | auto |
| ldapExpertUUIDUserAttr | cn |
| ldapExpertUsernameAttr | |
| ldapGroupDisplayName | cn |
| ldapGroupFilter | (&(|(objectclass=posixGroup))) |
| ldapGroupFilterGroups | |
| ldapGroupFilterMode | 0 |
| ldapGroupFilterObjectclass | posixGroup |
| ldapGroupMemberAssocAttr | memberUid |
| ldapHost | ldap://someip |
| ldapIgnoreNamingRules | |
| ldapLoginFilter | (&(|(objectclass=inetOrgPerson))(|(uid=%uid)(|(cn=%uid)))) |
| ldapLoginFilterAttributes | cn |
| ldapLoginFilterEmail | 0 |
| ldapLoginFilterMode | 0 |
| ldapLoginFilterUsername | 1 |
| ldapNestedGroups | 0 |
| ldapNoCase | 0 |
| ldapOverrideMainServer | 0 |
| ldapPagingSize | 500 |
| ldapPort | 389 |
| ldapQuotaAttribute | |
| ldapQuotaDefault | |
| ldapTLS | 0 |
| ldapUserDisplayName | cn |
| ldapUserFilter | (|(objectclass=inetOrgPerson)) |
| ldapUserFilterGroups | |
| ldapUserFilterMode | 0 |
| ldapUserFilterObjectclass | inetOrgPerson |
| ldapUuidGroupAttribute | auto |
| ldapUuidUserAttribute | auto |
| turnOffCertCheck | 0 |
| useMemberOfToDetectMembership | 1 |
+-------------------------------+------------------------------------------------------------+

Client configuration

Browser:
all
Operating system:
Win/OSX/Ubuntu

Logs

ownCloud log (data/owncloud.log)

Please remeber this log line is only the result that my db / vm rans out of memory but you can see the query owncloud wanted to do.

{"reqId":"Ya9r9/8DJS0cQWuGqckD","remoteAddr":"someip","app":"webdav","message":"Exception: {"Message":"An exception occurred while executing 'UPDATE oc_appconfig SET configvalue = ? WHERE appid = ?
AND configkey = ?' with params [1438354433.2183, \"files_sharing\", \"21\"]:\n\nSQLSTATE[HY000]: General error: 1205 Lock wait timeout exceeded; try restarting transaction","Code":0,"Trace":"#0 \/var\/www\/owncloud\/3rdparty\/doctrine\/dbal\/lib\/Doctrine\/DBAL\/DBALException.php ..... this goes on and on ...

What should / can i do?

If i may have dirty shares, how can i determine these shares and may fix?
I dont want to loose all my shares, if it is possible :-)

Thank you in advance
Thomas

[LukasReschke]

This is caused by

core/apps/files_sharing/lib/propagation/recipientpropagator.php

Lines 106 to 115 in fa0105d

	/**
	* @param array $share
	* @param int $time
	*/
	public function markDirty($share, $time = null) {
	if ($time === null) {
	$time = microtime(true);
	}
	$this->config->setAppValue('files_sharing', $share['id'], $time);
	}

which has been added by @icewind1991 with 30ad568.

@icewind1991 Do you have any idea what can cause this?

[LukasReschke]

Tentatively assigning sev2-high since it leads to a crash of the whole instance as discussed with @thomas955 via IRC. Machine is also quite a bigger one. (4GB memory, 4 cores a 2.6GHZ)

Symptoms: VM runs full, all cores are fully used

User data: 100GB

[LukasReschke]

New code path which was not existent with code before 8.1.0, should certainly addressed before EE.

cc @cmonteroluque @DeepDiver1975

[PVince81]

This looks like the new etag propagation code for shared files.

Might or might not be fixed by #18037 where infinite loops have been observed on upload for specific configurations of shares.

[icewind1991]

Might or might not be fixed by #18037 where infinite loops have been observed on upload for specific configurations of shares.

That problem would seem the most likely to me @thomas955 can you try applying #18037 and see if that solves the issue for you

[thomas955]

Im testing the patch atm. Ill send you a feedback unitl tomorow ~ 3pm, so you might be able to push this patch to 8.1.1 or so.

So far,
thank you

[thomas955]

The patch seems to solve my issue.

I encouraged my users to move, insert and delete some files on their shared folders.
Furthermore, yesterday i tried to upload lots of small files in reshared / shared folders which mostly causes problems before the patch was applied.
If users work on our owncloud instance, i still have some DB querys like this : UPDATE oc_appconfig SET configvalue = '1438605612.4871' WHERE appid = 'files_sharing' AND configkey = '21'
Is this a normal behaviour?

Also i have a question about the 30 - 40 rows in my owncloud db which looks like this:
ownclouddb->oc_appconfig>appid:files_sharing / configkey:"aNumberBetween1-300" / configvalue:"somethingThatLooksLikeATimestamp (e.g. 1438696228.44)"

Should or can i remove this rows?

I hope you can push this patch to 8.1.1. .

[LukasReschke]

Thanks for your testing, @thomas955. Much appreciated.

If users work on our owncloud instance, i still have some DB querys like this : UPDATE oc_appconfig SET configvalue = '1438605612.4871' WHERE appid = 'files_sharing' AND configkey = '21'
Is this a normal behaviour?

Also i have a question about the 30 - 40 rows in my owncloud db which looks like this:
ownclouddb->oc_appconfig>appid:files_sharing / configkey:"aNumberBetween1-300" / configvalue:"somethingThatLooksLikeATimestamp (e.g. 1438696228.44)"
Should or can i remove this rows?

@icewind1991 Can you help here? THX.

[PVince81]

@icewind1991 closed, but still expecting your explanation. Thanks.

[icewind1991]

Also i have a question about the 30 - 40 rows in my owncloud db which looks like this:
ownclouddb->oc_appconfig>appid:files_sharing / configkey:"aNumberBetween1-300" / configvalue:"somethingThatLooksLikeATimestamp (e.g. 1438696228.44)"

Should or can i remove this rows?

These rows still hold valid data so it's best not to delete them

[thomas955]

Thank you for your explanation.
If this is a normal behaviour ill hold this rows.

Thanks