Error reporting for too long filepaths (>255 chars)

[jzaers]

Steps to reproduce

1. Use filenames with a resulting path longer than 255 characters
2. This seems to result in various strange side effects like move not executed correctly and
   the following error message:

{"reqId":"xZAVDUDUMFn0yANyU6rA","level":4,"time":"2018-06-14T06:52:08+00:00","remoteAddr":"10.0.50.4","user":"87c48f84-4b31-4c8e-aac8-1c554f1b500b","app":"webdav","method":"MOVE","url":"\/owncloud\/remote.php\/dav\/files\/LONG_FILENAME_REPLACED_WITH_247_CHARACTERS","message":"Exception: {\"Exception\":\"Doctrine\\\\DBAL\\\\Exception\\\\DriverException\",\"Message\":\"An exception occurred while executing 'UPDATE \\\"oc_properties\\\" SET \\\"propertypath\\\" = ? WHERE \\\"userid\\\" = ? AND \\\"propertypath\\\" = ?' with params [\\\"files\\\\\\\LONG_FILENAME_REPLACED_WITH_247_CHARACTERS", \\\"LONG_FILENAME_REPLACED_WITH_247_CHARACTERS"]:\\n\\nSQLSTATE[22001]: String data, right truncated: **7 ERROR:  value too long for type character varying(255)**\",\"Code\":0,\"Trace\":\"#0 \\\/opt\\\/nextcloud\\\/3rdparty\\\/doctrine\\\/dbal\\\/lib\\\/Doctrine\\\/DBAL\\\/DBALException.php(128): Doctrine\\\\DBAL\\\\Driver\\\\AbstractPostgreSQLDriver->convertException('An exception oc...', Object(Doctrine\\\\DBAL\\\\Driver\\\\PDOException))\\n#1 \\\/opt\\\/nextcloud\\\/3rdparty\\\/doctrine\\\/dbal\\\/lib\\\/Doctrine\\\/DBAL\\\/Statement.php(178): Doctrine\\\\DBAL\\\\DBALException::driverExceptionDuringQuery(Object(Doctrine\\\\DBAL\\\\Driver\\\\PDOPgSql\\\\Driver), Object(Doctrine\\\\DBAL\\\\Driver\\\\PDOException), 'UPDATE \\\"oc_prop...', Array)\\n#2 \\\/opt\\\/nextcloud\\\/apps\\\/dav\\\/lib\\\/DAV\\\/CustomPropertiesBackend.php(179): Doctrine\\\\DBAL\\\\Statement->execute(Array)\\n#3 \\\/opt\\\/nextcloud\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/PropertyStorage\\\/Plugin.php(147): OCA\\\\DAV\\\\DAV\\\\CustomPropertiesBackend->move('files\\\/87c48f84-...', 'files\\\/87c48f84-...')\\n#4 [internal function]: Sabre\\\\DAV\\\\PropertyStorage\\\\Plugin->afterMove('files\\\/87c48f84-...', 'files\\\/87c48f84-...')\\n#5 \\\/opt\\\/nextcloud\\\/3rdparty\\\/sabre\\\/event\\\/lib\\\/EventEmitterTrait.php(105): call_user_func_array(Array, Array)\\n#6 \\\/opt\\\/nextcloud\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/CorePlugin.php(648): Sabre\\\\Event\\\\EventEmitter->emit('afterMove', Array)\\n#7 [internal function]: Sabre\\\\DAV\\\\CorePlugin->httpMove(Object(Sabre\\\\HTTP\\\\Request), Object(Sabre\\\\HTTP\\\\Response))\\n#8 \\\/opt\\\/nextcloud\\\/3rdparty\\\/sabre\\\/event\\\/lib\\\/EventEmitterTrait.php(105): call_user_func_array(Array, Array)\\n#9 \\\/opt\\\/nextcloud\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Server.php(479): Sabre\\\\Event\\\\EventEmitter->emit('method:MOVE', Array)\\n#10 \\\/opt\\\/nextcloud\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Server.php(254): Sabre\\\\DAV\\\\Server->invokeMethod(Object(Sabre\\\\HTTP\\\\Request), Object(Sabre\\\\HTTP\\\\Response))\\n#11 \\\/opt\\\/nextcloud\\\/apps\\\/dav\\\/lib\\\/Server.php(258): Sabre\\\\DAV\\\\Server->exec()\\n#12 \\\/opt\\\/nextcloud\\\/apps\\\/dav\\\/appinfo\\\/v2\\\/remote.php(33): OCA\\\\DAV\\\\Server->exec()\\n#13 \\\/opt\\\/nextcloud\\\/remote.php(164): require_once('\\\/opt\\\/nextcloud\\\/...')\\n#14 {main}\",\"File\":\"\\\/opt\\\/nextcloud\\\/3rdparty\\\/doctrine\\\/dbal\\\/lib\\\/Doctrine\\\/DBAL\\\/Driver\\\/AbstractPostgreSQLDriver.php\",\"Line\":91}","userAgent":"Mozilla\/5.0 (Macintosh) mirall\/2.4.1 (build 9367)","version":"12.0.6.1"}

Expected behaviour

I would like to

• receive an error in the client during the initial sync of a file with a too long path, so that I can correct this on the client.
• receive an sync error in the client when the move failed due to a too long filename
• maybe a nice little helper routine that checks that local filenames are not exceeding the expected length. Upps, this depends on the Username in the server, the username can be long in scenarios with synced useraccounts

Actual behaviour

Files with too long paths are synced to the server and encounter problems on later move operations.
It looks like the move is executed like a copy and the original file stays in the source directory. But this is not happening always. I assume that it depends on the resulting file path in the destination directory.

Server configuration

Operating system:
ubuntu 14.04 LTS

Web server:
Apache 2.4

Database:
postgresql 9.4

PHP version:
php 5.6

Nextcloud version: (see Nextcloud admin page)
see above, 12.0.6.1, (I updated the server to 13 afterwards, a few server configs below are from the new server, I try to reproduce this under 13.)

Updated from an older Nextcloud/ownCloud or fresh install:
Migration from owncloud 9 or 10.

Where did you install Nextcloud from:

Signing status:

Signing status

No errors have been found.

List of activated apps:
This is from the server after upgrade to v 13.

Enabled:

• activity: 2.6.1
• admin_audit: 1.3.0
• comments: 1.3.0
• dav: 1.4.7
• federatedfilesharing: 1.3.1
• federation: 1.3.0
• files: 1.8.0
• files_pdfviewer: 1.2.1
• files_sharing: 1.5.0
• files_texteditor: 2.5.1
• files_trashbin: 1.3.0
• files_versions: 1.6.0
• files_videoplayer: 1.2.0
• firstrunwizard: 2.2.1
• gallery: 18.0.0
• logreader: 2.0.0
• lookup_server_connector: 1.1.0
• nextcloud_announcements: 1.2.0
• notifications: 2.1.2
• oauth2: 1.1.1
• password_policy: 1.3.0
• provisioning_api: 1.3.0
• serverinfo: 1.3.0
• sharebymail: 1.3.0
• survey_client: 1.1.0
• systemtags: 1.3.0
• theming: 1.4.5
• twofactor_backupcodes: 1.2.3
• updatenotification: 1.3.0
• user_ldap: 1.3.1
• workflowengine: 1.3.0
  Disabled:
• bruteforcesettings
• encryption
• files_antivirus
• files_external
• user_external

Nextcloud configuration:

Config report

{
"system": {
"instanceid": "REMOVED SENSITIVE VALUE",
"passwordsalt": "REMOVED SENSITIVE VALUE",
"secret": "REMOVED SENSITIVE VALUE",
"trusted_domains": [
"REMOVED SENSITIVE VALUE",
"REMOVED SENSITIVE VALUE"
],
"datadirectory": "REMOVED SENSITIVE VALUE",
"overwrite.cli.url": "REMOVED SENSITIVE VALUE",
"version": "13.0.4.0",
"dbtype": "pgsql",
"dbhost": "REMOVED SENSITIVE VALUE",
"dbname": "REMOVED SENSITIVE VALUE",
"dbuser": "REMOVED SENSITIVE VALUE",
"dbpassword": "REMOVED SENSITIVE VALUE",
"dbtableprefix": "oc_",
"logtimezone": "UTC",
"installed": true,
"ldapIgnoreNamingRules": false,
"loglevel": 3,
"log_type": "owncloud",
"logfile": "/opt/oc-data/nextcloud.log",
"maintenance": false,
"singleuser": false,
"memcache.local": "\OC\Memcache\Redis",
"filelocking.enabled": "true",
"memcache.distributed": "\OC\Memcache\Redis",
"memcache.locking": "\OC\Memcache\Redis",
"redis": {
"host": "REMOVED SENSITIVE VALUE",
"port": 6379,
"timeout": 0,
"dbindex": 0
},
"trashbin_retention_obligation": "auto, 60",
"versions_retention_obligation": "auto",
"theme": "",
"updater.release.channel": "production",
"ldapProviderFactory": "\OCA\User_LDAP\LDAPProviderFactory",
"updater.secret": "REMOVED SENSITIVE VALUE"
}
}

Are you using external storage, if yes which one: local/smb/sftp/...
NO

Are you using encryption: yes/no
NO

Are you using an external user-backend, if yes which one: LDAP/ActiveDirectory/Webdav/...

YES, LDAP Apple OD, config not pasted, as not relevant.

Client configuration

Browser:

Operating system:
MACOS, nextcloud client 2.3.3

Logs

Web server error log

not relevant

Nextcloud log (data/nextcloud.log)

see above

Browser log

not relevant

[rafacouto]

I confirm this issue with docker images:

• postgres:10
• nextcloud:13-fpm
• nginx:latest

Just applied this path manually and waiting for results:

alter table properties alter column propertypath type character varying(1024);

[J0WI]

I'm able to reproduce during rename:

An exception occurred while executing 'UPDATE "oc_properties" SET "propertypath" = ? WHERE "userid" = ? AND "propertypath" = ?' with params ["***very long test path***", "***test user***", "***long test path***"]:
SQLSTATE[22001]: String data, right truncated: 7 ERROR:  value too long for type character 

server/core/Migrations/Version13000Date20170718121200.php

Lines 298 to 302 in ee5e767

	$table->addColumn('propertypath', 'string', [
	'notnull' => true,
	'length' => 255,
	'default' => '',
	]);

[kesselb]

server/core/Migrations/Version13000Date20170718121200.php

Lines 146 to 149 in ee5e767

	$table->addColumn('path', 'string', [
	'notnull' => false,
	'length' => 4000,
	]);

We should probably align this with path. Would you mind to create a pull request? ;)

[J0WI]

What is the right place for this change? I assume this is for 18 only?

[kesselb]

Yes. I probably forgot that this kind of change requires a bit more work. Just changing the value over there will not work 🙈

1. php occ migrations:generate core 18000 use occ to generate a migration

2. Delete preSchemaChange and postSchemaChange

		/** @var ISchemaWrapper $schema */
		$schema = $schemaClosure();

		if ($schema->hasTable('properties')) {
			$table = $schema->getTable('properties');

			$table->changeColumn('propertypath', [
				'notnull' => true,
				'length' => 4000,
				'default' => '',
			]);
		}

		return $schema;

3. Probably something like above for changeSchema but have not tested it.

4. php occ migrations:execute core 18000Date201911xzy run the migration

[ChristophWurst]

Discussed this with @rullzer briefly.

• We shouldn't do this migration by default as it will most likely time out on mid-sized to large installations.
• For 19 we can change the original migration so new installations get a more appropriate column width by default
• Add an optional repair step like we have for missing indices, so admins can later migrate the column with with an occ command

[J0WI]

For 19 we can change the original migration so new installations get a more appropriate column width by default

Since 19 has already been released I'll tag it for 20.

[susnux]

This is still an issue.

@J0WI are you sure? This should have been fixed with #19242