.ocTransferIDXXXXX.part makes filename too long #25425

juliangieseke · 2016-07-08T19:06:38Z

Just ran into that error on my Server, my exact Filename (encrypted by boxcryptor) is "倐哤幇岓奣划圵囩刜倃傤墑倩墌峂囱存底堝堁形屜孭啜屬囀凝匥媵忨噪厪婪媞妲匳墩密嵡寕征嫴啈宧吴厁宊埧姠偁儽忺入哄寘嚂屧剚室堙嶞初崵峋凗堡徟塮傳圔媀吽嶌孲執嬊夾埬塝䀋.bc" which is 245 bytes long. after adding the .ocTransfer suffix it has 272 bytes…

ssh@server:/www/htdocs$ echo 倐哤幇岓奣划圵囩刜倃傤墑倩墌峂囱存底堝堁形屜孭啜屬囀凝匥媵忨噪厪婪媞妲匳墩密嵡寕征嫴啈宧吴厁宊埧姠偁儽忺入哄寘嚂屧剚室堙嶞初崵峋凗堡徟塮傳圔媀吽嶌孲執嬊夾埬塝䀋.bc | wc -c
245
ssh@server:/www/htdocs$ echo 倐哤幇岓奣划圵囩刜倃傤墑倩墌峂囱存底堝堁形屜孭啜屬囀凝匥媵忨噪厪婪媞妲匳墩密嵡寕征嫴啈宧吴厁宊埧姠偁儽忺入哄寘嚂屧剚室堙嶞初崵峋凗堡徟塮傳圔媀吽嶌孲執嬊夾埬塝䀋.bc.ocTransferId371634213.part | wc -c
272

that suffix should be kept in mind when limiting uploads to OC - or even better: a shorter or no suffix should be used.

ownCloud 9.0.3

The text was updated successfully, but these errors were encountered:

juliangieseke · 2016-07-08T19:08:37Z

OC Logs:

Fatal webdav Exception: {"Message":"HTTP/1.1 500 Could not write file contents","Exception":"Sabre\DAV\Exception","Code":0,"Trace":"#0 /www/htdocs\…/owncloud-9.0.3/apps/dav/lib/connector/sabre/directory.php(134): OCA\DAV\Connector\Sabre\File->put(Resource id #338)\n#1 /www/htdocs\…/owncloud-9.0.3/3rdparty/sabre/dav/lib/DAV/Server.php(1036): OCA\DAV\Connector\Sabre\Directory->createFile('\xE5\x80\x90\xE5\x93\xA4\xE5\xB9\x87\xE5\xB2\x93\xE5\xA5\xA3...', Resource id #338)\n#2 /www/htdocs\…/owncloud-9.0.3/3rdparty/sabre/dav/lib/DAV/CorePlugin.php(523): Sabre\DAV\Server->createFile('Boxcryptor/Doku...', Resource id #338, NULL)\n#3 [internal function]: Sabre\DAV\CorePlugin->httpPut(Object(Sabre\HTTP\Request), Object(Sabre\HTTP\Response))\n#4 /www/htdocs\…/owncloud-9.0.3/3rdparty/sabre/event/lib/EventEmitterTrait.php(105): call_user_func_array(Array, Array)\n#5 /www/htdocs\…/owncloud-9.0.3/3rdparty/sabre/dav/lib/DAV/Server.php(459): Sabre\Event\EventEmitter->emit('method:PUT', Array)\n#6 /www/htdocs\…/owncloud-9.0.3/3rdparty/sabre/dav/lib/DAV/Server.php(248): Sabre\DAV\Server->invokeMethod(Object(Sabre\HTTP\Request), Object(Sabre\HTTP\Response))\n#7 /www/htdocs\…/owncloud-9.0.3/apps/dav/appinfo/v1/webdav.php(55): Sabre\DAV\Server->exec()\n#8 /www/htdocs\…/owncloud-9.0.3/remote.php(138): require_once('/www/htdocs/w01...')\n#9 {main}","File":"/www/htdocs\…/owncloud-9.0.3/apps/dav/lib/connector/sabre/file.php","Line":130,"User":"julian.gieseke"} 2016-07-08T18:53:01+00:00
Error webdav \OC\Files\Filesystem::fopen() failed 2016-07-08T18:53:01+00:00
Error PHP fopen(/www/htdocs/…/倐哲奄姢冟嶑婥寒坄寜䂄/倐听妃堲咛嚤嫓䀏/倐咫咫埩啹啡䀅/倐后困尦帟劻䃡/倐哤幇岓奣划圵囩刜倃傤墑倩墌峂囱存底堝堁形屜孭啜屬囀凝匥媵忨噪厪婪媞妲匳墩密嵡寕征嫴啈宧吴厁宊埧姠偁儽忺入哄寘嚂屧剚室堙嶞初崵峋凗堡徟塮傳圔媀吽嶌孲執嬊夾埬塝䀋.bc.ocTransferId371634213.part): failed to open stream: File name too long at /www/htdocs/…/owncloud-9.0.3/lib/private/files/storage/local.php#261 2016-07-08T18:53:01+00:00

PVince81 · 2016-07-11T09:09:16Z

Error PHP fopen(/www/htdocs/…/倐哲奄姢冟嶑婥寒坄寜䂄/倐听妃堲咛嚤嫓䀏/倐咫咫埩啹啡䀅/倐后困尦帟劻䃡/倐哤幇岓奣划圵囩刜倃傤墑倩墌峂囱存底堝堁形屜孭啜屬囀凝匥媵忨噪厪婪媞妲匳墩密嵡寕征嫴啈宧吴厁宊埧姠偁儽忺入哄寘嚂屧剚室堙嶞初崵峋凗堡徟塮傳圔媀吽嶌孲執嬊夾埬塝䀋.bc.ocTransferId371634213.part): failed to open stream: File name too long

This error message isn't generated in ownCloud, this means that the fopen call itself caused that issue.

Please use the issue template, would be good to know what system and filesystem you are running on that would cause such errors. Is that an external storage ?

https://raw.githubusercontent.com/owncloud/core/master/issue_template.md

PVince81 · 2016-07-11T09:11:41Z

I guess maybe part files don't even need to have the real filename in them as long as the transfer id is there for the final rename.

@owncloud/filesystem

juliangieseke · 2016-07-11T11:23:18Z

Steps to reproduce

create File with Name longer then 229(maybe 228) bytes

Expected behaviour

file should be uploaded

Actual behaviour

php error message above

Server configuration

Operating system:
ubuntu (shared hosting)
Web server:
shared hosting, apache 2.x
Database:
mysql (5.x)
PHP version:
7.0.6
ownCloud version: (see ownCloud admin page)
9.0.3
Updated from an older ownCloud or fresh install:
updated from 9.0.2
Where did you install ownCloud from:
owncloud.org website
Signing status (ownCloud 9.0 and above):
passed.

List of activated apps:

Enabled:
  - activity: 2.2.1
  - announcementcenter: 1.1.2
  - calendar: 1.2.2
  - comments: 0.2
  - contacts: 1.3.1.0
  - dav: 0.1.6
  - federatedfilesharing: 0.1.0
  - federation: 0.0.4
  - files: 1.4.4
  - files_pdfviewer: 0.8.1
  - files_sharing: 0.9.1
  - files_texteditor: 2.1
  - files_trashbin: 0.8.0
  - files_versions: 1.2.0
  - files_videoplayer: 0.9.8
  - firstrunwizard: 1.1
  - gallery: 14.5.0
  - notifications: 0.2.3
  - provisioning_api: 0.4.1
  - systemtags: 0.2
  - templateeditor: 0.1
  - updatenotification: 0.1.0
Disabled:
  - encryption
  - external
  - files_antivirus
  - files_external
  - mail
  - user_external
  - user_ldap

The content of config/config.php:

{
    "system": {
        "instanceid": "ocgbgmawcg1g",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "mydomain"
        ],
        "datadirectory": "\/www\/htdocs\/mydomain\/data",
        "overwrite.cli.url": "https:\/\/mydomain",
        "dbtype": "mysql",
        "version": "9.0.3.2",
        "dbname": "mydb",
        "dbhost": "localhost",
        "dbtableprefix": "oc_",
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "logtimezone": "UTC",
        "installed": true,
        "default_language": "de_DE",
        "tempdirectory": "\/www\/htdocs\/mydomain\/tmp",
        "mail_from_address": "technik",
        "mail_smtpmode": "smtp",
        "mail_domain": "mydomain",
        "trashbin_retention_obligation": "auto, 30",
        "versions_retention_obligation": "auto, 30",
        "log_rotate_size": 104857600,
        "mail_smtpsecure": "tls",
        "mail_smtpauthtype": "LOGIN",
        "mail_smtpauth": 1,
        "mail_smtphost": "mydomain",
        "mail_smtpname": "***REMOVED SENSITIVE VALUE***",
        "mail_smtppassword": "***REMOVED SENSITIVE VALUE***",
        "maintenance": false,
        "loglevel": 2
    }
}

Are you using external storage, if yes which one: local/smb/sftp/...
nope
Are you using encryption: yes/no
nope
Are you using an external user-backend, if yes which one: LDAP/ActiveDirectory/Webdav/...
nope

Client configuration

Browser:
OS X Desktop client?!
Operating system:
OS X

Logs

Web server error log

shared hosting

ownCloud log (data/owncloud.log)

see above

PVince81 · 2016-07-11T11:28:20Z

Shared hosting, ok. Maybe they added a limit to file name lengths.

juliangieseke · 2016-07-11T11:47:23Z

I dont think its a Shared-Hosting-Specific problem, ext4 has 255bytes as max filename length.

PVince81 · 2016-07-11T12:24:02Z

@DeepDiver1975 so we should limit file names to less than 255 due to suffixes.

juliangieseke · 2016-07-11T15:30:09Z

Maybe removing the original filename from .part files is a better idea… limiting filenames to <255 bytes could lead to the same problem: files generated by boxcryptor (and other encryption tools) cant be uploaded.

PVince81 · 2016-07-11T15:34:47Z

@dragotin @guruz something for you to have a look at ?

dragotin · 2016-07-11T15:52:45Z

also owncloud/enterprise#1366

Note that with the new chunking this problem is fixed anyway.

dragotin · 2016-07-11T16:13:17Z

A workaround might be:

If the filename of the chunk-file incl. path is bigger than 250 chars, the actual file component is checked if it is long and can be shortened to fit the chunked-<upload-id>-<count>-<overall-no> that comes with the chunk file. If that works, the client will do that and after the file was uploaded it remembers to send a rename.

I am actually unsure as a) that is hacky, b) it is fixed with the new chunking anyway, c) its still a big change.

PVince81 · 2016-07-11T16:27:43Z

@dragotin I think the fix doesn't require to affect whatever the client is sending.

The fix only for how the part file and chunk file are stored on-disk. The name already doesn't match 100% what the client sends. So we could maybe md5 the filename there.

PVince81 · 2016-07-11T16:29:45Z

Might be slightly related: #25214

Strange thing that boxcryptor generates names containing traditional chinese characters.

cdamken · 2016-07-14T10:47:05Z

https://dragotin.wordpress.com/tag/chunked-upload/

PVince81 · 2016-12-22T07:58:54Z

I suggest to simply md5 the base name of the chunk file name on disk on the server side.
Might be just a matter of changing some code in the FileChunking class.

@guruz are you still on this ?

sharidas · 2017-12-11T16:39:49Z

With this #29814 at least no more error logs appears. So far good. But there is one situation I am kind of stuck. While testing with delete operation, which results in moving the file to trashbin. This causes error:

{"reqId":"AhuNsH2nTlknbCZXM1s6","level":2,"time":"2017-12-11T16:18:07+00:00","remoteAddr":"127.0.0.1","user":"admin","app":"dav","method":"DELETE","url":"\/testing\/remote.php\/webdav\/hhhhhhhhhhhhhhhhhhhhhhhhheeeeeeeeeeeeeeeeeeeeeeeeeeelllllllllllllllllllllllllllllllllllloooooooooooooooooooooooooooooooooooooooooooohhhhhhhhhhhhhhhhhhhhhhhhhoooooooooooooooooooooooooooowwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwaaaaaaaaaarrrrrrrrreyoubro.txt","message":"Could not get node for path: \"hhhhhhhhhhhhhhhhhhhhhhhhheeeeeeeeeeeeeeeeeeeeeeeeeeelllllllllllllllllllllllllllllllllllloooooooooooooooooooooooooooooooooooooooooooohhhhhhhhhhhhhhhhhhhhhhhhhoooooooooooooooooooooooooooowwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwaaaaaaaaaarrrrrrrrreyoubro.txt\" : {$message}"}

https://github.com/owncloud/core/blob/master/lib/private/Files/Storage/Local.php#L269 is where the problem caused. The reason is for rename/move operation the target file has ".d" + timestamp associated with the file name. This makes the filename long ( I am using a filename which has 249 char length for testing ).

PVince81 · 2017-12-12T08:42:40Z

Hmmmm so this means we have other hidden issues with longer file names... Same will likely happen for versions.

PVince81 · 2017-12-12T09:02:19Z

@sharidas I suggest you ignore the trashbin/version issue for now as it would be for a different scope...

I have the feeling that we need to rework how we store any data that currently reuses the file name and makes it longer like encryption keys, trashed files, trashed versions, etc...

PVince81 · 2017-12-12T09:04:39Z

In theory for trashbin/versions, the data is anyway stored separately and not visible for end-users, so if we'd use md5's there as well it wouldn't hurt. I've raised #29819 to look into this.

PVince81 · 2017-12-12T09:08:46Z

@sharidas by the way, we already had a PR #26978. Please either reuse if that makes sense or close it.

Let's focus only on the upload case here for now.

ownclouders · 2018-01-22T01:06:31Z

Hey, this issue has been closed because the label status/STALE is set and there were no updates for 7 days. Feel free to reopen this issue if you deem it appropriate.

(This is an automated comment from GitMate.io.)

cdamken · 2018-01-22T12:09:45Z

@IljaN @PVince81 Can we keep this open?

ownclouders · 2018-03-02T01:09:11Z

Hey, this issue has been closed because the label status/STALE is set and there were no updates for 7 days. Feel free to reopen this issue if you deem it appropriate.

(This is an automated comment from GitMate.io.)

felixboehm · 2018-04-10T09:08:31Z

We will have much more trouble with md5 hashes, this is not really a solution IMHO.
The simple solution is, when there is an issue (in FS or DB) with handling too long filenames / pathes, use shorter filenames / pathes.

I don't see clean / other solutions for now.

PVince81 · 2018-04-10T10:32:11Z

we might need to prevent OC to accept long filenames at all and fail gracefully

scysys · 2018-09-06T12:10:07Z

Problem still exist with these Filenames. Actually I can solve this error with Boxcryptor when I disable the filename encryption.

PVince81 added the Type:Bug label Jul 11, 2016

PVince81 added this to the 9.1.1 milestone Jul 11, 2016

PVince81 added the comp:filesystem label Jul 11, 2016

guruz self-assigned this Jul 11, 2016

cdamken added the blue-ticket label Jul 14, 2016

PVince81 mentioned this issue Aug 22, 2016

part/chunk files over filename size limit aborting sync #25891

Closed

PVince81 modified the milestones: 9.1.1, 9.1.2 Sep 21, 2016

PVince81 modified the milestones: 9.1.3, 9.1.2 Oct 20, 2016

PVince81 modified the milestones: 9.1.4, 9.1.3 Dec 22, 2016

PVince81 added the sev2-high label Dec 22, 2016

guruz assigned PVince81 and unassigned guruz Jan 2, 2017

sharidas mentioned this issue Dec 11, 2017

[WIP] Fix filename long issue for part files #29814

Closed

9 tasks

PVince81 mentioned this issue Dec 12, 2017

Trashbin and version filenames as md5 to avoid file name length limit #29819

Closed

PVince81 mentioned this issue Dec 12, 2017

Deleting a 250 char file does not move it to the trash but throws an exception #28095

Closed

PVince81 unassigned sharidas Dec 14, 2017

ownclouders added the status/STALE label Jan 14, 2018

ownclouders closed this as completed Jan 22, 2018

ownclouders removed the status/STALE label Jan 22, 2018

IljaN self-assigned this Jan 22, 2018

cdamken reopened this Jan 22, 2018

ownclouders added the status/STALE label Feb 22, 2018

ownclouders closed this as completed Mar 2, 2018

ownclouders removed the status/STALE label Mar 2, 2018

sharidas reopened this Mar 2, 2018

ownclouders added the status/STALE label Apr 2, 2018

felixboehm closed this as completed Apr 9, 2018

felixboehm removed this from the triage milestone Apr 10, 2018

ownclouders removed the status/STALE label Apr 10, 2018

michaelstingl mentioned this issue Nov 20, 2018

Document platform-wide known limitations, excluded files owncloud/docs#365

Closed

9 tasks

lock bot locked as resolved and limited conversation to collaborators Sep 6, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

.ocTransferIDXXXXX.part makes filename too long #25425

.ocTransferIDXXXXX.part makes filename too long #25425

juliangieseke commented Jul 8, 2016 •

edited by DeepDiver1975

Loading

juliangieseke commented Jul 8, 2016

PVince81 commented Jul 11, 2016

PVince81 commented Jul 11, 2016

juliangieseke commented Jul 11, 2016 •

edited

Loading

PVince81 commented Jul 11, 2016

juliangieseke commented Jul 11, 2016

PVince81 commented Jul 11, 2016

juliangieseke commented Jul 11, 2016

PVince81 commented Jul 11, 2016

dragotin commented Jul 11, 2016

dragotin commented Jul 11, 2016

PVince81 commented Jul 11, 2016

PVince81 commented Jul 11, 2016

cdamken commented Jul 14, 2016

PVince81 commented Dec 22, 2016

sharidas commented Dec 11, 2017

PVince81 commented Dec 12, 2017

PVince81 commented Dec 12, 2017

PVince81 commented Dec 12, 2017

PVince81 commented Dec 12, 2017

ownclouders commented Jan 22, 2018

cdamken commented Jan 22, 2018 •

edited

Loading

ownclouders commented Mar 2, 2018

felixboehm commented Apr 10, 2018 •

edited

Loading

PVince81 commented Apr 10, 2018

scysys commented Sep 6, 2018

.ocTransferIDXXXXX.part makes filename too long #25425

.ocTransferIDXXXXX.part makes filename too long #25425

Comments

juliangieseke commented Jul 8, 2016 • edited by DeepDiver1975 Loading

juliangieseke commented Jul 8, 2016

PVince81 commented Jul 11, 2016

PVince81 commented Jul 11, 2016

juliangieseke commented Jul 11, 2016 • edited Loading

Steps to reproduce

Expected behaviour

Actual behaviour

Server configuration

Client configuration

Logs

Web server error log

ownCloud log (data/owncloud.log)

PVince81 commented Jul 11, 2016

juliangieseke commented Jul 11, 2016

PVince81 commented Jul 11, 2016

juliangieseke commented Jul 11, 2016

PVince81 commented Jul 11, 2016

dragotin commented Jul 11, 2016

dragotin commented Jul 11, 2016

PVince81 commented Jul 11, 2016

PVince81 commented Jul 11, 2016

cdamken commented Jul 14, 2016

PVince81 commented Dec 22, 2016

sharidas commented Dec 11, 2017

PVince81 commented Dec 12, 2017

PVince81 commented Dec 12, 2017

PVince81 commented Dec 12, 2017

PVince81 commented Dec 12, 2017

ownclouders commented Jan 22, 2018

cdamken commented Jan 22, 2018 • edited Loading

ownclouders commented Mar 2, 2018

felixboehm commented Apr 10, 2018 • edited Loading

PVince81 commented Apr 10, 2018

scysys commented Sep 6, 2018

juliangieseke commented Jul 8, 2016 •

edited by DeepDiver1975

Loading

juliangieseke commented Jul 11, 2016 •

edited

Loading

cdamken commented Jan 22, 2018 •

edited

Loading

felixboehm commented Apr 10, 2018 •

edited

Loading