Can't upload file greater than ~5mb with desktop client - (expected filesize 5242880 got 0)

[CriticalNoob]

Steps to reproduce

1. Put a file over 5mb in a sync folder or trigger an auto update in the android client.
2. Wait for it to "chunk"

Expected behaviour

File should upload without problem most of the time, and most of the files shouln't have to be reuploaded many times.

Actual behaviour

The upload will faill with : (expected filesize 5242880 got 0)
The desktop client will try again until the file get blacklisted, or sometime for some reason, it manage to upload it.

Server configuration

Operating system:
Ubuntu server 14.04

Web server:
Server version: Apache/2.4.7 (Ubuntu)
Server built: Jan 14 2016 17:45:23

Database:
mysql 5.5.46-MariaDB-1ubuntu0.14.04.2

PHP version:
PHP 5.5.9-1ubuntu4.14 (cli) (built: Oct 28 2015 01:34:46)
Copyright (c) 1997-2014 The PHP Group
Zend Engine v2.5.0, Copyright (c) 1998-2014 Zend Technologies
with Zend OPcache v7.0.3, Copyright (c) 1999-2014, by Zend Technologies

ownCloud version: (see ownCloud admin page)
ownCloud 8.2.2

Updated from an older ownCloud or fresh install:
Fresh

List of activated apps:

Enabled:
  - activity: 2.1.3
  - calendar: 0.8.1
  - contacts: 0.5.0.0
  - files: 1.2.0
  - files_pdfviewer: 0.7
  - files_sharing: 0.7.0
  - files_texteditor: 2.0
  - files_trashbin: 0.7.0
  - files_versions: 1.1.0
  - files_videoviewer: 0.1.3
  - firstrunwizard: 1.1
  - gallery: 14.2.0
  - notifications: 0.1.0
  - provisioning_api: 0.3.0
  - templateeditor: 0.1
  - updater: 0.6
Disabled:
  - encryption
  - external
  - files_external
  - user_external
  - user_ldap

The content of config/config.php:

{
    "system": {
        "instanceid": "ocqrxtoqv89g",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "http://example.com"
        ],
        "datadirectory": "\/var\/owncloud-data",
        "overwrite.cli.url": "https:\/\/example.com",
        "dbtype": "mysql",
        "version": "8.2.2.2",
        "dbname": "owncloud",
        "dbhost": "localhost",
        "dbtableprefix": "oc_",
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "logtimezone": "UTC",
        "installed": true,
        "memcache.local": "\\OC\\Memcache\\APCu",
        "mail_smtpmode": "php",
        "mail_from_address": "owncloud",
        "mail_domain": "example.com",
        "custom_csp_policy": "default-src 'self'; script-src 'self' https:\/\/stats.example.com 'unsafe-inline' 'unsafe-eval'; style-src 'self' 'unsafe-inline'; frame-src *; img-src *; font-src 'self' data:; media-src *"
    }
}

Client configuration

Browser:
No problem with browser, html upload works flawlessly.

Operating system:

Logs

Mac sync client :

01/02/2016 23:17:28test/08 Freedom Run.ogg                                         ownCloud                      Conserver sur liste noire : Error downloading https://example.com/remote.php/webdav/test/08 Freedom Run.ogg-chunking-2065184963-2-0 - server replied: Bad Request (expected filesize 5242880 got 0)
01/02/2016 23:16:04test/04 Ciel errant.ogg                                         ownCloud                      Cet élément n'a pas été synchronisé en raison des erreurs précédentes : Error downloading https://example.com/remote.php/webdav/test/04 Ciel errant.ogg-chunking-3974020243-2-0 - server replied: Bad Request (expected filesize 5242880 got 0)
01/02/2016 23:16:04test/02 The Thing That Should Not Be (live).ogg                 ownCloud                      Cet élément n'a pas été synchronisé en raison des erreurs précédentes : Error downloading https://example.com/remote.php/webdav/test/02 The Thing That Should Not Be (live).ogg-chunking-38187236-2-0 - server replied: Bad Request (expected filesize 5242880 got 0)

Web server error log

None in /var/log/apache2/

ownCloud log (data/owncloud.log)

Bunch of :

Sabre\DAV\Exception\BadRequest: HTTP/1.1 400 expected filesize 5242880 got 0
/var/www/owncloud/lib/private/connector/sabre/file.php - line 101: OC\Connector\Sabre\File->createFileChunked(Resource id #30)
/var/www/owncloud/lib/private/connector/sabre/directory.php - line 115: OC\Connector\Sabre\File->put(Resource id #30)
/var/www/owncloud/3rdparty/sabre/dav/lib/DAV/Server.php - line 1053: OC\Connector\Sabre\Directory->createFile('2-05 Grantchest...', Resource id #30)
/var/www/owncloud/3rdparty/sabre/dav/lib/DAV/CorePlugin.php - line 513: Sabre\DAV\Server->createFile('test/2-05 Grant...', Resource id #30, NULL)
[internal function] Sabre\DAV\CorePlugin->httpPut(Object(Sabre\HTTP\Request), Object(Sabre\HTTP\Response))
/var/www/owncloud/3rdparty/sabre/event/lib/EventEmitterTrait.php - line 105: call_user_func_array(Array, Array)
/var/www/owncloud/3rdparty/sabre/dav/lib/DAV/Server.php - line 469: Sabre\Event\EventEmitter->emit('method PUT', Array)
/var/www/owncloud/3rdparty/sabre/dav/lib/DAV/Server.php - line 254: Sabre\DAV\Server->invokeMethod(Object(Sabre\HTTP\Request), Object(Sabre\HTTP\Response))
/var/www/owncloud/apps/files/appinfo/remote.php - line 56: Sabre\DAV\Server->exec()
/var/www/owncloud/remote.php - line 137: require_once('/var/www/ownclo...')
{main}

I tried :

SSLRenegBufferSize 10486000 in apache owncloud.conf
LimitRequestBody 0

I changed upload limits in php.ini
output_buffering = on, off, 16G, whatever
post_max_size = 16G
max_input_time 3600
max_execution_time 3600

I have Read :
#3816
owncloud/core#9832
owncloud/core#9832

I'm aware that it is considered as a server configuration issue but my server is a complete "out of the box" virgin :
Vanilla ubuntu 14.04 with default configuration files, nothing custom (exept for debug purpose and as written in OC documentation), apache and php installed byt apt-get.
Vanilla ownCloud with stock config files, except the one that are adviced to change in the documentation.

The lack of any clear error message make it hard to debug, I haven't gotten something any more explicit than :
"expected filesize 5242880 got 0" it like something got wrong somewhere, but who is in fault ? Nothing in any other logs...

Upload works fine with other Webdav clients like cyberduck, that's weird, isn't the owncloud client supposed to use webdav too ?
So may it comes from the way the owncloud client works ? Is there a way I could check ?

[guruz]

Which client version? Did you try 2.1.1 rc1?
Which client OS?

I'm not sure why you say the Android client can trigger this too though..

[guruz]

(BTW, if you agree that it is a vanilla installation without any bells and whistles, http://owncloud.org/support is a better place)

[CriticalNoob]

Which client version? Did you try 2.1.1 rc1?

2.1.0 on Mac
1.9.0 on Android
2.1.0 on Linux (Ubuntu & Fedora) give the same issue, I just checked.

Which client OS?

The log I gave is from the Mac client as stated.

I'm not sure why you say the Android client can trigger this too though..

I'm sorry I'm not a native english speaker and I sometime do mistakes...
I said "trigger an auto update in the android client" but I meant "trigger an auto upload in the android client", and by that I meant picture auto upload : when a small picture is auto-upload (i.e. : front camera picture) it works, but when a bigger picture (more than ~5mb) is auto-uploaded it fails the same way the desktop client does, with a (expected filesize 5242880 got 0) in the client logs.

Please correct me if I'm wrong but 5242880 bits is the size of a chunk right ?
So it means that the problem is happening with the "chunk management" of OC.
I'm not familiar with the way the choking process is working, is it something made by OC or by Sabredav ? I'v looked for this issue in Sabredav git but I haven't found anything that seems related.

I just tried :
SSLRenegBufferSize 10486000
and
KeepAlive On
KeepAliveTimeout 100
MaxKeepAliveRequests 200
Without any change.

[ghost]

@CriticalNoob Also checked owncloud/core#9832 (comment), especially the known bug in Apache 2.4.6+ ?

[CriticalNoob]

It seems I'm too dumb to understand ubuntu way of managing packages, but I don't think the fix has been applied in latest apache update for ubuntu 14.04.

I will let you know if I find something.

[CriticalNoob]

Okay I upgraded apache to 2.4.18 which should supposedly fix the problem but still got bunch of expected filesize 5242880 got 0

[CriticalNoob]

For some reason now some upload works, but there is something interesting.
I can predict if an upload is going to fail.
When the upload is being done, as you a know chunking.part files are created in the cache folder.
But sometime, the file is not created at all, even if the client is showing upload progress and then the "expected filesize 5242880 got 0" will happen for a good reason, the file is not created.

The question is why ?

@CriticalNoob Also checked owncloud/core#9832 (comment), especially the known bug in Apache 2.4.6+ ?

I must precise that I have got only "expected filesize 5242880 got 0" errors, no "expected filesize 5242880 got xxxxx"

btw If you want to check my apache config :

Apache :

<IfModule mod_ssl.c>
<VirtualHost *:443>
ServerName example.com
ServerAlias www.example.com

  <IfModule mod_headers.c>
      Header always set Strict-Transport-Security "max-age=15768000; includeSubDomains; preload"
        <FilesMatch "\.(css|js|svg|gif|woff)$">
        ExpiresActive On
        ExpiresDefault "access plus 2 week"
        Header set Cache-Control "max-age=1209600, public"
        </FilesMatch>
   </IfModule>

DocumentRoot /var/www/owncloud
ErrorLog ${APACHE_LOG_DIR}/error.example.com.log
CustomLog ${APACHE_LOG_DIR}/access.example.log combined
SetEnvIf Authorization "(.*)" HTTP_AUTHORIZATION=$1
Alias / "/var/www/owncloud/"
<Directory "/var/www/owncloud">
    Options +FollowSymLinks
    AllowOverride All
    SetEnv HOME /var/www/owncloud
    SetEnv HTTP_HOME /var/www/owncloud
</Directory>

<Directory "/var/owncloud-data/">
SSLRenegBufferSize 10486000
  # just in case if .htaccess gets disabled
  Require all denied
</Directory>

Redirect 301 /.well-known/carddav /remote.php/carddav
Redirect 301 /.well-known/caldav /remote.php/caldav
SSLCertificateFile /etc/letsencrypt/live/example.com-0001/cert.pem
SSLCertificateKeyFile /etc/letsencrypt/live/example.com-0001/privkey.pem
Include /etc/letsencrypt/options-ssl-apache.conf
SSLCertificateChainFile /etc/letsencrypt/live/example.com-0001/chain.pem
</VirtualHost>

</IfModule>

[CriticalNoob]

Today I tried something else :
I installed owncloud from the repo and only added

 <IfModule mod_headers.c>
      Header always set Strict-Transport-Security "max-age=15768000; includeSubDomains; preload"
    </IfModule>

to owncloud.conf

and

'memcache.local' => '\OC\Memcache\APCu',

to config.php

I then installed some apps, and let it sync some files.
It worked pretty well until I got this in the log :

{"reqId":"mppwfui\/qR7osad1BLGy","remoteAddr":"xx.xx.xx.xx","app":"PHP","message":"touch(): Unable to create file \/var\/www\/owncloud\/data\/testuser\/cache\/ftest-file2-2.sql 2.zip-chunking-1125685375-0.k6fwZbsp04DNvTdp.part because No such file or directory at \/var\/www\/owncloud\/lib\/private\/files\/storage\/local.php#167","level":3,"time":"2016-02-04T11:11:31+00:00"}
{"reqId":"mppwfui\/qR7osad1BLGy","remoteAddr":"xx.xx.xx.xx","app":"core","message":"unable to rename, source directory is not writable : cache","level":3,"time":"2016-02-04T11:11:31+00:00"}
{"reqId":"mppwfui\/qR7osad1BLGy","remoteAddr":"xx.xx.xx.xx","app":"webdav","message":"Exception: {\"Message\":\"HTTP\\\/1.1 400 expected filesize 5242880 got 0\",\"Exception\":\"Sabre\\\\DAV\\\\Exception\\\\BadRequest\",\"Code\":0,\"Trace\":\"#0 \\\/var\\\/www\\\/owncloud\\\/lib\\\/private\\\/connector\\\/sabre\\\/file.php(101): OC\\\\Connector\\\\Sabre\\\\File->createFileChunked(Resource id #36)\\n#1 \\\/var\\\/www\\\/owncloud\\\/lib\\\/private\\\/connector\\\/sabre\\\/directory.php(115): OC\\\\Connector\\\\Sabre\\\\File->put(Resource id #36)\\n#2 \\\/var\\\/www\\\/owncloud\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Server.php(1053): OC\\\\Connector\\\\Sabre\\\\Directory->createFile('ftest-file2-2.sql...', Resource id #36)\\n#3 \\\/var\\\/www\\\/owncloud\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/CorePlugin.php(513): Sabre\\\\DAV\\\\Server->createFile('ftest-file2-2.sql...', Resource id #36, NULL)\\n#4 [internal function]: Sabre\\\\DAV\\\\CorePlugin->httpPut(Object(Sabre\\\\HTTP\\\\Request), Object(Sabre\\\\HTTP\\\\Response))\\n#5 \\\/var\\\/www\\\/owncloud\\\/3rdparty\\\/sabre\\\/event\\\/lib\\\/EventEmitterTrait.php(105): call_user_func_array(Array, Array)\\n#6 \\\/var\\\/www\\\/owncloud\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Server.php(469): Sabre\\\\Event\\\\EventEmitter->emit('method:PUT', Array)\\n#7 \\\/var\\\/www\\\/owncloud\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Server.php(254): Sabre\\\\DAV\\\\Server->invokeMethod(Object(Sabre\\\\HTTP\\\\Request), Object(Sabre\\\\HTTP\\\\Response))\\n#8 \\\/var\\\/www\\\/owncloud\\\/apps\\\/files\\\/appinfo\\\/remote.php(56): Sabre\\\\DAV\\\\Server->exec()\\n#9 \\\/var\\\/www\\\/owncloud\\\/remote.php(137): require_once('\\\/var\\\/www\\\/ownclo...')\\n#10 {main}\",\"File\":\"\\\/var\\\/www\\\/owncloud\\\/lib\\\/private\\\/connector\\\/sabre\\\/file.php\",\"Line\":361}","level":4,"time":"2016-02-04T11:11:31+00:00"}

And then I got back to "expected filesize 5242880 got 0" forever.

"cache" permission are :

drwxr-xr-x 2 www-data www-data 4096 Feb 4 12:17 cache

[ckamm]

@CriticalNoob This is very likely a configuration issue that we unfortunately don't have the resources to help you track down. It's unlikely to be a client bug since you get similar problems with the desktop and android clients.