External Storage - Files over 6MB upload fails

[Orbital-Bit]

Steps to reproduce

1. Configure Google Drive as External Storage
2. Attempt to upload or sync file to External Storage Google Drive larger than 6MB

Expected behaviour

Files over 6MB should upload or sync, as files under 6MB do.

Actual behaviour

via website: the upload progress bar hangs/stops at 100%, and log shows;
Error PHP Out of memory (allocated 25165824) (tried to allocate 9441911 bytes) at /../files_external/3rdparty/google-api-php-client/src/Google/Http/MediaFileUpload.php#227
Desktop Sync Client: shows the file upload to 100%, then shows various ETag errors then crashes after a few retries. The server did not acknowledge the last chunk. (No e-tag were present) 6.7 MiB

!! uploading directly to OwnCloud storage, files of all sizes upload and sync as expected !!

Server configuration

Operating system: Linux (3rd party Host)
Web server: Apache
Database: MySQL 5.1
PHP version: 5.6
ownCloud version: 8.0.2
Updated from an older ownCloud or fresh install: Fresh

List of activated apps:
Activity 1.2.0
External storage support 0.2.3
First Run Wizard 1.1
Mail Template Editor 0.1
PDF Viewer 0.7
Pictures 0.6.0
Provisioning API 0.2
Share Files 0.6.1
Text Editor 0.4
Updater 0.4

The content of config/config.php:

<?php
$CONFIG = array (
  'maintenance' => false,
  'instanceid' => 'oc4rj5atx3p9',
  'passwordsalt' => 'xxx',
  'secret' => "xxx',
  'trusted_domains' => 
  array (
    0 => 'sync.heal-my-pc.com',
  ),
  'datadirectory' => '/homepages/34/d94395727/htdocs/healmyPC/SYNC/data',
  'overwrite.cli.url' => 'http://sync.heal-my-pc.com/owncloud',
  'dbtype' => 'mysql',
  'version' => '8.0.2.0',
  'dbname' => 'db568212355',
  'dbhost' => 'db568212355.db.1and1.com',
  'dbtableprefix' => 'oc_',
  'dbuser' => 'xxx',
  'dbpassword' => 'xxx',
  'installed' => true,
  'forcessl' => true,
  'forceSSLforSubdomains' => true,
  'mail_from_address' => 'ownCloud',
  'mail_smtpmode' => 'php',
  'mail_domain' => 'xxx.com',
  'check_for_working_htaccess' => true,
  'blacklisted_files' => 
  array (
    0 => '.htaccess',
  ),
  'theme' => '',
  'loglevel' => 1,
);

Are you using external storage, if yes which one: Google Drive
Are you using encryption: No

Client configuration

Browser: Chrome, Firefox
Operating system: Windows: 7,8,10,2012

Logs

Web server error log

N/A shared hosting

ownCloud log (data/owncloud.log)

Error   PHP Automatically populating $HTTP_RAW_POST_DATA is deprecated and will be removed in a future version. To avoid this warning set 'always_populate_raw_post_data' to '-1' in php.ini and use the php://input stream instead. at Unknown#0   2015-04-01T14:12:18+00:00
Error   PHP Automatically populating $HTTP_RAW_POST_DATA is deprecated and will be removed in a future version. To avoid this warning set 'always_populate_raw_post_data' to '-1' in php.ini and use the php://input stream instead. at Unknown#0   2015-04-01T14:04:05+00:00
Fatal   webdav  Exception: {"Message":"expected filesize 5242880 got 0","Code":0,"Trace":"#0 \/homepages\/34\/d94395727\/htdocs\/healmyPC\/SYNC\/owncloud\/lib\/private\/connector\/sabre\/file.php(74): OC_Connector_Sabre_File->createFileChunked(Resource id #428)\n#1 \/homepages\/34\/d94395727\/htdocs\/healmyPC\/SYNC\/owncloud\/lib\/private\/connector\/sabre\/directory.php(84): OC_Connector_Sabre_File->put(Resource id #428)\n#2 \/homepages\/34\/d94395727\/htdocs\/healmyPC\/SYNC\/owncloud\/3rdparty\/sabre\/dav\/lib\/Sabre\/DAV\/Server.php(1647): OC_Connector_Sabre_Directory->createFile('NamMade.7z-chun...', Resource id #428)\n#3 \/homepages\/34\/d94395727\/htdocs\/healmyPC\/SYNC\/owncloud\/3rdparty\/sabre\/dav\/lib\/Sabre\/DAV\/Server.php(900): Sabre\\DAV\\Server->createFile('GoogleDrive\/MyW...', Resource id #428, NULL)\n#4 [internal function]: Sabre\\DAV\\Server->httpPut('GoogleDrive\/MyW...')\n#5 \/homepages\/34\/d94395727\/htdocs\/healmyPC\/SYNC\/owncloud\/3rdparty\/sabre\/dav\/lib\/Sabre\/DAV\/Server.php(474): call_user_func(Array, 'GoogleDrive\/MyW...')\n#6 \/homepages\/34\/d94395727\/htdocs\/healmyPC\/SYNC\/owncloud\/3rdparty\/sabre\/dav\/lib\/Sabre\/DAV\/Server.php(214): Sabre\\DAV\\Server->invokeMethod('PUT', 'GoogleDrive\/MyW...')\n#7 \/homepages\/34\/d94395727\/htdocs\/healmyPC\/SYNC\/owncloud\/apps\/files\/appinfo\/remote.php(61): Sabre\\DAV\\Server->exec()\n#8 \/homepages\/34\/d94395727\/htdocs\/healmyPC\/SYNC\/owncloud\/remote.php(54): require_once('\/homepages\/34\/d...')\n#9 {main}","File":"\/homepages\/34\/d94395727\/htdocs\/healmyPC\/SYNC\/owncloud\/lib\/private\/connector\/sabre\/file.php","Line":288}    2015-04-01T13:58:52+00:00
Fatal   webdav  Exception: {"Message":"expected filesize 5242880 got 0","Code":0,"Trace":"#0 \/homepages\/34\/d94395727\/htdocs\/healmyPC\/SYNC\/owncloud\/lib\/private\/connector\/sabre\/file.php(74): OC_Connector_Sabre_File->createFileChunked(Resource id #428)\n#1 \/homepages\/34\/d94395727\/htdocs\/healmyPC\/SYNC\/owncloud\/lib\/private\/connector\/sabre\/directory.php(84): OC_Connector_Sabre_File->put(Resource id #428)\n#2 \/homepages\/34\/d94395727\/htdocs\/healmyPC\/SYNC\/owncloud\/3rdparty\/sabre\/dav\/lib\/Sabre\/DAV\/Server.php(1647): OC_Connector_Sabre_Directory->createFile('NamMade.7z-chun...', Resource id #428)\n#3 \/homepages\/34\/d94395727\/htdocs\/healmyPC\/SYNC\/owncloud\/3rdparty\/sabre\/dav\/lib\/Sabre\/DAV\/Server.php(900): Sabre\\DAV\\Server->createFile('GoogleDrive\/MyW...', Resource id #428, NULL)\n#4 [internal function]: Sabre\\DAV\\Server->httpPut('GoogleDrive\/MyW...')\n#5 \/homepages\/34\/d94395727\/htdocs\/healmyPC\/SYNC\/owncloud\/3rdparty\/sabre\/dav\/lib\/Sabre\/DAV\/Server.php(474): call_user_func(Array, 'GoogleDrive\/MyW...')\n#6 \/homepages\/34\/d94395727\/htdocs\/healmyPC\/SYNC\/owncloud\/3rdparty\/sabre\/dav\/lib\/Sabre\/DAV\/Server.php(214): Sabre\\DAV\\Server->invokeMethod('PUT', 'GoogleDrive\/MyW...')\n#7 \/homepages\/34\/d94395727\/htdocs\/healmyPC\/SYNC\/owncloud\/apps\/files\/appinfo\/remote.php(61): Sabre\\DAV\\Server->exec()\n#8 \/homepages\/34\/d94395727\/htdocs\/healmyPC\/SYNC\/owncloud\/remote.php(54): require_once('\/homepages\/34\/d...')\n#9 {main}","File":"\/homepages\/34\/d94395727\/htdocs\/healmyPC\/SYNC\/owncloud\/lib\/private\/connector\/sabre\/file.php","Line":288}    2015-04-01T13:58:52+00:00
Fatal   webdav  Exception: {"Message":"expected filesize 1485018 got 0","Code":0,"Trace":"#0 \/homepages\/34\/d94395727\/htdocs\/healmyPC\/SYNC\/owncloud\/lib\/private\/connector\/sabre\/file.php(74): OC_Connector_Sabre_File->createFileChunked(Resource id #428)\n#1 \/homepages\/34\/d94395727\/htdocs\/healmyPC\/SYNC\/owncloud\/lib\/private\/connector\/sabre\/directory.php(84): OC_Connector_Sabre_File->put(Resource id #428)\n#2 \/homepages\/34\/d94395727\/htdocs\/healmyPC\/SYNC\/owncloud\/3rdparty\/sabre\/dav\/lib\/Sabre\/DAV\/Server.php(1647): OC_Connector_Sabre_Directory->createFile('logob.bmp-chunk...', Resource id #428)\n#3 \/homepages\/34\/d94395727\/htdocs\/healmyPC\/SYNC\/owncloud\/3rdparty\/sabre\/dav\/lib\/Sabre\/DAV\/Server.php(900): Sabre\\DAV\\Server->createFile('GoogleDrive\/MyW...', Resource id #428, NULL)\n#4 [internal function]: Sabre\\DAV\\Server->httpPut('GoogleDrive\/MyW...')\n#5 \/homepages\/34\/d94395727\/htdocs\/healmyPC\/SYNC\/owncloud\/3rdparty\/sabre\/dav\/lib\/Sabre\/DAV\/Server.php(474): call_user_func(Array, 'GoogleDrive\/MyW...')\n#6 \/homepages\/34\/d94395727\/htdocs\/healmyPC\/SYNC\/owncloud\/3rdparty\/sabre\/dav\/lib\/Sabre\/DAV\/Server.php(214): Sabre\\DAV\\Server->invokeMethod('PUT', 'GoogleDrive\/MyW...')\n#7 \/homepages\/34\/d94395727\/htdocs\/healmyPC\/SYNC\/owncloud\/apps\/files\/appinfo\/remote.php(61): Sabre\\DAV\\Server->exec()\n#8 \/homepages\/34\/d94395727\/htdocs\/healmyPC\/SYNC\/owncloud\/remote.php(54): require_once('\/homepages\/34\/d...')\n#9 {main}","File":"\/homepages\/34\/d94395727\/htdocs\/healmyPC\/SYNC\/owncloud\/lib\/private\/connector\/sabre\/file.php","Line":288}    2015-04-01T13:58:52+00:00
Error   PHP Out of memory (allocated 20709376) (tried to allocate 15961041 bytes) at /homepages/34/d94395727/htdocs/healmyPC/SYNC/owncloud/apps/files_external/3rdparty/google-api-php-client/src/Google/Http/MediaFileUpload.php#227   2015-04-01T13:58:01+00:00
Fatal   webdav  Exception: {"Message":"expected filesize 5242880 got 558080","Code":0,"Trace":"#0 \/homepages\/34\/d94395727\/htdocs\/healmyPC\/SYNC\/owncloud\/lib\/private\/connector\/sabre\/file.php(74): OC_Connector_Sabre_File->createFileChunked(Resource id #428)\n#1 \/homepages\/34\/d94395727\/htdocs\/healmyPC\/SYNC\/owncloud\/lib\/private\/connector\/sabre\/directory.php(84): OC_Connector_Sabre_File->put(Resource id #428)\n#2 \/homepages\/34\/d94395727\/htdocs\/healmyPC\/SYNC\/owncloud\/3rdparty\/sabre\/dav\/lib\/Sabre\/DAV\/Server.php(1647): OC_Connector_Sabre_Directory->createFile('NamMade.7z-chun...', Resource id #428)\n#3 \/homepages\/34\/d94395727\/htdocs\/healmyPC\/SYNC\/owncloud\/3rdparty\/sabre\/dav\/lib\/Sabre\/DAV\/Server.php(900): Sabre\\DAV\\Server->createFile('GoogleDrive\/MyW...', Resource id #428, NULL)\n#4 [internal function]: Sabre\\DAV\\Server->httpPut('GoogleDrive\/MyW...')\n#5 \/homepages\/34\/d94395727\/htdocs\/healmyPC\/SYNC\/owncloud\/3rdparty\/sabre\/dav\/lib\/Sabre\/DAV\/Server.php(474): call_user_func(Array, 'GoogleDrive\/MyW...')\n#6 \/homepages\/34\/d94395727\/htdocs\/healmyPC\/SYNC\/owncloud\/3rdparty\/sabre\/dav\/lib\/Sabre\/DAV\/Server.php(214): Sabre\\DAV\\Server->invokeMethod('PUT', 'GoogleDrive\/MyW...')\n#7 \/homepages\/34\/d94395727\/htdocs\/healmyPC\/SYNC\/owncloud\/apps\/files\/appinfo\/remote.php(61): Sabre\\DAV\\Server->exec()\n#8 \/homepages\/34\/d94395727\/htdocs\/healmyPC\/SYNC\/owncloud\/remote.php(54): require_once('\/homepages\/34\/d...')\n#9 {main}","File":"\/homepages\/34\/d94395727\/htdocs\/healmyPC\/SYNC\/owncloud\/lib\/private\/connector\/sabre\/file.php","Line":288}   2015-04-01T13:57:58+00:00
Fatal   webdav  Exception: {"Message":"PUT is not allowed on non-files.","Code":0,"Trace":"#0 [internal function]: Sabre\\DAV\\Server->httpPut('GoogleDrive\/MyW...')\n#1 \/homepages\/34\/d94395727\/htdocs\/healmyPC\/SYNC\/owncloud\/3rdparty\/sabre\/dav\/lib\/Sabre\/DAV\/Server.php(474): call_user_func(Array, 'GoogleDrive\/MyW...')\n#2 \/homepages\/34\/d94395727\/htdocs\/healmyPC\/SYNC\/owncloud\/3rdparty\/sabre\/dav\/lib\/Sabre\/DAV\/Server.php(214): Sabre\\DAV\\Server->invokeMethod('PUT', 'GoogleDrive\/MyW...')\n#3 \/homepages\/34\/d94395727\/htdocs\/healmyPC\/SYNC\/owncloud\/apps\/files\/appinfo\/remote.php(61): Sabre\\DAV\\Server->exec()\n#4 \/homepages\/34\/d94395727\/htdocs\/healmyPC\/SYNC\/owncloud\/remote.php(54): require_once('\/homepages\/34\/d...')\n#5 {main}","File":"\/homepages\/34\/d94395727\/htdocs\/healmyPC\/SYNC\/owncloud\/3rdparty\/sabre\/dav\/lib\/Sabre\/DAV\/Server.php","Line":885}   2015-04-01T13:57:57+00:00
Error   PHP Out of memory (allocated 33030144) (tried to allocate 13864955 bytes) at /homepages/34/d94395727/htdocs/healmyPC/SYNC/owncloud/apps/files_external/3rdparty/google-api-php-client/src/Google/Http/MediaFileUpload.php#227   2015-04-01T13:57:53+00:00
Error   PHP Out of memory (allocated 33292288) (tried to allocate 24905557 bytes) at /homepages/34/d94395727/htdocs/healmyPC/SYNC/owncloud/apps/files_external/3rdparty/google-api-php-client/src/Google/IO/Curl.php#79 2015-04-01T13:57:35+00:00
Error   PHP Out of memory (allocated 22282240) (tried to allocate 13787131 bytes) at /homepages/34/d94395727/htdocs/healmyPC/SYNC/owncloud/apps/files_external/3rdparty/google-api-php-client/src/Google/IO/Curl.php#79 2015-04-01T13:57:32+00:00
Error   PHP Out of memory (allocated 23330816) (tried to allocate 14861248 bytes) at /homepages/34/d94395727/htdocs/healmyPC/SYNC/owncloud/apps/files_external/3rdparty/google-api-php-client/src/Google/IO/Curl.php#79 2015-04-01T13:57:28+00:00
Error   PHP Out of memory (allocated 25165824) (tried to allocate 9329579 bytes) at /homepages/34/d94395727/htdocs/healmyPC/SYNC/owncloud/apps/files_external/3rdparty/google-api-php-client/src/Google/Http/MediaFileUpload.php#227

[nickvergessen]

Looks like you only have 16MB of ram for your php?

[Orbital-Bit]

may have 16MB allocated to PHP.
this is shared hosting and i do not have direct access to many things (SSH, SQL superuser, etc).
i previously added following parameters to .htaccess;
php_value post_max_size 513M
php_value memory_limit 512M

[ghost]

Hi,

a memory limit of 16MB definitely won't work for ownCloud. Where the parameters from your .htaccess used?

[Orbital-Bit]

only .htaccess at the root of ownCloud installation was modified with those values.
phpinfo reports memory_limit 120M for local and master value.

[PVince81]

Hmm strange, I would expect the SDK API to not put the whole file in RAM when uploading but stream it from disk.

[PVince81]

Last time I checked I noticed that the library we use DOES store the whole file in memory...
Closing in favor of #21357

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.