Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Using object storage is slow because of « 429 too many request » on objectstorage auth API #3248

Closed
AntoineVe opened this issue Jan 25, 2017 · 5 comments
Labels

Comments

@AntoineVe
Copy link

Steps to reproduce

  1. Use swift object storage as primary storage
  2. Upload a lot of files quickly (many small files or use 100 Mbits link)

Expected behaviour

Files are stored directly on the bucket.

Actual behaviour

Error 500 is returned by nextcloud server to nextcloud client, because of an error in objectstore app : « Could not create object: Client error response [status code] 429 [reason phrase] Too Many Requests [url] https://auth.cloud.ovh.net/v2.0/tokens »

Server configuration

Operating system: FreeBSD 11
Web server: nginx + php-fpm
Database: mariaDB
PHP version: 7.0.14
Nextcloud version: 11.0.1
Updated from an older Nextcloud/ownCloud or fresh install: fresh install
Where did you install Nextcloud from: tarball archive
Signing status:

Signing status
No errors have been found.

The content of config/config.php:

Config report
{
    "system": {
        "instanceid": "oceuv3hz6tp6",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "nuage.van-elstraete.net"
        ],
        "datadirectory": "\/usr\/local\/www\/nextcloud_data",
        "overwrite.cli.url": "http:\/\/nuage.van-elstraete.net",
        "dbtype": "mysql",
        "version": "11.0.1.2",
        "dbname": "nextcloud",
        "dbhost": "localhost",
        "dbport": "",
        "dbtableprefix": "oc_",
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "logtimezone": "UTC",
        "installed": true,
        "objectstore": {
            "class": "OC\\Files\\ObjectStore\\Swift",
            "arguments": {
                "username": "***REMOVED SENSITIVE VALUE***",
                "password": "***REMOVED SENSITIVE VALUE***",
                "bucket": "nextcloud-ve-data",
                "autocreate": false,
                "region": "GRA1",
                "url": "https:\/\/auth.cloud.ovh.net\/v2.0",
                "tenantName": "***REMOVED SENSITIVE VALUE***",
                "serviceName": "swift"
            }
        },
        "memcache.local": "\\OC\\Memcache\\APCu",
        "filelocking.enabled": true,
        "memcache.locking": "\\OC\\Memcache\\Redis",
        "redis": {
            "host": "localhost",
            "port": 6379,
            "timeout": 0
        },
        "loglevel": 2
    }
}

Are you using external storage, if yes which one: none
Are you using encryption: no
Are you using an external user-backend, if yes which one: none

Logs

Nextcloud log (data/nextcloud.log)

Nextcloud log ``` {"reqId":"jrte5uCaPZF1+liXHDVr","remoteAddr":"188.165.203.76","app":"objectstore","message":"Could not create object: Client error response\n[status code] 429\n[reason phrase] Too Many Requests\n[url] https:\/\/auth.cloud.ovh.net\/v2.0\/tokens","level":3,"time":"2017-01-25T04:52:31+00:00","method":"PUT","url":"\/remote.php\/webdav\/Documents\/lilypond\/html\/Documentation\/internals\/sequentialmusic.html","user":"vanelstraete.antoine","version":"11.0.1.2"} {"reqId":"jrte5uCaPZF1+liXHDVr","remoteAddr":"188.165.203.76","app":"webdav","message":"Exception: {\"Message\":\"HTTP\\\/1.1 500 Client error response\\n[status code] 429\\n[reason phrase] Too Many Requests\\n[url] https:\\\/\\\/auth.cloud.ovh.net\\\/v2.0\\\/tokens\",\"Exception\":\"Sabre\\\\DAV\\\\Exception\",\"Code\":0,\"Trace\":\"#0 \\\/usr\\\/local\\\/www\\\/nextcloud\\\/apps\\\/dav\\\/lib\\\/Connector\\\/Sabre\\\/File.php(158): OCA\\\\DAV\\\\Connector\\\\Sabre\\\\File->convertToSabreException(Object(Guzzle\\\\Http\\\\Exception\\\\ClientErrorResponseException))\\n#1 \\\/usr\\\/local\\\/www\\\/nextcloud\\\/apps\\\/dav\\\/lib\\\/Connector\\\/Sabre\\\/Directory.php(137): OCA\\\\DAV\\\\Connector\\\\Sabre\\\\File->put(Resource id #31)\\n#2 \\\/usr\\\/local\\\/www\\\/nextcloud\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Server.php(1072): OCA\\\\DAV\\\\Connector\\\\Sabre\\\\Directory->createFile('sequentialmusic...', Resource id #31)\\n#3 \\\/usr\\\/local\\\/www\\\/nextcloud\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/CorePlugin.php(525): Sabre\\\\DAV\\\\Server->createFile('Documents\\\/lilyp...', Resource id #31, NULL)\\n#4 [internal function]: Sabre\\\\DAV\\\\CorePlugin->httpPut(Object(Sabre\\\\HTTP\\\\Request), Object(Sabre\\\\HTTP\\\\Response))\\n#5 \\\/usr\\\/local\\\/www\\\/nextcloud\\\/3rdparty\\\/sabre\\\/event\\\/lib\\\/EventEmitterTrait.php(105): call_user_func_array(Array, Array)\\n#6 \\\/usr\\\/local\\\/www\\\/nextcloud\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Server.php(479): Sabre\\\\Event\\\\EventEmitter->emit('method:PUT', Array)\\n#7 \\\/usr\\\/local\\\/www\\\/nextcloud\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Server.php(254): Sabre\\\\DAV\\\\Server->invokeMethod(Object(Sabre\\\\HTTP\\\\Request), Object(Sabre\\\\HTTP\\\\Response))\\n#8 \\\/usr\\\/local\\\/www\\\/nextcloud\\\/apps\\\/dav\\\/appinfo\\\/v1\\\/webdav.php(60): Sabre\\\\DAV\\\\Server->exec()\\n#9 \\\/usr\\\/local\\\/www\\\/nextcloud\\\/remote.php(165): require_once('\\\/usr\\\/local\\\/www\\\/...')\\n#10 {main}\",\"File\":\"\\\/usr\\\/local\\\/www\\\/nextcloud\\\/apps\\\/dav\\\/lib\\\/Connector\\\/Sabre\\\/File.php\",\"Line\":557,\"User\":\"vanelstraete.antoine\"}","level":4,"time":"2017-01-25T04:52:31+00:00","method":"PUT","url":"\/remote.php\/webdav\/Documents\blahblahblah...","user":"vanelstraete.antoine","version":"11.0.1.2"} ```
@LukasReschke
Copy link
Member

cc @icewind1991

@zizzfizzix
Copy link

Hi @LukasReschke,

I seem to be having the same problem but downloading files to my local computer.
My setup is pretty much exactly like @AntoineVe's. I'm also using OVH's object storage, could this be the culprit?
I'd be happy to provide any more details as this is preventing me from using nextcloud.

This is what the desktop client is showing me:
screen shot 2017-04-19 at 20 54 19

And this is what I found in the nextcloud.log:

{"reqId":"WPepfady9cAAAH8dqvIAAAAA","remoteAddr":"XX.XXX.XXX.XXX","app":"objectstore","message":"Could not get object: Client error response\n[status code] 429\n[reason phrase] Too Many Requests\n[url] https:\/\/auth.cloud.ovh.net\/v2.0\/tokens","level":3,"time":"2017-04-19T18:16:30+00:00","method":"GET","url":"\/remote.php\/webdav\/directory\/assets\/Triangle.svg","user":"kuba","version":"11.0.2.7"}
{"reqId":"WPepfady9cAAAH8dqvIAAAAA","remoteAddr":"XX.XXX.XXX.XXX","app":"objectstore","message":"Could not get object: Client error response\n[status code] 429\n[reason phrase] Too Many Requests\n[url] https:\/\/auth.cloud.ovh.net\/v2.0\/tokens","level":3,"time":"2017-04-19T18:16:30+00:00","method":"GET","url":"\/remote.php\/webdav\/directory\/assets\/Triangle.png","user":"kuba","version":"11.0.2.7"}

@icewind1991
Copy link
Member

#4116 adds caching for auth tokens to swift

@MorrisJobke
Copy link
Member

Please retry with 12.0.2. We improved a lot in the 12 series for object storage.

@Arvi89
Copy link

Arvi89 commented Jan 5, 2019

I don't know what happened but this doesn't work anymore. I'm using nextcloud 14.0.4 and I get the error all the time, making this unusable.
#12189

This was referenced Jun 7, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

6 participants