Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Retries option ignored for async requests #1214

Closed
NinoSkopac opened this issue Mar 13, 2017 · 10 comments
Closed

Retries option ignored for async requests #1214

NinoSkopac opened this issue Mar 13, 2017 · 10 comments
Labels
guidance Question that needs advice or information.

Comments

@NinoSkopac
Copy link
Contributor

I am using AWS Polly with AWS SDK for PHP in async mode.

Polly has a rate limit of 80 per second, whereas I want to be able to burst and do more than that, and in case Polly API returns 503 Rate Limiting, I want to retry the request (my script works fine if under rate limit).

The docs state there's a retries option, but it does nothing: it doesn't matter if i set it to 0 or 1000, it doesn't seem to retry and I get rejected promises from the SDK.

$polly = new PollyClient($config);
// $config
array (size=4)
  'version' => string 'latest' (length=6)
  'credentials' => 
    array (size=2)
      'key' => string 'xxx' (length=20)
      'secret' => string 'xxx' (length=40)
  'region' => string 'eu-west-1' (length=9)
  'retries' => int 1000
@imshashank imshashank added guidance Question that needs advice or information. bug This issue is a bug. and removed guidance Question that needs advice or information. labels Mar 13, 2017
@cjyclaire cjyclaire added response-requested Waiting on additional info and feedback. Will move to "closing-soon" in 7 days. and removed bug This issue is a bug. labels Mar 13, 2017
@cjyclaire
Copy link
Contributor

cjyclaire commented Mar 13, 2017

@NinoSkopac Could you explain a bit how you find this doesn't work? I tried to reproduce this with a PollyClient, yet retries parameter works fine for me. I can see that when I changes retries number, retry attempts changes accordingly.

You could try to verify by turning on debug or http-debug options to see what's going on wire.

@cjyclaire
Copy link
Contributor

@NinoSkopac I noticed that you have mentioned 'async' behavior, different people might do this quite differently, so it would be great if you could provide some code snippets that might help us reproduce the issue : )

@NinoSkopac
Copy link
Contributor Author

@cjyclaire cjyclaire removed the response-requested Waiting on additional info and feedback. Will move to "closing-soon" in 7 days. label Mar 13, 2017
@imshashank imshashank added the guidance Question that needs advice or information. label Mar 13, 2017
@NinoSkopac
Copy link
Contributor Author

@cjyclaire Did you find out anything?

@imshashank
Copy link
Contributor

@NinoSkopac Sorry for the delay. Have you tried to contact the Polly team to increase your rate limit?

@imshashank imshashank added the response-requested Waiting on additional info and feedback. Will move to "closing-soon" in 7 days. label Mar 23, 2017
@NinoSkopac
Copy link
Contributor Author

Yeah, and they declined.

@imshashank
Copy link
Contributor

@NinoSkopac Sorry for the late reply. The PHP SDK use Guzzle library to make any requests. All async requests if failed are automatically retried.

Here is some documentation on the concept of Promise: https://docs.aws.amazon.com/aws-sdk-php/v3/guide/guide/promises.html

It will be a lot more helpful if you can share us what errors you are getting so that we can reproduce it.

@NinoSkopac
Copy link
Contributor Author

Hey @imshashank. I am familiar with Guzzle and Promises.

All async requests if failed are automatically retried.

Question: is their order preserved in case of a failed request?
Let's say I fire 10 async reqs, and request #5 fails - is it still going to be in 5th place once it's been retried and it succeeds?

It will be a lot more helpful if you can share us what errors you are getting so that we can reproduce it.

Sure. Keep in mind that these errors are from 23 days ago, when I wasn't capping the concurrent request count to 80 with CommandPool:

Promise rejection: exception 'Aws\Polly\Exception\PollyException' with message 'Error executing "SynthesizeSpeech" on "https://polly.eu-west-1.amazonaws.com/v1/speech"; AWS HTTP error: Server error: `POST https://polly.eu-west-1.amazonaws.com/v1/speech` resulted in a `503 Service Unavailable` response:

 Unable to parse error information from response - Error parsing JSON: Syntax error'

GuzzleHttp\Exception\ServerException: Server error: `POST https://polly.eu-west-1.amazonaws.com/v1/speech` resulted in a `503 Service Unavailable` response:

 in /var/www/read2me.online/vendor/guzzlehttp/guzzle/src/Exception/RequestException.php:111
Stack trace:
#0 /var/www/read2me.online/vendor/guzzlehttp/guzzle/src/Middleware.php(65): GuzzleHttp\Exception\RequestException::create(Object(GuzzleHttp\Psr7\Request), Object(GuzzleHttp\Psr7\Response))
#1 /var/www/read2me.online/vendor/guzzlehttp/promises/src/Promise.php(203): GuzzleHttp\Middleware::GuzzleHttp\{closure}(Object(GuzzleHttp\Psr7\Response))
#2 /var/www/read2me.online/vendor/guzzlehttp/promises/src/Promise.php(156): GuzzleHttp\Promise\Promise::callHandler(1, Object(GuzzleHttp\Psr7\Response), Array)
#3 /var/www/read2me.online/vendor/guzzlehttp/promises/src/TaskQueue.php(47): GuzzleHttp\Promise\Promise::GuzzleHttp\Promise\{closure}()
#4 /var/www/read2me.online/vendor/guzzlehttp/guzzle/src/Handler/CurlMultiHandler.php(96): GuzzleHttp\Promise\TaskQueue->run()
#5 /var/www/read2me.online/vendor/guzzlehttp/guzzle/src/Handler/CurlMultiHandler.php(123): GuzzleHttp\Handler\CurlMultiHandler->tick()
#6 /var/www/read2me.online/vendor/guzzlehttp/promises/src/Promise.php(246): GuzzleHttp\Handler\CurlMultiHandler->execute(true)
#7 /var/www/read2me.online/vendor/guzzlehttp/promises/src/Promise.php(223): GuzzleHttp\Promise\Promise->invokeWaitFn()
#8 /var/www/read2me.online/vendor/guzzlehttp/promises/src/Promise.php(267): GuzzleHttp\Promise\Promise->waitIfPending()
#9 /var/www/read2me.online/vendor/guzzlehttp/promises/src/Promise.php(225): GuzzleHttp\Promise\Promise->invokeWaitList()
#10 /var/www/read2me.online/vendor/guzzlehttp/promises/src/Promise.php(62): GuzzleHttp\Promise\Promise->waitIfPending()
#11 /var/www/read2me.online/vendor/guzzlehttp/promises/src/Promise.php(65): GuzzleHttp\Promise\Promise->wait(true)
#12 /var/www/read2me.online/vendor/guzzlehttp/promises/src/functions.php(201): GuzzleHttp\Promise\Promise->wait()
#13 /var/www/read2me.online/src/Read2Me/Aws.php(71): GuzzleHttp\Promise\unwrap(Array)
#14 /var/www/read2me.online/src/Read2Me/AudioConversion.php(199): Read2Me\Aws::pollySynthesize('Donald Trump co...')
#15 /var/www/read2me.online/src/Read2Me/AudioConversion.php(184): Read2Me\AudioConversion->createAudio()
#16 /var/www/read2me.online/bin/conversion.php(99): Read2Me\AudioConversion->getAudio()
#17 /var/www/read2me.online/bin/conversion.php(49): class@anonymous->audioConversion()
#18 /var/www/read2me.online/vendor/symfony/console/Command/Command.php(262): class@anonymous->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#19 /var/www/read2me.online/vendor/symfony/console/Application.php(826): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#20 /var/www/read2me.online/vendor/symfony/console/Application.php(189): Symfony\Component\Console\Application->doRunCommand(Object(class@anonymous), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#21 /var/www/read2me.online/vendor/symfony/console/Application.php(120): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#22 /var/www/read2me.online/bin/conversion.php(111): Symfony\Component\Console\Application->run()
#23 {main}

Next Aws\Polly\Exception\PollyException: Error executing "SynthesizeSpeech" on "https://polly.eu-west-1.amazonaws.com/v1/speech"; AWS HTTP error: Server error: `POST https://polly.eu-west-1.amazonaws.com/v1/speech` resulted in a `503 Service Unavailable` response:

 Unable to parse error information from response - Error parsing JSON: Syntax error in /var/www/read2me.online/vendor/aws/aws-sdk-php/src/WrappedHttpHandler.php:192
Stack trace:
#0 /var/www/read2me.online/vendor/aws/aws-sdk-php/src/WrappedHttpHandler.php(97): Aws\WrappedHttpHandler->parseError(Array, Object(GuzzleHttp\Psr7\Request), Object(Aws\Command), Array)
#1 /var/www/read2me.online/vendor/guzzlehttp/promises/src/Promise.php(203): Aws\WrappedHttpHandler->Aws\{closure}(Array)
#2 /var/www/read2me.online/vendor/guzzlehttp/promises/src/Promise.php(174): GuzzleHttp\Promise\Promise::callHandler(2, Array, Array)
#3 /var/www/read2me.online/vendor/guzzlehttp/promises/src/RejectedPromise.php(40): GuzzleHttp\Promise\Promise::GuzzleHttp\Promise\{closure}(Array)
#4 /var/www/read2me.online/vendor/guzzlehttp/promises/src/TaskQueue.php(47): GuzzleHttp\Promise\RejectedPromise::GuzzleHttp\Promise\{closure}()
#5 /var/www/read2me.online/vendor/guzzlehttp/guzzle/src/Handler/CurlMultiHandler.php(96): GuzzleHttp\Promise\TaskQueue->run()
#6 /var/www/read2me.online/vendor/guzzlehttp/guzzle/src/Handler/CurlMultiHandler.php(123): GuzzleHttp\Handler\CurlMultiHandler->tick()
#7 /var/www/read2me.online/vendor/guzzlehttp/promises/src/Promise.php(246): GuzzleHttp\Handler\CurlMultiHandler->execute(true)
#8 /var/www/read2me.online/vendor/guzzlehttp/promises/src/Promise.php(223): GuzzleHttp\Promise\Promise->invokeWaitFn()
#9 /var/www/read2me.online/vendor/guzzlehttp/promises/src/Promise.php(267): GuzzleHttp\Promise\Promise->waitIfPending()
#10 /var/www/read2me.online/vendor/guzzlehttp/promises/src/Promise.php(225): GuzzleHttp\Promise\Promise->invokeWaitList()
#11 /var/www/read2me.online/vendor/guzzlehttp/promises/src/Promise.php(62): GuzzleHttp\Promise\Promise->waitIfPending()
#12 /var/www/read2me.online/vendor/guzzlehttp/promises/src/Promise.php(65): GuzzleHttp\Promise\Promise->wait(true)
#13 /var/www/read2me.online/vendor/guzzlehttp/promises/src/functions.php(201): GuzzleHttp\Promise\Promise->wait()
#14 /var/www/read2me.online/src/Read2Me/Aws.php(71): GuzzleHttp\Promise\unwrap(Array)
#15 /var/www/read2me.online/src/Read2Me/AudioConversion.php(199): Read2Me\Aws::pollySynthesize('Donald Trump co...')
#16 /var/www/read2me.online/src/Read2Me/AudioConversion.php(184): Read2Me\AudioConversion->createAudio()
#17 /var/www/read2me.online/bin/conversion.php(99): Read2Me\AudioConversion->getAudio()
#18 /var/www/read2me.online/bin/conversion.php(49): class@anonymous->audioConversion()
#19 /var/www/read2me.online/vendor/symfony/console/Command/Command.php(262): class@anonymous->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#20 /var/www/read2me.online/vendor/symfony/console/Application.php(826): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#21 /var/www/read2me.online/vendor/symfony/console/Application.php(189): Symfony\Component\Console\Application->doRunCommand(Object(class@anonymous), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#22 /var/www/read2me.online/vendor/symfony/console/Application.php(120): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#23 /var/www/read2me.online/bin/conversion.php(111): Symfony\Component\Console\Application->run()
#24 {main}

@kstich kstich removed the response-requested Waiting on additional info and feedback. Will move to "closing-soon" in 7 days. label Apr 28, 2017
@imshashank
Copy link
Contributor

@NinoSkopac PHP SDK will use multi_curl for sending async requests. A new request is retried when a previous once fails and hence the order is preserved until it's scheduled using multi_curl.

Are you still facing this issue?

@NinoSkopac
Copy link
Contributor Author

@imshashank

I'm not facing the issue since I've migrated to CommandPool. Thanks for clarification about order. I'm gonna close this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
guidance Question that needs advice or information.
Projects
None yet
Development

No branches or pull requests

4 participants