Tasks constantly getting stuck – Possible issue with task queue?

[jsunsawyer]

Description

I am having issues with Deleting Stale Template Caches tasks getting stuck. I ran an Update asset indexes task on a single asset volume and noticed that tasks were regularly getting stuck. I checked my max_execution_time and it was set to 30s, so I bumped it up, cleared my queue and re-ran my task, and all was good.

What I don't understand is why 30s is not long enough. Does my max_execution_time apply to the entire queue or each task individually? I am thinking that it must be the entire queue. But what if a lot of stuff gets queued up? What should my max_execution_time be set to to make sure tasks never get stuck? Shouldn't each task reset the max_execution_time?

Additional info

• Craft version: 3.0.11
• PHP version: 7.0.30
• Database driver & version: MySQL 5.7.22

[brandonkelly]

Yes, all jobs are executed in a single request, because that’s how the Yii2 Queue Extension (and queue workers in general) works.

However you that request doesn’t have to be a web request, where PHP is generally configured with a strict max execution time and memory limit. You could run the queue from the console instead by setting the runQueueAutomatically config setting to false, and then running this from your terminal:

./craft queue/run

Or have the console continue looking for new jobs, even after it has finished executing any existing jobs, by running this:

./craft queue/listen

You could even set up a daemon that runs queue/listen automatically when the system boots up. Details about how to do that can be found here: https://github.com/yiisoft/yii2-queue/blob/master/docs/guide/worker.md

[jsunsawyer]

I feel like this is an issue that people will always run into without a custom server setup and some sort of fix should be implemented. Or at least a detailed recommendation explained somewhere in the docs to help people avoid running into this issue.

Keep in mind that I know basically nothing about Yii and very little about PHP, but would doing something like what is suggested in this help avoid the issue? https://stackoverflow.com/questions/26398750/yii-background-process-issue?rq=1

I just don't understand why, when the server is running fine and acceptable tasks are running, that an arbitrary setting halts tasks at a certain point.

[brandonkelly]

Craft 2, and Craft3 by default, basically have a “poor-man’s queue”, where the queue is run by a web-based worker rather than a daemon or dedicated queue service like Amazon SQS. We went this route because we realize that most people don’t want to deal with setting up a more traditional queue worker. But this approach has its limits (because it’s run as a normal web request), which is why in Craft 3 we switched to the Yii2 Queue Extension, which at least has the option to be run via the terminal, daemon, or even can be swapped out with a different driver that uses something like SQS (see https://github.com/yiisoft/yii2-queue/tree/master/docs/guide).

I just added a Queue Component section to the Configuration docs. Hopefully that helps a little.

would doing something like what is suggested in this help avoid the issue? https://stackoverflow.com/questions/26398750/yii-background-process-issue?rq=1

Craft 2 did call set_time_limit(0) (which attempts to disable PHP’s max execution time) before running the queue, but apparently we forgot to do the same in Craft 3, so I just added it. Yes it will help, as long as PHP isn’t running as a FastCGI process.

[jsunsawyer]

In case it helps anyone else...

Queue Component documentation now lives here: https://docs.craftcms.com/v3/config/app.html#queue-component

After struggling with failed tasks on another project I found that my main issue was with the queue's ttr being set to 300, so any of my queue tasks that took longer than this would fail at that point.

This is configurable in the queue component in app.php.
See: #3147

[khalwat]

For anyone else finding this, this article may help: Robust queue job handling in Craft CMS