[10.x] Add resume method to batched jobs as an option

[bmckay959]

With the new Job Chain/Batching feature (#48633), I am moving my custom implementation over to the "Laravel Way".

Currently, you can do the following with a Job Batch:

1. Cancel the batch and all remaining jobs get cancelled
2. Allow failures

Without being inside of a Chain, this was good enough for my use case because I would allow failures, but then only move to the next 'step' after all failed jobs were remediated. Now that I can take advantage of the Job Chain with Batches inside, I need to be able to cancel a Batch on failure....fix the issue, and restart the jobs in the batch that failed.

My Use Case:
Batch 1 has 100 jobs. All 100 jobs must be completed(not failed) before going to Batch 2.
Batch 2 has 30 jobs. All 30 jobs are dependent upon the data received in the Batch 1 jobs.

What I Am Currently Doing:
Batch 1 created a unique lock based on an ID. Once Batch 1 completed, an event was sent to remove the 'lock'
Batch 2 would check if that lock still existed. If so, return the job to the queue and try again.

I need to be able to have a job inside of Batch 1 fail and cancel the batch(Therefore, cancelling the Chain it is in). After I remedy the failure, I need to be able to run queue:retry-batch {id}.

The problem: Retrying the batch does not "un-cancel" the batch, meaning that inside of the job when checking if cancelled, it doesn't perform the actions.

/**
 * Execute the job.
 */
public function handle(): void
{ 
    if ($this->batch()->cancelled()) {
        return;
    }

    // Business logic runs here...
}

The solution: Retrying the batch has an option to --resume which "un-cancels" the batch, allowing the business logic to be ran.

queue:retry-batch {id} --resume

---

Scenario(End Goal):

$batch1 = [
   new Job(1),
   new Job(2),
   new Job(3),
];

$batch2 = [
   new OtherJob(1),
   new OtherJob(2),
   new OtherJob(3),
];

Bus::chain([
    Bus::batch($batch1),
    Bus::batch($batch2),
])->dispatch();

Job(1) successfully runs.
Job(2) fails and cancels the batch.
// The chain has now stopped.
// Fix failure reason manually and run queue:retry-batch {id} --resume
Job(2) successfully runs now.
Job(3) successfully runs.
$batch2 now runs

[bmckay959]

@khepin Would love to get your thoughts here if possible. Your PR is what triggered this move for me, and honestly, I'm not near a good enough developer to be making PRs here. :)

[khepin]

I don't think there's much that can be done to actually resume a batch. What's being proposed here isn't a resume but a skip method it seems to me.

In the example you gave, I would call this resume if it were to re-run Job(2) or maybe omit Job(2) but run the remaining jobs in the batch like Job(3) here.

This unfortunately isn't possible with most queue drivers as the jobs won't be persisted.

That's one of the issues.

The second issue is that simply clearing a job's status does not restart it, so currently nothing happens after you've cleared the status. You'd have to trigger the job's then callbacks manually.

---

To truly accomplish what you're after, we'd need to fully persist the chains and batches so they could be restarted.
What we have done at my work is persist a status object that allows us to not repeat steps that were already finished. That way we can restart the entire workflow and the first few steps might just get skipped.

Otherwise I'd suggest you look into solutions like https://github.com/laravel-workflow/laravel-workflow or https://temporal.io/
Those do persist everything and allow you to restart from anywhere. But they're a slightly different model where you can't express an entire workflow from the start like Laravel offers.

[bmckay959]

Hmmm...I'm going to convert this to a draft while I dig into this.

I hear what you are saying. I thought that it would operate the same as a failed job retry since it's being triggered by the batch-retry command.

Thanks for the feedback.

[driesvints]

Feel free to resend this once you have time.