Some way to bulk dispatch events after queued job finish executes

[sts-ryan-holton]

Hi 👋

I'm not sure whether this functionality already exists, but I think it would be quite useful. My Laravel application contains a lot of queued jobs. Jobs by definition are essentially a way to run intensive logic at a later point in time without affecting the request lifecycle.

My job though contains various queued event listeners that are dispatched from it, each containing different types of data, I'll attach a slim version of what I mean.

Rather than actually dispatching these events at the point in time, it would be great if Laravel could essentially mock the dispatch in someway, obtain it's data in memory and then somehow dispatch them after the job finishes?

Here's a slimed version of my job illustrating what I mean by dispatching events from the job

<?php

namespace App\Jobs;

use App\Events\Reports\BuyerTierSummaryStatCreation;
use Illuminate\Bus\Queueable;
use Illuminate\Database\Eloquent\Collection;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Queue\SerializesModels;
use Throwable;

class ProcessApplication implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

    /**
     * The number of times the job may be attempted.
     *
     * @var int
     */
    public $tries = 1;

    /**
     * The number of seconds the job can run before timing out.
     *
     * @var int
     */
    public $timeout = 700;

    /**
     * The application's thread ID
     *
     * @return string
     */
    public $threadID;

    /**
     * Syncable or not
     */
    public bool $syncable = false;

    /**
     * Create a new job instance.
     *
     * @return void
     */
    public function __construct(string $threadID, bool $syncable = false)
    {
        $this->onConnection('redis-long-running');
        $this->onQueue('applications');

        $this->threadID = $threadID;
        $this->syncable = $syncable;
    }

    /**
     * Get the tags that should be assigned to the job.
     *
     * @return array<int, string>
     */
    public function tags(): array
    {
        return ['applications'];
    }

    /**
     * Execute the job.
     */
    public function handle(): void
    {
        BuyerTierSummaryStatCreation::dispatch(
            Carbon::parse($endedAt),
            'total_submits',
            $pingtreeEntry->buyerTier->user_id,
            $pingtreeEntry->buyerTier->company_id,
            $application->affiliate_id,
            $pingtreeEntry->buyerTier->buyer_id,
            $pingtreeEntry->buyerTier->id,
            $application->product_id,
        );

        $count = 0;

        while ($count < 10) {
            BuyerTierSummaryStatCreation::dispatch(
                Carbon::parse($endedAt),
                'total_submits',
                $pingtreeEntry->buyerTier->user_id,
                $pingtreeEntry->buyerTier->company_id,
                $application->affiliate_id,
                $pingtreeEntry->buyerTier->buyer_id,
                $pingtreeEntry->buyerTier->id,
                $application->product_id,
            );

            $count++;
        }

        BuyerTierSummaryStatCreation::dispatch(
            Carbon::parse($endedAt),
            'total_submits',
            $pingtreeEntry->buyerTier->user_id,
            $pingtreeEntry->buyerTier->company_id,
            $application->affiliate_id,
            $pingtreeEntry->buyerTier->buyer_id,
            $pingtreeEntry->buyerTier->id,
            $application->product_id,
        );
    }

    /**
     * Handle a job failure.
     */
    public function failed(?Throwable $e)
    {
        // ...
    }
}

Maybe some new method or some new job functionality that holds things until job completion and then dispatches them?

Why am I proposing this?

Simply, my application is dispatching a tonne of events, and without refactoring the whole thing, I'd like to speed up the class execution and dispatch at the very end!

Would be great to get other thoughts on this!

[henzeb]

Laravel 11 has https://laravel.com/docs/11.x/helpers#deferred-functions

[henzeb]

Well, it will never be in one go. As there will be a loop through an array somewhere down the plumbing. Using defer it will get dispatched after the job has finished.

[sts-ryan-holton]

I'm looking for a solution that sort of holds N jobs in memory, and then after the job completes, or fails, then dispatch them

[henzeb]

That's kind of what defer can do for you...

[sts-ryan-holton]

But it processes it straight away, my Job runs for around 5 minutes, and currently dispatches a few hundred events throughout that timeframe. I want to defer some of these to after the job processes, and then bulk dispatch events.

From what I gather 'defer' just does it similar to how the queues do it

[henzeb]

If you mean dispatchAfterResponse, yes.

https://laravel.com/docs/11.x/queues#dispatching-after-the-response-is-sent-to-browser