Using Queues and Jobs to postpone Tasks in Laravel

- Andrés Cruz

En español
Using Queues and Jobs to postpone Tasks in Laravel

In this post we are going to learn about the use of Queues and Jobs in Laravel (Queues and Jobs) to be able to perform jobs in the background.

The idea of this API is that we can postpone tasks (leave them in the background) that require a high computational consumption and all this is to improve the user experience and avoid application crashes; in such a way that these high computing tasks will be executed in another thread or process in our server and will be resolved as they arrive or depending on their priority.

Basically all this process is carried out between the tasks/jobs (the high computational processes) and the queues (mechanism to register the jobs).

In the same way, we are going to talk a little more in detail about these two fundamental components.

The jobs and the queues

Jobs are the heavy or processing-intensive task that we want to defer; and they can be of any type such as massive or simple sending of emails, processing images, videos, etc; therefore these are tasks that we register as pending so that Laravel processes them one by one as it becomes available; so the next thing you can ask yourself is, how is this organization managed?; that is, who is in charge of managing the priority between tasks or jobs (queues) and how do we maintain an organization of all this.

The jobs

In Laravel, jobs are an important part of the framework's job queue functionality, that is, jobs are used to process the heavy jobs stored by “queues”; these heavy tasks can be anything like sending emails, processing images, videos, among others; this is ideal since these tasks are not part of the user's main session and it avoids consuming resources from the user's main session, apart from avoiding the famous "not responding" windows and in general, you have a better experience in using the app as it does not affect the performance or responsiveness of the main app.

Jobs in Laravel can be created to perform whatever task is needed in the back-end without the user having to wait for the task to complete; that is, how to send an email or export data in an excel or a similar format.

The great thing about jobs is that they are executed sequentially, suppose you create a job to process emails, the emails are sent one by one without the need to overload the page, which would not happen if the email delivery was handled from the user's main session and suddenly, 5 or more users come to send emails from the application, therefore, at the same moment of time, you would have multiple emails and spend the resources that this requires.

In short, jobs in Laravel are an essential part of the framework's job queue, allowing heavy or non-essential tasks to be performed in the background to improve the performance and responsiveness of the main application.

The queues

Already at this point, we have introduced the concept of "queues" in the use of jobs, in the same way, we are going to explain it quickly; the job queues correspond to the operation that you want to perform, which, as we mentioned before, corresponds to the "heavy" operation that you want to perform, that is, instead of sending an email from a controller, it is stored in a queue that is then It is processed by another process.

Create your own queues in Laravel to maintain order

The answer to what was commented above is quite simple, it would be the queues as a mechanism that allows registering or adding these works or jobs; we can register or have as many queues as we want and we can specify to Laravel via artisan which queues we want to process first; for example, let's look at the following example:

php artisan queue:work --queue=high,default

Through the previous command we are telling Laravel to first process the jobs from a queue called "hight" and then from a queue called "default", which by the way, is the queue that we have defined by default.

Connections for the structure to maintain the tails

The connections are the default way we have to indicate how the entire process of the queues is going to be carried out (since it remembers that the queue absorbs the task, therefore, at this point, once the task or job is registered, in a queue, in our queue the task no longer paints anything); but to be able to use all this structure we must indicate the connection and with it we must indicate:

  1. The controller or driver
  2. Default or Configuration Values

Therefore, through the connections we can specify the driver that we are going to use, which can be any such as the database or other services; for example:

Database:

  1. Amazon SQS: aws/aws-sdk-php ~3.0
  2. Beanstalkd: pda/pheanstalk ~4.0
  3. Redis: predis/predis ~1.0 or phpredis PHP extension

And some extra parameters for configuration.

Configure the Driver or connector of the queues for the database

Now we are going to open the config/queue.php file and the .env, in which we are going to look for a configuration called QUEUE_CONNECTION

Which by default is called QUEUE_CONNECTION and has the sync configuration, which if you analyze it a bit you will see that we have several types of connections that we can use, including the database call that is the one we are going to use, therefore configure it at least in the .env file; in my case I'm going to leave it like this in the .env:

QUEUE_CONNECTION=database
Y en el config/queue.php quede:
    'default' => env('QUEUE_CONNECTION', 'database'),

Create the jobs table in our database

Now the next thing you can ask would be, how can we configure the table in our database?; following the official documentation:

php artisan queue:table php artisan migrate

We already have an artisan command that helps us in this whole process, which would be the one that allows us to generate the migration to create the table; and with this we are ready.

Creating our first Job in Laravel

Now we are going to create our first Job that we are going to configure with the default queue; for that, we are going to create one using our artisan:

php artisan make:job ProcessImageSmall

And with this we have a file generated in App\Jobs

Which consists of two main blocks:

  1. __construct
  2. handle

The constructor function to initialize the basic data that we are going to use and the handle function to do the heavy processing; It's that simple, therefore in the handle function we can send emails, process videos or images, for our example we are going to assume that we have an image that we are receiving by parameters:

  protected $image;
 
    /**
     * Create a new job instance.
     *
     * @return void
     */
    public function __construct(PostImage $image)
    {
        $this->image = $image;
    }

This image is basically an instance of a model that looks like this:

class PostImage extends Model
{
    protected $fillable = ['post_id', 'image'];
 
    public function post(){
        return $this->belongsTo(Post::class);
    }
 
    public function getImageUrl(){
        return URL::asset('images/'.$this->image);
        //return Storage::url($this->image);
    }
}

And now, we are going to do a heavy process like scaling an image; for this, we are going to use the following package that we can install using composer:

composer require intervention/image

The code that we are going to use is basically self-explanatory:

<?php
 
namespace App\Jobs;
 
use App\PostImage;
use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Queue\SerializesModels;
 
use Intervention\Image\ImageManagerStatic;
 
class ProcessImageSmall implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;
 
    protected $image;
 
    /**
     * Create a new job instance.
     *
     * @return void
     */
    public function __construct(PostImage $image)
    {
        $this->image = $image;
    }
 
    /**
     * Execute the job.
     *
     * @return void
     */
    public function handle()
    {
        $image = $this->image;
 
        $fullPath = public_path('images' . DIRECTORY_SEPARATOR . $image->image);
        $fullPathSmall = public_path('images' . DIRECTORY_SEPARATOR . 'small' . DIRECTORY_SEPARATOR . $image->image);
 
        $img = ImageManagerStatic::make($fullPath)->resize(300,200);
        $img->save($fullPathSmall);
    }
}

In which we simply create an instance of our package, which we installed earlier, specify the location of the image and using the resize function scale the image to the specified dimensions; then we save the new image and with this we are ready.

Dispatch a job

The next thing we have to specify is where we are going to call this job, which can be anywhere in our application, such as when we load an image, etc; suppose we have a method to load an image in Laravel for example:

  public function image(Request $request, Post $post)
    {
        $request->validate([
            'image' => 'required|mimes:jpeg,bmp,png|max:10240', //10Mb
        ]);
 
        $filename = time() . "." . $request->image->extension();
 
        $request->image->move(public_path('images'), $filename);
 
        //$path = $request->image->store('public/images');
 
        $image = PostImage::create(['image' => /*$path*/ $filename, 'post_id' => $post->id]);
 
        ProcessImageSmall::dispatch($image);
 
        return back()->with('status', 'Imagen cargada con exito');
 
    }

From here; If you look closely at the code, you'll see that we define an instance of our queue with the dispatch method to dispatch the job ProcessImageSmall::dispatch($image)

Raise daemon to process queues

Now if we review the previous table, we will see that a job was registered, but it has not been dispatched, to dispatch it, we have to activate the daemon that is in charge of doing this job, start the process that is in charge of doing the jobs pending in the jobs table:

php artisan queue:work

And with this we will see that we have everything ready and the job has already been dispatched:

Despachar trabajo

With this, we learned how to use jobs and queues in Laravel; remember that this is one of the many topics that we deal with in depth in our Laravel course that you can take on this platform in the courses section.

Andrés Cruz

Desarrollo con Laravel, Django, Flask, CodeIgniter, HTML5, CSS3, MySQL, JavaScript, Vue, Android, iOS, Flutter

Andrés Cruz en Udemy

Acepto recibir anuncios de interes sobre este Blog.