Efficiently handling large Eloquent results in Laravel

We sometimes need to handle large results of models when working with Eloquent. If we use the regular method of retrieving the full result set and working with the models, then it’s going to overflow our memory – essentially failing the entire process.

In order to bring efficiency to that process, we can lazy load the models in chunks and process them. Laravel provides two methods for that: lazy() and lazyById().

Consider a scenario where in a console command, we need to update a course’s average rating (example from Laravel Courses). Here’s how we can do that using the lazy() method, where courses will be fetched in chunks, and we can work with each of the retrieved models individually:

use App\Models\Course;
 
foreach (Course::lazy() as $course) {
$course->updateAverageRating();
}

Consider another use case where we want to perform bulk updates to a model but want to do that in chunks. Let’s see how that can be achieved:

use App\Models\Invoice;
 
Invoice::where('status', 'pending')
->lazyById(100)
->each->update(['status' => 'abandoned']);

The official documentation also has some good examples you can look into as well.

Structuring Laravel Eloquent Model with Custom Accessors and Mutators

This is NOT a recommended way to work with Laravel. At the time of writing this, strong typing and static analysis tools were not available – with their wide usage now, there is no need to use hacks like this anymore.

In a few recent projects, we have experimented with a slight alteration of Eloquent Models and have found some great value. I thought to share this in case someone else benefits the same way we did 🚀

Let’s first bring a Product model to life for an imaginary e-commerce store:

php artisan make:model Product

We now have our brand new Product model under the App namespace:

<?php
 
namespace App;
 
use Illuminate\Database\Eloquent\Model;
 
class Product extends Model
{
//
}

We do some initial setup at the beginning:

  • Enforcing strict types
  • Marking class as final
  • Adding logical code sections

…resulting in this:

<?php declare(strict_types=1);
 
namespace App;
 
use Illuminate\Database\Eloquent\Model;
 
final class Product extends Model
{
#-----------------------------------------------------------------
# Class Constants
#-----------------------------------------------------------------
 
//
 
#-----------------------------------------------------------------
# Class Properties
#-----------------------------------------------------------------
 
//
 
#-----------------------------------------------------------------
# Relationships
#-----------------------------------------------------------------
 
//
 
#-----------------------------------------------------------------
# Accessors and Mutators
#-----------------------------------------------------------------
 
//
}

Next, we adhere to some general team conventions regarding table and primary key name and include the necessary relations:

<?php declare(strict_types=1);
 
// ...
 
final class Product extends Model
{
// ...
 
#-----------------------------------------------------------------
# Properties
#-----------------------------------------------------------------
 
/**
* @var string
*/
protected $table = 'products';
 
/**
* @var string
*/
protected $primaryKey = 'product_id';
 
#-----------------------------------------------------------------
# Relationships
#-----------------------------------------------------------------
 
/**
* @return HasOne
*/
public function currency(): HasOne
{
return $this->hasOne(Currency::class, 'currencyCode', 'currency');
}
 
// ...
}

You may be wondering what’s new here, this is standard Laravel – you’re entirely correct. I’ve included these details to make sure someone new can also follow along 😌

At this stage, we introduce our own style accessors and mutators to the model (also known as getters/setters):

<?php declare(strict_types=1);
 
// ...
 
final class Product extends Model
{
// ...
 
#-----------------------------------------------------------------
# Accessors and Mutators
#-----------------------------------------------------------------
 
/**
* @return string
*/
public function getName(): string
{
return $this->getAttribute('name');
}
 
/**
* @param string $name
*/
public function setName(string $name): void
{
$this->setAttribute('name', $name);
}
 
/**
* @return float|null
*/
public function getPrice(): ?float
{
return $this->getAttribute('price');
}
 
/**
* @param float|null $value
*/
public function setPrice(?float $value): void
{
$this->setAttribute('price', $value);
}
 
/**
* @return bool
*/
public function isActive(): bool
{
return $this->getAttribute('is_active');
}
 
// ...
}

That’s pretty straightforward, but you may be wondering about the verbosity and why someone would write all these methods by hand.

What’s the benefit?

Well, we have seen these benefits since introducing this overall structure:

  • Model is divided into logical sections, so easy to navigate
  • Model properties are available through IDE auto-completion
  • Model properties are type-hinted, thus using the power of strict type
  • It still preserves the power of Eloquent, so no hacky overrides

In our team, we have seen an overall improvement in DX (developer experience) as a result of these small changes. There are fewer errors, code comprehension has improved and cognitive load has reduced while working with someone else’s code.

Why not just use public properties?

One question that comes up naturally when I present this to anyone: why not just use the public properties? Well, in my opinion, there are a few limitations:

  1. The data type of a given property cannot be enforced, you can set a float value to a property that is defined to store it in the DB
  2. The return type of a given property cannot be determined, so on a language level, you don’t know if a status field is supposed to return a string or int
  3. Property level data validation is difficult, you can set a negative value to a property that is clearly invalid (ex: negative price of a product)
  4. If a property column name is changed, that change needs to be done manually in all places where the property was accessed; as the properties are dynamic, thus cannot make use of IDE usage search

Closing Thoughts

You may want to give this a try and see if this benefits you in any way. This heavily favors the strong typing of PHP 7 so teams who has a preference towards that would get the most out of this. And lastly, your mileage may vary 🙂

Share your ideas and thoughts with me on Twitter!

Installing Symfony 4 (3.4.0-BETA1)

I wanted to install the Symfony 4 beta to play with the new features. While I was able to get most of it working, but was facing issues due to package dependency when trying to install some common bundles. I then settled with the 3.4 branch, which has the exact same functionalities except for some backward compatibility.

Here are the steps to get the framework up & running with the most used bundles:

1composer create-project symfony/skeleton symfony "3.4.0-BETA1"
2cd symfony/
3composer req web-server "3.4.0-BETA1"
4composer req annotations
5composer req orm
6composer req annotations
7composer req doctrine-migrations
8composer req debug "3.4.0-BETA1"
9composer req log
10composer req profiler
11composer req security "3.4.0-BETA1"
12composer req mailer
13composer req validation "3.4.0-BETA1"
14composer req encore
15composer req twig-extensions
16composer req serialization "3.4.0-BETA1"
17composer require friendsofsymfony/rest-bundle
18composer require stof/doctrine-extensions-bundle

Once the installation is done, you can start the application using the web server and it should be running.

Concurrent HTTP requests in PHP using pecl_http

The pecl_http extension has a little gem that can be handy at times – HttpRequestPool. Using this, you can send concurrent HTTP requests and can gain efficiency in fetching non-related data at once. For example, from an external source if your application needs to retrieve an user’s profile, their order history and current balance, you can send parallel requests to the API and get everything together.

Here is a simple example code to illustrate that:

1<?php
2 
3$endpoint = "http://api.someservice.com";
4$userId = 101;
5 
6$urls = array(
7 $endpoint . '/profile/' . $userId,
8 $endpoint . '/orderHistory/' . $userId,
9 $endpoint . '/currentBalance/' . $userId
10);
11 
12$pool = new HttpRequestPool;
13 
14foreach ($urls as $url) {
15 $req = new HttpRequest($url, HTTP_METH_GET);
16 $pool->attach($req);
17}
18 
19// send all the requests. control is back to the script once
20// all the requests are complete or timed out
21$pool->send();
22 
23foreach ($pool as $request) {
24 echo $request->getUrl(), PHP_EOL;
25 echo $request->getResponseBody(), PHP_EOL . PHP_EOL;
26}