How to chunk results from a custom query in Laravel?

In Laravel, the chunk() method is essential for processing large datasets efficiently. It retrieves data in smaller batches, preventing memory exhaustion when dealing with thousands of records. The method works with both DB facade and Eloquent models.

Using DB Facade

The chunk() method fetches records in specified batch sizes and processes them within a closure

<?php
namespace App\Http\Controllers;
use Illuminate\Http\Request;
use Illuminate\Support\Facades\DB;

class UserController extends Controller {
    public function index() {
        DB::table('users')->orderBy('id')->chunk(100, function ($users) {
            foreach ($users as $user) {
                echo $user->id."=>".$user->name." ";
            }
        });
    }
}
?>

This example processes 100 records at a time until all records are fetched and displayed.

Stopping Chunk Processing

You can halt the chunking process by returning false from the closure

<?php
namespace App\Http\Controllers;
use Illuminate\Http\Request;
use Illuminate\Support\Facades\DB;

class UserController extends Controller {
    public function index() {
        DB::table('users')->orderBy('id')->chunk(100, function ($users) {
            foreach ($users as $user) {
                echo $user->id."=>".$user->name." ";
                if ($user->id == 150) {
                    return false; // Stops further processing
                }
            }
        });
    }
}
?>

Using Eloquent Models

The chunk() method works seamlessly with Eloquent models

<?php
namespace App\Http\Controllers;
use Illuminate\Http\Request;
use App\Models\User;

class UserController extends Controller {
    public function index() {
        User::orderBy('id')->chunk(100, function ($users) {
            foreach ($users as $user) {
                echo $user->id."=>".$user->name." ";
            }
        });
    }
}
?>

Collection Chunking

For already loaded collections, use the collection's chunk() method

<?php
namespace App\Http\Controllers;
use Illuminate\Http\Request;
use App\Models\User;

class UserController extends Controller {
    public function index() {
        $users = User::all();
        $chunkedUsers = $users->chunk(10);
        
        foreach ($chunkedUsers as $records) {
            foreach($records as $user) {
                echo $user->id."=>".$user->name." ";
            }
        }
    }
}
?>

Note: This approach loads all records into memory first, which defeats the purpose of chunking for large datasets.

Conclusion

Use Laravel's chunk() method to process large datasets efficiently without memory issues. The DB facade and Eloquent model approaches are ideal for database chunking, while collection chunking works best for smaller, already-loaded datasets.