# Traversing Large Datasets with PHP 5.5 Generators and yield

> The language-level approach to building large loops without blowing up memory: PHP 5.5 generators and the yield keyword.

- Published: 2014-12-07
- Category: Languages
- Tags: PHP
- Reading time: 4 min read
- Source: https://www.muhammetsafak.com.tr/en/blog/traversing-large-datasets-with-php-55-generators-and-yield/
- Language: en-US
- Author: Muhammet Şafak

---
This year was all about tools like [Laravel](/en/blog/getting-started-with-my-first-laravel-4-project), [Composer](/en/blog/switching-to-composer-stop-managing-php-dependencies-by-hand), and [Git](/en/blog/switching-to-git-leaving-ftp-behind) — but for this last post I wanted to zoom in on a language-level feature. I've been exploring the **generator** and `yield` keyword that shipped with PHP 5.5, and I want to share how useful it is when working with large datasets.

## The Problem: Large Arrays Blowing Up Memory

Say you have a CSV file with 500,000 rows and you need to process them. The naive approach is to load everything into an array:

```php
<?php
function csvSatirlariniOku($dosya)
{
    $satirlar = [];
    $f = fopen($dosya, 'r');
    while (($satir = fgetcsv($f)) !== false) {
        $satirlar[] = $satir;
    }
    fclose($f);
    return $satirlar;
}

$tumSatirlar = csvSatirlariniOku('buyuk.csv');
foreach ($tumSatirlar as $satir) {
    // işle
}
```

This code loads all 500,000 rows into memory at once. Fine for small files. On a large file, you'll hit the `memory_limit` wall.

I ran into this on a real project: a client's data import system where daily CSV files were reaching 300–400k rows. The script would die halfway through with `Fatal error: Allowed memory size exhausted`. Bumping `memory_limit` in php.ini looked like a fix, but it just kicked the can down the road — the file sizes kept growing.

## What Is a Generator?

A **generator** is a type of function that produces values one at a time, on demand, instead of computing them all at once. The difference from a regular function is that it uses `yield` instead of `return`. It yields a value, pauses, and resumes from where it left off on the next call.

This was added to the language in PHP 5.5.

## The Same Example with yield

```php
<?php
function csvSatirlariniOkuGenerator($dosya)
{
    $f = fopen($dosya, 'r');
    while (($satir = fgetcsv($f)) !== false) {
        yield $satir;
    }
    fclose($f);
}

foreach (csvSatirlariniOkuGenerator('buyuk.csv') as $satir) {
    // Only one row in memory at a time
}
```

It looks like a small change, but the runtime behavior is completely different. Each time `yield` executes, it hands the current `$satir` value out and pauses the function. When `foreach` asks for the next value, the function picks up right where it left off. Instead of 500,000 rows in memory, there is always exactly one.

## Number Range Example

A simple number-sequence generator to illustrate the point:

```php
<?php
function aralik($baslangic, $bitis)
{
    for ($i = $baslangic; $i <= $bitis; $i++) {
        yield $i;
    }
}

foreach (aralik(1, 1000000) as $sayi) {
    echo $sayi . "\n";
}
```

PHP's built-in `range(1, 1000000)` creates an array of one million elements and loads them all into memory. The generator version holds only the current number at each step.

## The Generator Object

Calling a generator function does not execute it immediately — it returns a `Generator` object. You can iterate over it with `foreach`, or control it manually:

```php
<?php
function ureticim()
{
    yield 'birinci';
    yield 'ikinci';
    yield 'ucuncu';
}

$uretec = ureticim();
echo $uretec->current(); // birinci
$uretec->next();
echo $uretec->current(); // ikinci
```

`current()` returns the current value; `next()` advances to the next step. In practice, this level of manual control is rarely needed — `foreach` handles it in almost every case.

## Key-Value Pairs with yield

You can yield key-value pairs just like an associative array:

```php
<?php
function veriUret()
{
    yield 'ad'    => 'Ahmet';
    yield 'email' => 'ahmet@ornek.com';
    yield 'yas'   => 30;
}

foreach (veriUret() as $anahtar => $deger) {
    echo "{$anahtar}: {$deger}\n";
}
```

## Watch Out: Generators Cannot Be Rewound

You can only traverse a generator once. Once the `foreach` loop finishes, trying to loop over the same generator object again will produce nothing — the values are not re-generated. If you need to process the same data twice, you have to call the function again to get a fresh generator object. This is a fundamental difference from regular arrays, and forgetting it can cause silent, incorrect behavior.

## When Should You Use It?

Using generators everywhere is overkill. If the dataset is small and fits comfortably in memory, a regular array is simpler.

Generators earn their keep when:
- Processing large files line by line
- [Fetching a large number of database records](/en/blog/simplifying-database-operations-with-eloquent-orm) when you don't need them all at once
- Working with an infinite or very large sequence (API pagination, streaming data)

I learned this feature because I had a concrete need: a daily CSV import in one project kept hitting the memory limit. Generators solved it. Now, whenever I'm about to write a loop over a large dataset, this option is always somewhere in the back of my mind.

As the last post of 2014, I wanted to pick a slightly different topic. Next year I'll be back to covering the new version of Laravel.