Efficiently Process Massive PHP Files with SplFileObject and Generators

In this article we learn how to handle large files in PHP, avoiding memory limits and improving user experience.

Standard functions like file_get_contents() and file() load the entire file into memory, which is constrained by memory_limit. Processing a 20 GB file with these functions is impractical.

Two main issues arise: memory limit and poor user experience due to long waits or blank pages.

Using the yield keyword together with the SplFileObject class solves these problems.

BigFile class

The class accepts a filename and uses SplFileObject for efficient reading.

class BigFile {
    protected $file;

    public function __construct($filename, $mode = "r") {
        if (!file_exists($filename)) {
            throw new Exception("File not found");
        }
        $this->file = new SplFileObject($filename, $mode);
    }

    protected function iterateText() {
        $count = 0;
        while (!$this->file->eof()) {
            yield $this->file->fgets();
            $count++;
        }
        return $count;
    }

    protected function iterateBinary($bytes) {
        $count = 0;
        while (!$this->file->eof()) {
            yield $this->file->fread($bytes);
            $count++;
        }
    }

    public function iterate($type = "Text", $bytes = NULL) {
        if ($type == "Text") {
            return new NoRewindIterator($this->iterateText());
        } else {
            return new NoRewindIterator($this->iterateBinary($bytes));
        }
    }
}

The class provides two iteration methods: iterateText() uses fgets() for line‑by‑line reading of text files, while iterateBinary() uses fread() for binary data.

Both methods are wrapped by iterate(), which returns a NoRewindIterator to enforce forward‑only reading.

Usage example

$largeFile = new BigFile("file.csv");
$iterator = $largeFile->iterate("Text"); // or "Binary"
foreach ($iterator as $line) {
    echo $line;
}

This approach can read files of any size without hitting memory limits, making it suitable for extremely large datasets. The class can be added to a Laravel project via Composer autoloading.

Now you can easily parse and process massive files in PHP.