How to Measure and Optimize PHP Memory Usage When Reading Large Files

Testing Success Criteria

The only way to ensure our code is optimized is to measure bad situations and compare the results after applying fixes. Two key metrics are CPU usage (how fast or slow the program runs) and memory usage (how much RAM a PHP script consumes). These metrics are often inversely related.

In asynchronous models such as multi‑process or multi‑threaded PHP applications, CPU and memory become critical considerations, especially when the server reaches its limits.

Real‑time CPU measurement is impractical in PHP; you can use top -like commands on Linux (CentOS, Ubuntu, macOS) or switch to Linux if you are on Windows.

What We Want

We want to measure memory consumption of traditional PHP scripts, apply optimization strategies, and make an informed decision.

Functions used to inspect memory usage:

// formatBytes is taken from the php.net documentation
memory_get_peak_usage();
function formatBytes($bytes, $precision = 2) {
    $units = array("b", "kb", "mb", "gb", "tb");
    $bytes = max($bytes, 0);
    $pow = floor(($bytes ? log($bytes) : 0) / log(1024));
    $pow = min($pow, count($units) - 1);
    $bytes /= (1 << (10 * $pow));
    return round($bytes, $precision) . " " . $units[$pow];
}

These functions will be called after normal script execution to identify the most memory‑hungry scripts.

Line‑by‑Line Reading

Two scenarios are considered: processing each line while reading, or simply streaming data without loading it all into memory.

Example of reading a file into an array:

function readTheFile($path) {
    $lines = [];
    $handle = fopen($path, "r");
    while (!feof($handle)) {
        $lines[] = trim(fgets($handle));
    }
    fclose($handle);
    return $lines;
}
readTheFile("shakespeare.txt");
require "memory.php";

Using a generator to read line by line reduces peak memory dramatically:

function readTheFile($path) {
    $handle = fopen($path, "r");
    while (!feof($handle)) {
        yield trim(fgets($handle));
    }
    fclose($handle);
}
readTheFile("shakespeare.txt");
require "memory.php";

Reading the 5.5 MB Shakespeare text with the generator peaks at only 393 KB, compared to 12.8 MB with the array approach.

Further processing can split the file into chunks and operate on a buffer:

$iterator = readTheFile("shakespeare.txt");
$buffer = "";
foreach ($iterator as $iteration) {
    if (preg_match("/
{3}/", $buffer, $matches)) {
        print ".";
        $buffer = "";
    } else {
        $buffer .= $iteration . PHP_EOL;
    }
}
require "memory.php";

Even when dividing the text into 1 216 blocks, memory usage stays under 459 KB, demonstrating the efficiency of generators.

Pipe Operations

Sometimes we only need to transfer data from one file to another without processing it. Using streams (pipes) avoids loading the whole file into memory.

// Simple copy (high memory usage)
file_put_contents("piping-files-1.txt", file_get_contents("shakespeare.txt"));
require "memory.php";

Using stream_copy_to_stream streams the data directly:

$handle1 = fopen("shakespeare.txt", "r");
$handle2 = fopen("piping-files-2.txt", "w");
stream_copy_to_stream($handle1, $handle2);
fclose($handle1);
fclose($handle2);
require "memory.php";

This approach keeps memory usage low even for large files.